Training method of fake face detection model, fake face detection method and electronic device

Abstract

A counterfeit face detection model training method, a counterfeit face detection method, and an electronic device are disclosed. The counterfeit face detection method includes obtaining a to-be-detected video including face information; obtaining a feature vector corresponding to the to-be-detected video by performing feature extraction on the face information included in the to-be-detected video; obtaining a counterfeit face detection model trained by a counterfeit face detection model training method; and obtaining a prediction probability of the feature vector corresponding to the to-be-detected video in each of two categories based on the feature vector corresponding to the to-be-detected video using the counterfeit face detection model.

Description

Technical Field

[0001] This disclosure relates to the fields of deep learning and computer vision technology, specifically to a training method for a fake face detection model, a fake face detection method, and an electronic device. Background Technology

[0002] In recent years, with the rapid development of face generation and editing technologies (such as generative adversarial networks and style transfer-based image generation techniques), applications such as virtual makeup, face reshaping, and post-production face replacement in film and television have been widely adopted. However, when these technologies are misused, they can cause serious social problems, such as identity fraud, the spread of false information, and the risks of misleading public opinion and fraud caused by deepfake videos. Therefore, how to effectively and reliably detect face fraud has become a key issue in ensuring information security and social trust. Summary of the Invention

[0003] According to a first aspect of this disclosure, a training method for a fake face detection model is provided, comprising: obtaining a plurality of videos including facial information; extracting features from the facial information included in each of the plurality of videos to obtain a feature set including feature vectors corresponding to each video, wherein each feature vector in the feature set has a label indicating one of two categories, the two categories including a real face category and a fake face category; for each feature vector in the feature set, obtaining a predicted probability of the feature vector in each category based on the similarity between the feature vector and each proxy vector in each of the two categories, wherein each category includes a plurality of proxy vectors; calculating a cross-entropy loss for the feature set based on the label of each feature vector and the predicted probability in each category; and adjusting the parameters of the fake face detection model according to the cross-entropy loss.

[0004] Optionally, by extracting features from the facial information included in each of the plurality of videos, a feature set including feature vectors corresponding to each video is obtained, including: constructing a feature set based on N feature vectors corresponding to each video obtained by performing the following operations on each video: dividing the video into N video segments, where N is a positive integer; performing the following operations on each of the N video segments: selecting multiple consecutive frames in the video segment; performing face detection on each of the multiple frames to extract the corresponding face image; and arranging the multiple face images corresponding to the multiple frames in spatial order according to time to obtain a spatiotemporal plane image of the video segment; and obtaining N feature vectors corresponding to the video by inputting the N spatiotemporal plane images obtained for the video into a feature extraction model and normalizing the output of the feature extraction model respectively.

[0005] Optionally, based on the similarity between the feature vector and each proxy vector in each of the two categories, the predicted probability of the feature vector in each category is obtained, including: calculating the similarity between the feature vector and each proxy vector in that category; obtaining the weight corresponding to each proxy vector in that category based on the similarity; determining the weighted similarity score in that category by weighted aggregation of the similarity between the feature vector and each proxy vector in that category using the weights; and obtaining the predicted probability of the feature vector in that category based on the weighted similarity score in that category.

[0006] Optionally, the training method for the fake face detection model further includes: using the multiple proxy vectors included in the category to which each feature vector belongs, obtaining the reconstructed feature vector corresponding to each feature vector through mask reconstruction, and calculating the proxy refinement loss based on each feature vector and the corresponding reconstructed feature vector; and wherein, adjusting the parameters of the fake face detection model according to the cross-entropy loss includes: adjusting the parameters of the fake face detection model according to the cross-entropy loss and the proxy refinement loss.

[0007] Optionally, using the multiple surrogate vectors included in the category to which each feature vector belongs, a reconstructed feature vector corresponding to each feature vector is obtained through mask reconstruction. This includes, for each feature vector: masking the feature vector using a feature mask ratio to obtain a mask feature corresponding to the feature vector; inputting the mask feature corresponding to the feature vector into a preset encoder and normalizing the output of the encoder to obtain a preliminary feature representation corresponding to the feature vector; calculating the similarity between the preliminary feature representation corresponding to the feature vector and each surrogate vector in the category to which the feature vector belongs; obtaining the weight corresponding to each surrogate vector in the category to which the feature vector belongs based on the similarity; performing weighted aggregation on each surrogate vector in the category to which the feature vector belongs using the weight corresponding to each surrogate vector in the category to which the feature vector belongs; and inputting the result of the weighted aggregation into a preset decoder, which outputs the reconstructed feature vector corresponding to the feature vector.

[0008] Optionally, the training method for the fake face detection model further includes: calculating the proxy anchor point loss by using each proxy vector under the two categories as a proxy anchor point; and wherein, the parameters of the fake face detection model are adjusted by training based on the cross-entropy loss and the proxy refinement loss, including: adjusting the parameters of the fake face detection model based on the cross-entropy loss, the proxy refinement loss, and the proxy anchor point loss.

[0009] Optionally, the training method for the fake face detection model further includes: constructing an extended feature set, which includes multiple extended feature vectors with pseudo-labels, wherein the pseudo-labels indicate one of the two categories; obtaining a contrastive loss based on the extended feature set using a supervised contrastive learning method; and updating the parameters of the fake face detection model by training with the extended feature set according to the contrastive loss.

[0010] Optionally, constructing an extended feature set includes: obtaining the predicted probability of each input feature vector under the two categories using the fake face detection model based on each input feature vector in the input feature set, wherein the input feature set includes multiple input feature vectors; calculating the entropy value of each input feature vector based on the predicted probability of each input feature vector under the two categories; constructing an empirical cumulative distribution function based on the entropy value of each input feature vector; determining a quantile threshold based on the empirical cumulative distribution function and a preset selection ratio; and adding input feature vectors with entropy values ​​less than or equal to the quantile threshold in the input feature set as extended feature vectors to the extended feature set, wherein the input feature vectors added to the extended feature set have pseudo-labels indicating the category with the highest predicted probability.

[0011] Optionally, constructing the extended feature set further includes: for each forged feature vector in the extended feature set, obtaining a synthetic feature vector by interpolating the forged feature vector with another selected forged feature vector, wherein the forged feature vector is an extended feature vector with a pseudo-label indicating the category of the forged face; and adding the synthetic feature vector obtained for all forged feature vectors in the extended feature set as an extended feature vector to the extended feature set, wherein the synthetic feature vector has a pseudo-label indicating the category of the forged face.

[0012] Optionally, based on the extended feature set, a supervised contrastive learning method is used to obtain the contrastive loss, including: summing the losses obtained for each extended feature vector in the extended feature set by performing the following operations: calculating the first similarity between the extended feature vector and multiple positive extended feature vectors in the extended feature set, obtaining multiple first similarities corresponding to the multiple positive extended feature vectors, wherein the multiple positive extended feature vectors have the same pseudo-label as the extended feature vector, and the first similarity is the similarity after temperature scaling and exponential transformation; summing the first similarities between the extended feature vector and other extended feature vectors in the extended feature set other than the extended feature vector; and obtaining the loss for the extended feature vector based on the multiple first similarities and the result of the summation of the first similarities.

[0013] According to a second aspect of this disclosure, a method for detecting fake faces is provided, comprising: obtaining a video to be detected that includes facial information; extracting features from the facial information included in the video to be detected to obtain a feature vector corresponding to the video to be detected; obtaining a fake face detection model trained by the training method described above; and, based on the feature vector corresponding to the video to be detected, using the fake face detection model to obtain the predicted probability of the feature vector corresponding to the video to be detected in each of two categories.

[0014] According to a third aspect of this disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the training method for a fake face detection model as described above, or to implement the fake face detection method as described above.

[0015] According to a fourth aspect of this disclosure, a computer-readable storage medium is provided, wherein when instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform either the training method for a fake face detection model as described above, or the fake face detection method as described above.

[0016] The technical solutions provided according to the embodiments of this disclosure produce at least the following beneficial effects: On the one hand, by introducing a multi-agent learning mechanism to construct multiple learnable agent vectors for each category, it is possible to adapt to diverse forgery variants, effectively overcoming the bottleneck of traditional methods in capturing diverse forgery patterns and improving the generalization ability of the model. On the other hand, using cross-entropy loss to train the model can provide a clear optimization direction for the model and improve the training efficiency of the model.

[0017] The technical solutions provided by the embodiments of this disclosure also produce the following beneficial effects: First, using surrogate refinement loss can guide surrogate vectors to learn finer-grained forgery discrimination features, thereby enhancing the expressive power of surrogate vectors. Second, using surrogate anchor loss can explicitly refine surrogate vectors and normalize the metric space. Furthermore, a selective expansion adaptive strategy is introduced, which constructs an expanded feature set and uses contrastive loss to further optimize the entire model. In addition, through spatiotemporal plane layout stitching and video-level aggregation judgment, accurate capture of temporal inconsistencies is ensured, significantly improving the accuracy and robustness of video detection in practical application scenarios.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0019] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0020] Figure 1 A flowchart illustrating a training method for a fake face detection model according to an embodiment of the present disclosure is shown.

[0021] Figure 2 A schematic diagram of a training phase employing a multi-agent learning mechanism according to an embodiment of the present disclosure is shown.

[0022] Figure 3 A schematic diagram of the spatiotemporal planar layout constructed according to an embodiment of the present disclosure is shown.

[0023] Figure 4 A schematic diagram of a selective expansion adaptive strategy according to an embodiment of the present disclosure is shown.

[0024] Figure 5 A schematic diagram of the training structure of the present disclosure according to an embodiment of the present disclosure is shown.

[0025] Figure 6 A flowchart illustrating a method for detecting fake faces according to an embodiment of the present disclosure is shown.

[0026] Figure 7 A block diagram of a computing device according to an embodiment of the present disclosure is shown. Detailed Implementation

[0027] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0028] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following examples do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0029] It should be noted that the phrase "at least one of several items" in this disclosure refers to three parallel cases: "any one of the several items", "a combination of any number of the several items", and "all of the several items". For example, "including at least one of A and B" includes the following three parallel cases: (1) including A; (2) including B; (3) including A and B. Another example is "performing at least one of step one and step two", which means the following three parallel cases: (1) performing step one; (2) performing step two; (3) performing both step one and step two.

[0030] Existing methods for detecting fake faces mainly focus on improving the generalization ability of the model, that is, maintaining stable detection performance when faced with different generation methods or no fake types. To this end, researchers have proposed a variety of enhancement strategies. One mainstream strategy is to synthesize fake samples to simulate the forgery traces inherent in real deepfake videos. Another type of research aims to learn generalizable representations from existing training data, using techniques such as spatiotemporal consistency, domain alignment, and prototype learning. Despite these advances, almost all generalized detection methods rely on extracting domain-invariant features to establish a unified decision boundary. This paradigm faces two key limitations: (1) the diversity of forgery patterns among different generation techniques hinders the effective extraction of general features; (2) the static decision boundary learned during training is often difficult to adapt to new forgery techniques and data distributions encountered during deployment. These challenges highlight the urgent need for a detection framework that goes beyond general feature extraction and static training objectives.

[0031] Agent learning methods have shown significant success in classification tasks and improving model generalization. Since deepfake detection is essentially a binary classification problem, applying agent learning to this task may be a promising yet underexplored direction. (See Appendix below.) Figures 2 to 7 This disclosure specifically describes a training method for a fake face detection model, a fake face detection method, and an electronic device.

[0032] Figure 1 A flowchart illustrating a training method for a fake face detection model according to an embodiment of the present disclosure is shown.

[0033] Reference Figure 1 During operation S110, multiple videos, including facial information, are obtained.

[0034] In operation S120, feature extraction is performed on the facial information included in each of the multiple videos to obtain a feature set including feature vectors corresponding to each video. Each feature vector in the feature set has a label indicating one of two categories: real faces and fake faces.

[0035] In embodiments of this disclosure, each of the multiple videos includes facial information, and a video may include information about the same face. Each video may include multiple frames, and each frame may include facial information. Feature vectors corresponding to each video can be obtained by extracting features from the facial information included in each of the multiple videos, and a feature set is constructed using the obtained feature vectors. The feature set includes the feature vectors corresponding to each of the multiple videos. Each feature vector in the feature set has a label indicating a real face category (referred to as the real class) or a fake face category (referred to as the fake class). There may be N feature vectors extracted from each video (N is a positive integer), and the N feature vectors corresponding to the same video include the same label.

[0036] In operation S130, for each feature vector in the feature set, the predicted probability of the feature vector in each category is obtained based on the similarity between the feature vector and each surrogate vector in each of the two categories. Each category includes multiple surrogate vectors.

[0037] In embodiments of this disclosure, a learnable proxy vector pool can be constructed. This pool can be a real face category. and fake face categories Maintain separately ( It is a positive integer, for example, By default, 6 learnable agent vectors are selected. .

[0038] Figure 2 A schematic diagram of a training phase employing a multi-agent learning mechanism according to an embodiment of the present disclosure is shown.

[0039] exist Figure 2 In the middle, a system was maintained for real face categories. A learnable agent vector (Right now, 真实类 ), and also maintained for the category of fake faces. A learnable agent vector (Right now, 伪造类 ).

[0040] According to embodiments of this disclosure, obtaining the predicted probability of a feature vector in each category based on the similarity between the feature vector and each proxy vector in each of the two categories may include, in each category: calculating the similarity between the feature vector and each proxy vector in that category; obtaining a weight corresponding to each proxy vector in that category based on the similarity; determining a weighted similarity score for that category by weighted aggregation of the similarity between the feature vector and each proxy vector in that category using the weights; and obtaining the predicted probability of the feature vector in that category based on the weighted similarity score for that category.

[0041] In embodiments of this disclosure, the feature set is calculated. The Middle eigenvectors ( =1, 2, 3, …, B, where B is the feature set. The total number of feature vectors in the data and the category ( Or 1, where 0 represents the real face category and 1 represents the fake face category, for each proxy vector. Similarity between them. Constructing a similarity-based system with categories. The weights corresponding to the agent vectors (also known as agent attention weights) Similarity can be, for example, but is not limited to, cosine similarity. Depend on It consists of weight elements, Each weight element corresponds to a category. Below The Proxy Vector, where the Proxy Vector is the Proxy Vector. Weight elements as follows: (Formula 1) in, Represents cosine similarity. Represented by natural constant An exponential function with base 0. Representing the eigenvector With category The next The cosine similarity between the surrogate vectors. The numerator in Formula 1 represents the similarity between the feature vectors. With category The next The cosine similarity between the proxy vectors is exponentially transformed ( ), the denominator is the eigenvector With category Below The result of summing the cosine similarity between each proxy vector. Characterizes the feature vector relative to the category The distribution contribution of different proxy vectors.

[0042] In embodiments of this disclosure, proxy attention weight vectors are utilized. The elements in the feature vector With category The similarity between the proxy vectors is weighted and aggregated (e.g., weighted summation) to obtain the category. Weighted similarity score (Also known as the classification logic score) is as follows: (Formula 2) In embodiments of this disclosure, categories may be utilized. Weighted similarity score Calculate eigenvectors Category The predicted probabilities are as follows: (Formula 3) in, Let represent the set of categories, where 0 represents the real face category and 1 represents the fake face category. Here, the numerator in Equation 3 is the set of categories. The weighted similarity scores under the two categories are subjected to an exponential transformation, and the denominator is the sum of the weighted similarity scores under the two categories after the exponential transformation.

[0043] In operation S140, the cross-entropy loss for the feature set is calculated based on the label of each feature vector and the predicted probability under each category.

[0044] In embodiments of this disclosure, cross-entropy loss can be calculated for a feature set. Based on feature vectors Predicted probability With feature vectors Given the true labels (i.e., the tags), calculate the cross-entropy loss for the feature set. ,as follows: (Formula 4) in, Representation of feature set The total number of eigenvectors in the dataset. Indicates the first The true labels of each feature vector (where 0 represents the real face category and 1 represents the fake face category). and These correspond to the probability values ​​of the feature vector being predicted as a fake face category and a real face category, respectively. In Formula 4, log can be, but is not limited to, ln (natural logarithm, base e).

[0045] In operation S150, the parameters of the fake face detection model are adjusted based on the cross-entropy loss.

[0046] In embodiments of this disclosure, the parameters of the fake face detection model can be adjusted through training based on the cross-entropy loss.

[0047] The training method for the forged face detection model described above, on the one hand, introduces a multi-agent learning mechanism to construct multiple learnable agent vectors for each category, which can adapt to diverse forgery variants, effectively overcoming the bottleneck of traditional methods in capturing diverse forgery patterns and improving the model's generalization ability. On the other hand, using cross-entropy loss to train the model can provide a clear direction for optimization and improve the model's training efficiency.

[0048] In this disclosure, feature extraction is first performed on the video to construct a feature set.

[0049] Figure 3 A schematic diagram of the spatiotemporal planar layout constructed according to an embodiment of the present disclosure is shown.

[0050] According to embodiments of this disclosure, a feature set is constructed based on N feature vectors corresponding to each video obtained by performing the following operations on each video: dividing the video into N video segments, where N is a positive integer; performing the following operations on each of the N video segments: selecting multiple consecutive frames in the video segment; performing face detection on each of the multiple frames to extract the corresponding face image; and arranging the multiple face images corresponding to the multiple frames in spatial order according to time to obtain a spatiotemporal plane image of the video segment; and obtaining N feature vectors corresponding to the video by inputting the N spatiotemporal plane images obtained for the video into a feature extraction model and normalizing the output of the feature extraction model respectively.

[0051] In embodiments of this disclosure, each of a plurality of videos including facial information may be an input video. Where T is the number of frames in the video, C is the number of channels, and H and W are the height and width of the frame, respectively. The video is divided into N (N is a positive integer, for example, the default is 8) equal video segments, each with a length of... This facilitates subsequent processing. Within each video segment, t consecutive frames (t is a positive integer, for example, the default value is 4) are randomly selected to form a clip, thereby capturing the temporal sequence information of the video.

[0052] Face detection can be performed on each frame of each clip to extract the corresponding face image, thus obtaining t face images. (See reference...) Figure 3Face detection can be performed on each frame using, for example, a multi-task cascaded convolutional network (MTCNN). Specifically, this involves extracting only the largest face bounding box and adding 30% boundary space on each side to remove background interference and focus on facial features. The t face images (also called face frames) corresponding to the t frames can be arranged spatially in chronological order to obtain a spatiotemporal plane image (also called spatiotemporal plane layout) of the video clip (editor). Specifically, the consecutive t face images obtained from the clip can be scaled and arranged from left to right into a single image. of Thumbnail layout. Each face image (also called a sub-image or sub-frame) is sized as follows: This ensures that the spatiotemporal information of frames is preserved within a single layout. Here, a spatiotemporal plane image is obtained for each video segment, so N spatiotemporal plane images can be obtained for each video.

[0053] In embodiments of this disclosure, the obtained spatiotemporal plane image can be input into a feature extraction model, and the output of the feature extraction model can be normalized to obtain a feature vector. (Return to reference) Figure 2 The spatiotemporal plane image 201 can be input into the pre-trained DINOv3 network model 202, and the last layer can be selected. The output is used as a global feature representation, and this global feature representation is subjected to L2 normalization to obtain feature vectors. The same processing can be performed on the N spatiotemporal plane images obtained for each video to obtain N feature vectors corresponding to that video, and these N feature vectors can have the same label. Based on the obtained N feature vectors corresponding to each video, a feature set is constructed. 203, where the feature set It can include N feature vectors from each of multiple videos. Here, DINOv3 is a self-supervised visual Transformer model that is capable of extracting high-quality, general visual features.

[0054] In embodiments of this disclosure, when a video corresponds to N (N≥2) feature vectors, the N feature vectors include the same label and each feature vector can be processed as an independent feature vector.

[0055] It should be understood that the use of MTCNN for face detection and the use of the DINOv3 network model for feature extraction are merely examples, and any model or module capable of performing the same operations can be used. Furthermore, the face detection and feature extraction methods are only examples, and this disclosure is not limited thereto.

[0056] In this disclosure, a structured agent refinement loss can also be computed to enhance the expressiveness of the agent vector.

[0057] According to embodiments of this disclosure, multiple proxy vectors included in the category to which each feature vector belongs are used to obtain the reconstructed feature vector corresponding to each feature vector through mask reconstruction, and a proxy refinement loss is calculated based on each feature vector and the corresponding reconstructed feature vector; and wherein, adjusting the parameters of the fake face detection model according to the cross-entropy loss may include: adjusting the parameters of the fake face detection model according to the cross-entropy loss and the proxy refinement loss.

[0058] According to embodiments of this disclosure, obtaining a reconstructed feature vector corresponding to each feature vector by using multiple surrogate vectors included in the category to which each feature vector belongs through mask reconstruction may include, for each feature vector: masking the feature vector by using a feature mask ratio to obtain a mask feature corresponding to the feature vector; inputting the mask feature corresponding to the feature vector into a preset encoder and normalizing the output of the encoder to obtain a preliminary feature representation corresponding to the feature vector; calculating the similarity between the preliminary feature representation corresponding to the feature vector and each surrogate vector in the category to which the feature vector belongs; obtaining a weight corresponding to each surrogate vector in the category to which the feature vector belongs based on the similarity; performing weighted aggregation on each surrogate vector in the category to which the feature vector belongs using the weight corresponding to each surrogate vector in the category to which the feature vector belongs; and inputting the weighted aggregation result into a preset decoder, which outputs a reconstructed feature vector corresponding to the feature vector.

[0059] In embodiments of this disclosure, reference is made to Figure 2 It can mask feature vectors (204). The feature mask ratio can be set. (For example, the default value is 0.15), for the first eigenvectors Perform random masking to remove some feature information and obtain the corresponding mask features. .

[0060] For example, after setting the mask ratio, a binary mask can be randomly generated according to the set ratio, and then the feature vector can be multiplied with the mask (set to zero) or replaced with a specific value.

[0061] In embodiments of this disclosure, mask features can be... Extract preliminary feature representations. Figure 2 In the middle, the mask features Input preset encoder 205 performs a nonlinear mapping and L2 normalization on the output to obtain the result with respect to the feature vector. The corresponding preliminary feature representation. Among them, the encoder... This may include, for example, linear layers, layer normalization, and rectified linear units (ReLU).

[0062] In embodiments of this disclosure, weights (also known as reconstruction attention weights) corresponding to each proxy vector within the category to which the feature vector belongs can be calculated. Here, the category to which the feature vector belongs refers to the category indicated by the label of the feature vector. The feature vector can be... ( =1, 2, 3, …, B, where B is the feature set. The total number of feature vectors in the dataset and the set of positive proxies corresponding to the true labels of those feature vectors. Perform matching, for example, in the feature vector In cases where the true label indicates a forged face category, the forward proxy set It can be It can be computed with eigenvectors. The corresponding preliminary feature representations indicate the affinity between the feature representation and each agent vector in the positive agent set (i.e., the positive agent vector), thus yielding the reconstructed attention weights. As follows. Here, association affinity can be characterized by similarity, such as, but not limited to, cosine similarity.

[0063] (Formula 5) in, This indicates that cosine similarity is used to construct reconstruct attention weights, which are used to quantify the dependence of the damaged feature vector on each positive proxy vector. Here, Depend on It consists of weight elements, Each weight element corresponds to a set of positive proxies. In The Proxy Vector, where the Proxy Vector is the Proxy Vector. The calculation of each weight element can be referred to Formula 1 (i.e., (Calculation of weighted elements) only requires... and Replace with and .exist Figure 2 In When the default value of 6 is taken, by Calculated It can be a one-dimensional vector containing 6 weight elements.

[0064] In embodiments of this disclosure, feature decoding and reconstruction can be performed. Reconstruction attention weights can be utilized. For the set of forward proxies Each proxy vector in the dataset undergoes weighted aggregation (e.g., weighted summation). Figure 2 In this process, the weighted aggregation result can be input into a preset decoder. 206 (the decoder may include, for example, linear layers and layer normalization) to reconstruct the feature vector. The estimated value is the reconstructed feature vector output by the decoder corresponding to the feature vector. ,as follows: (Formula 6) in, Indicates based on the set of forward proxies The weighted feature representation of the proxy vector in Equation 6 is used to restore the mapping of the proxy space to the original feature space.

[0065] In embodiments of this disclosure, structured proxy refining losses can be calculated. Figure 2 In this approach, the mean squared error between all feature vectors in the feature set and their corresponding reconstructed feature vectors can be used to construct a structured proxy refinement loss. 207, as follows: (Formula 7) in, This represents the total number of feature vectors in the feature set. This loss term guides the proxy vectors to learn finer-grained forgery discrimination features by forcing the proxy vectors to complete the masked feature information.

[0066] It should be understood that the use of mean squared error to measure the difference between an eigenvector and its corresponding reconstructed eigenvector is merely an example, and mean absolute error, root mean square error, etc., may also be used; this disclosure is not limited to this.

[0067] In embodiments of this disclosure, the parameters of the fake face detection model can be adjusted based on the cross-entropy loss and the surrogate refinement loss. For example, the sum of the cross-entropy loss and the surrogate refinement loss can be used as the total loss, and the parameters of the fake face detection model can be adjusted through training.

[0068] In this disclosure, the surrogate anchor loss of the metric space can also be calculated to explicitly refine the surrogate vector and normalize the metric space.

[0069] According to embodiments of this disclosure, the proxy anchor loss is calculated by using each proxy vector under two categories as a proxy anchor; and wherein, adjusting the parameters of the fake face detection model by training based on the cross-entropy loss and the proxy refinement loss may include: adjusting the parameters of the fake face detection model based on the cross-entropy loss, the proxy refinement loss, and the proxy anchor loss.

[0070] In embodiments of this disclosure, proxy anchor point loss The calculation is as follows: (Formula 8) in, and These represent the interval factor (e.g., 0.1 by default) and the scaling factor (e.g., 32 by default), respectively. This represents the set of positive proxies (or simply the positive proxy set) that corresponds to the categories presented by the feature vectors in the feature set. Represents the set of all proxy vectors; for a given proxy vector , and Representing feature sets respectively The middle is used as the proxy vector The feature subset of positive eigenvectors (i.e., eigenvectors of the same class) and negative eigenvectors (i.e., eigenvectors of different classes). For If the feature set contains only feature vectors belonging to the real face category, Including real faces If there are two surrogate vectors in the feature set, then... Including two categories There are surrogate vectors. log can be, but is not limited to, ln (the natural logarithm, base e).

[0071] The significance of the first term on the right side of Formula 8 lies in the traversal. The proxy vector in the vector is the proxy anchor. It pulls the positive eigenvectors toward itself. Iterated through the proxy anchors All positive eigenvectors (i.e., the eigenset of positive eigenvectors) All positive feature vectors in (where, with the proxy anchor point) Positive eigenvectors with low similarity (e.g., but not limited to cosine similarity) will produce large values, becoming the dominant term in the summation; conversely, those with low similarity to surrogate anchors will produce large values. Positive eigenvectors with high similarity will produce very small values. The logic is that if all positive feature vectors are related to the surrogate anchor point... The similarity is high, and the loss is almost zero; if there exists a positive feature vector that is similar to the surrogate anchor point... If the similarity is low, then a loss is incurred. Therefore, the first term on the right side of Equation 8 represents the proxy vector for all feature vectors appearing in the feature set under the corresponding category (i.e., The loss generated by the proxy vector in the model is averaged.

[0072] The significance of the second term on the right side of Formula 8 lies in the traversal. The proxy vector in the vector is the proxy anchor. It pushes the negative eigenvectors away from itself. Iterated through the proxy anchors All negative eigenvectors (i.e., the feature subset of negative eigenvectors) All negative feature vectors in (where, with the proxy anchor point) Negative feature vectors with low similarity (e.g., but not limited to cosine similarity) will produce very small values; conversely, those with low similarity to surrogate anchors will produce very small values. Negative feature vectors with high similarity will produce large values. The logic is that if all negative feature vectors are related to the surrogate anchor... The similarity is low, and the loss is almost zero; if there exists a negative feature vector with the proxy anchor point If the similarity is high, then a loss is incurred. Therefore, the second term on the right side of Equation 8 applies to all proxy vectors under both categories (i.e., The loss generated by the proxy vector in the data was averaged.

[0073] In embodiments of this disclosure, the parameters of the fake face detection model can be adjusted based on cross-entropy loss, surrogate refinement loss, and surrogate anchor point loss. For example, the sum of the three losses—cross-entropy loss, surrogate refinement loss, and surrogate anchor point loss—can be used as the total loss, and the parameters of the fake face detection model can be adjusted through training. For the three losses mentioned above, any one can be selected as the loss to train the model individually, or any combination of them can be selected, and then the losses in the combination are added together as the total loss to train the model. This disclosure is not limited to this.

[0074] In this disclosure, a selective expansion adaptive strategy is introduced during the testing phase. This strategy first utilizes entropy based on the surrogate vector to ensure the reliability of pseudo-labels. Subsequently, to mitigate the loss of information samples during filtering, spherical linear interpolation is introduced at the feature level, and the entire model is further optimized using contrastive loss.

[0075] According to embodiments of this disclosure, an extended feature set is constructed, comprising multiple extended feature vectors with pseudo-labels, wherein the pseudo-labels indicate one of two categories; a contrastive loss is obtained based on the extended feature set using a supervised contrastive learning method; and the parameters of a fake face detection model are updated by training with the extended feature set based on the contrastive loss.

[0076] According to embodiments of this disclosure, constructing an extended feature set may include: obtaining the predicted probability of each input feature vector under the two categories using the fake face detection model based on each input feature vector in the input feature set, wherein the input feature set includes multiple input feature vectors; calculating the entropy value of each input feature vector based on the predicted probability of each input feature vector under the two categories; constructing an empirical cumulative distribution function based on the entropy value of each input feature vector; determining a quantile threshold based on the empirical cumulative distribution function and a preset selection ratio; and adding input feature vectors with entropy values ​​less than or equal to the quantile threshold in the input feature set as extended feature vectors to the extended feature set, wherein the input feature vectors added to the extended feature set have pseudo-labels indicating the category with the highest predicted probability.

[0077] Figure 4 A schematic diagram of a selective expansion adaptive strategy according to an embodiment of the present disclosure is shown.

[0078] During the testing phase, multiple input videos including facial information can be obtained. Features can be extracted from the facial information in each of the multiple input videos to obtain an input feature set including the input feature vector corresponding to each input video. The construction of the input feature set can be referenced in the section on feature set construction, and will not be elaborated upon in this disclosure.

[0079] exist Figure 4 The descriptions of 401, 402, and 403 are consistent with... Figure 2 The descriptions of 201, 202, and 203 are the same, except that... Figure 4 The input feature set is constructed in the middle. This will not be elaborated further in this publication.

[0080] In embodiments of this disclosure, an input feature vector can be calculated. Predicted probabilities for both categories. The input feature vector can be obtained using a fake face detection model. Predicted probabilities for both categories (i.e., prediction 404), and based on the input feature vector. Calculate the input feature vector to predict the probabilities for both categories. entropy value This is used to assess the uncertainty of the model's classification result for this sample, as follows: (Formula 9) in, Represents the input feature vector Category The prediction probability is such that the larger the entropy value, the less reliable the prediction result is.

[0081] In embodiments of this disclosure, an adaptive selection threshold can be established. The entropy set of all input feature vectors in the input feature set is statistically analyzed, and their empirical cumulative distribution function (ECDF) is constructed. Then, a preset selection ratio is applied. (For example, the default value is 0.45) Determine the quantile threshold. ,as follows: (Formula 10) in, Represents the set of real numbers. Let be the empirical cumulative distribution function of the entropy value. In Equation 10, Indicates all that satisfy The set of real numbers under this condition, and This indicates taking the lower bound (minimum) of this set. This dynamic threshold... Ensure that only the most reliable input feature vectors under the current distribution are retained.

[0082] In embodiments of this disclosure, a high-confidence extended feature set can be constructed and assigned pseudo-labels. (See also...) Figure 4 Select input feature sets whose entropy values ​​are less than or equal to the quantile threshold. The input feature vectors are added to the extended feature set (or candidate feature set). 405 is used as an extended feature vector; for adding Each input feature vector The category with the highest predicted probability is selected as its pseudo-label. ,Right now Pseudo-labels can indicate whether a face is real or fake.

[0083] According to embodiments of this disclosure, constructing an extended feature set may further include: for each forged feature vector in the extended feature set, obtaining a synthetic feature vector by interpolating the forged feature vector with another selected forged feature vector, wherein the forged feature vector is an extended feature vector having a pseudo-label indicating the category of the forged face; and adding the synthetic feature vector obtained for all forged feature vectors in the extended feature set as an extended feature vector to the extended feature set, wherein the synthetic feature vector has a pseudo-label indicating the category of the forged face.

[0084] In embodiments of this disclosure, a synthetic feature vector located between two fake feature vectors can be effectively generated by interpolating the fake feature vectors in the extended feature set, thereby filling gaps in the feature space and implicitly extending the coverage of potentially unseen fake face types. The fake feature vectors have pseudo-labels indicating the fake face category.

[0085] Spherical linear interpolation (SLEP) can be performed on two forged feature vectors, where each forged feature vector is an extended feature vector in the extended feature set with pseudo-labels indicating the category of the forged face. For each forged feature vector... Randomly sample another fake feature vector And can extract interpolation coefficients. Then, the synthesized feature vector is calculated using the slerp method, as follows: (Formula 11) in, , and All are L2-normalized feature vectors. For After all the fake feature vectors have been generated into synthetic feature vectors, all the generated synthetic feature vectors and their corresponding fake labels can be merged into the extended feature set. In the process, an extended feature set for optimization during testing is constructed. .

[0086] It should be understood that the Slerp interpolation method is only an example, and other interpolation methods between two feature vectors can also be used; this disclosure is not limited to this.

[0087] According to embodiments of this disclosure, obtaining contrastive loss using a supervised contrastive learning method based on an extended feature set may include: summing the losses obtained for each extended feature vector in the extended feature set by performing the following operations: calculating a first similarity between the extended feature vector and a plurality of positive extended feature vectors in the extended feature set, obtaining a plurality of first similarities corresponding to the plurality of positive extended feature vectors, wherein the plurality of positive extended feature vectors have the same pseudo-label as the extended feature vector, and the first similarity is a similarity after temperature scaling and exponential transformation; summing the first similarities between the extended feature vector and other extended feature vectors in the extended feature set besides the extended feature vector; and obtaining the loss for the extended feature vector based on the plurality of first similarities and the result of summing the first similarities.

[0088] In embodiments of this disclosure, an extended feature set can be used. For the training objective, supervised contrastive loss is used. (exist Figure 4 China 406) Perform online gradient updates on the fake face detection model and compare the loss. The calculation is as follows: (Formula 12) in, It is an extended feature set The set of indices, Representation and Index The corresponding extended feature vectors. (Set) Includes anchor index All indexes other than, and Indicates and Having the same pseudo-label (i.e., The set of indices of the extended eigenvectors (i.e., the positive extended eigenvectors) of ). The temperature parameter (e.g., 0.07 by default). The logarithm can be, but is not limited to, ln(natural logarithm, base e).

[0089] In Formula 12, the molecule This indicates the computation of the extended eigenvector. With positive expansion eigenvectors The similarity between them after temperature scaling and exponential transformation (i.e., the first similarity). Here, and It can be the extended feature vector after L2 normalization, then Can be considered as computation and Cosine similarity between them and The cosine similarity between them was adjusted by temperature parameters. Scaling is performed, and the scaled result is then subjected to an exponential transformation. In the denominator Represents the extended feature vector Compared with one of the other extended feature vectors in the extended feature set besides this extended feature vector. ( The temperature scaling and exponential transformation between () The similarity after (i.e., the first similarity). Similarly, Can be considered as computation and Cosine similarity between them and The cosine similarity between them was adjusted by temperature parameters. Scaling is performed, and the scaled result is then subjected to an exponential transformation. ). Denominator It can be an extended feature vector The first similarity between the vectors and other extended feature vectors is summed. (Formula 12) Represents the extended eigenvector The contrastive loss can be obtained by summing the losses obtained for each expanded feature vector. .

[0090] The training method for the forged face detection model described above firstly employs a surrogate refinement loss to guide the surrogate vectors to learn finer-grained forgery-detection features, thereby enhancing the expressive power of the surrogate vectors. Secondly, a surrogate anchor loss is used to explicitly refine the surrogate vectors and normalize the metric space. Furthermore, a selective expansion adaptive strategy is introduced, constructing an expanded feature set and employing a contrastive loss to further optimize the entire model. Additionally, spatiotemporal plane layout stitching and video-level aggregation judgment ensure accurate capture of temporal inconsistencies, significantly improving the accuracy and robustness of video detection in practical applications.

[0091] Figure 5 A schematic diagram of the training structure of the present disclosure according to an embodiment of the present disclosure is shown.

[0092] Reference Figure 5 The training structure will Figure 2 The training phase of the multi-agent learning mechanism in the middle and Figure 4 The test-time training phase, which employs a selective expansion adaptive strategy, is combined with the training phase. Figure 5 The corresponding description can be found in [reference]. Figure 2 and Figure 4 The description is as follows. During the training phase, this disclosure introduces a multi-agent learning mechanism, which establishes multiple agent vectors for each category to adapt to diverse forgery variants. Simultaneously, it utilizes raw features extracted from the backbone network and combines cross-entropy loss, agent anchor loss, and structured agent refinement loss to optimize the model. During the training phase at test time, a selective expansion adaptive strategy is introduced. This strategy first utilizes entropy based on agent vectors to ensure the reliability of pseudo-labels. Subsequently, to mitigate the loss of information samples during filtering, spherical linear interpolation is introduced at the feature level, and contrastive loss is used to further optimize the entire model.

[0093] Figure 6 A flowchart illustrating a method for detecting fake faces according to an embodiment of the present disclosure is shown.

[0094] In step S610, a video to be detected, including facial information, is obtained. In step S620, feature vectors corresponding to the video to be detected are obtained by extracting features from the facial information included in the video to be detected.

[0095] In embodiments of this disclosure, feature extraction of facial information included in the video to be detected can refer to the above description for each of the multiple videos. Feature extraction is performed to obtain the feature vector corresponding to the video to be detected. .

[0096] In step S630, a fake face detection model is obtained by training the fake face detection model using the training method. In step S640, based on the feature vector corresponding to the video to be detected, the predicted probability of the feature vector corresponding to the video to be detected in each of the two categories is obtained using the fake face detection model.

[0097] In the embodiments of this disclosure, a trained fake face detection model can be used to calculate feature vectors respectively. The feature vector is obtained by calculating the weighted similarity score between the proxy vectors in the two categories (e.g., refer to Equation 2). Predicted probabilities under both categories (For example, refer to Formula 3).

[0098] In the embodiments of this disclosure, when a video to be detected corresponds to N (N≥2) feature vectors to be detected, each of these N feature vectors can be processed as an independent feature vector to be detected. After obtaining the predicted probability of each feature vector to be detected under the two categories, the predicted probabilities of the N feature vectors to be detected under each category are averaged to obtain the average predicted probability under the two categories. Subsequently, the classification result of whether the video to be detected is a real face or a fake face can be obtained based on the average predicted probability under the two categories.

[0099] Figure 7 A block diagram of an electronic device 700 according to an embodiment of the present disclosure is shown.

[0100] Reference Figure 7 The electronic device 700 includes at least one memory 701 and at least one processor 702. The at least one memory 701 stores instructions that, when executed by the at least one processor 702, execute a training method for a fake face detection model or a fake face detection method according to exemplary embodiments of the present disclosure.

[0101] As an example, electronic device 700 may be a PC, tablet, personal digital assistant, smartphone, or other device capable of executing the aforementioned instructions. Here, electronic device 700 is not necessarily a single electronic device, but may be a collection of any devices or circuits capable of executing the aforementioned instructions (or instruction sets) individually or in combination. Electronic device 700 may also be part of an integrated control system or system manager, or may be configured to interconnect with a portable electronic device locally or remotely (e.g., via wireless transmission) through an interface.

[0102] In electronic device 700, processor 702 may include a central processing unit (CPU), a graphics processing unit (GPU), a programmable logic device, a dedicated processor system, a microcontroller, or a microprocessor. By way of example and not limitation, processor may also include analog processors, digital processors, microprocessors, multi-core processors, processor arrays, network processors, etc.

[0103] The processor 702 can execute instructions or code stored in the memory 701, which can also store data. Instructions and data can also be sent and received via a network through a network interface device, which can employ any known transmission protocol.

[0104] The memory 701 can be integrated with the processor 702, for example, by placing RAM or flash memory within an integrated circuit microprocessor. Alternatively, the memory 701 can include a separate device, such as an external disk drive, a storage array, or other storage device usable by any database system. The memory 701 and the processor 702 can be operatively coupled, or can communicate with each other, for example, via I / O ports, network connections, etc., enabling the processor 702 to read files stored in the memory.

[0105] In addition, the electronic device 700 may also include a video display (such as a liquid crystal display) and a user interaction interface (such as a keyboard, mouse, touch input device, etc.). All components of the electronic device 700 can be interconnected via a bus and / or network.

[0106] According to exemplary embodiments of the present disclosure, a computer-readable storage medium may also be provided, which, when executed by a processor of an electronic device, enables the electronic device to perform a training method for a fake face detection model or a fake face detection method according to the present disclosure. Examples of computer-readable storage media include: read-only memory (ROM), random access programmable read-only memory (PROM), electrically erasable programmable read-only memory (EEPROM), random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), flash memory, non-volatile memory, CD-ROM, CD-R, CD+R, CD-RW, CD+RW, DVD-ROM, DVD-R, DVD+R, DVD-RW, DVD+RW, DVD-RAM, BD-ROM, BD-R, BD-R LTH, BD-RE, Blu-ray or optical disc storage, hard disk drive (HDD), solid-state drive (SSD), card storage (such as multimedia cards, secure digital (SD) cards, or ultra-fast digital (XD) cards), magnetic tape, floppy disk, magneto-optical data storage device, optical data storage device, hard disk, solid-state drive, and any other device configured to store a computer program and any associated data, data files, and data structures in a non-transitory manner and to provide the computer program and any associated data, data files, and data structures to a processor or computer so that the processor or computer can execute the computer program. The computer program in the aforementioned computer-readable storage medium can run in an environment deployed in computer devices such as clients, hosts, agent devices, servers, etc. Furthermore, in one example, the computer program and any associated data, data files, and data structures are distributed across a networked computer system, such that the computer program and any associated data, data files, and data structures are stored, accessed, and executed in a distributed manner through one or more processors or computers.

[0107] According to exemplary embodiments of the present disclosure, a computer program product may also be provided, including a computer program that, when executed by a processor, implements a training method for a fake face detection model or a fake face detection method according to the present disclosure.

[0108] The training method, detection method, and electronic device for the fake face detection model described above first stitch together consecutive cropped face frames into a specific spatiotemporal layout as initial input; then, DINOv3 is used for preliminary spatiotemporal feature extraction. During the training phase, a multi-agent learning mechanism is introduced, which establishes multiple agent vectors for each category to adapt to diverse fake variants. Simultaneously, the original features extracted from the backbone network are used, along with three combined losses, to perform initial model optimization. During the training phase in testing, a selective expansion adaptive strategy is introduced. This strategy ensures the reliability of fake labels through entropy filtering, while using spherical linear interpolation to expand features, and further optimizes the entire model through contrastive loss. Furthermore, both the training and testing phases achieve more robust real / fake discrimination by calculating the similarity between input features and agent vectors.

[0109] Furthermore, by constructing multiple learnable proxy vectors for each category through a multi-proxy learning mechanism, the bottleneck of traditional methods in capturing diverse forgery patterns is effectively overcome. Combined with a structured proxy refinement strategy, a mask reconstruction task forces the proxy vectors to capture finer-grained discriminative clues, significantly improving feature robustness. A selective expansion adaptive strategy is introduced, utilizing entropy filtering and spherical linear interpolation to achieve dynamic optimization during testing, effectively solving the problem of poor generalization of static decision boundaries in unknown domains. Simultaneously, spatiotemporal plane layout stitching and video-level aggregation judgment ensure accurate capture of temporal inconsistencies, significantly improving the accuracy and robustness of video detection in practical applications.

[0110] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0111] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A training method for a fake face detection model, characterized in that, The training method includes: Obtain multiple videos, including facial information; By extracting features from the facial information included in each of the plurality of videos, a feature set including feature vectors corresponding to each video is obtained, wherein each feature vector in the feature set has a label indicating one of two categories, namely, a real face category and a fake face category; For each feature vector in the feature set, the predicted probability of the feature vector in each category is obtained based on the similarity between the feature vector and each proxy vector in each of the two categories, wherein each category includes multiple proxy vectors; Based on the label of each feature vector and the predicted probability under each category, calculate the cross-entropy loss for the feature set; and The parameters of the fake face detection model are adjusted based on the cross-entropy loss. The training method further includes: Construct an extended feature set, which includes multiple extended feature vectors with pseudo-labels, wherein the pseudo-labels indicate one of the two categories; Based on the extended feature set, a supervised contrastive learning method is used to obtain the contrastive loss; and Based on the contrastive loss, the parameters of the fake face detection model are updated by training with an expanded feature set, and The construction of the extended feature set includes: Based on each input feature vector in the input feature set, the predicted probability of each input feature vector under the two categories is obtained using the fake face detection model, wherein the input feature set includes multiple input feature vectors; Calculate the entropy value of each input feature vector based on the predicted probability of each input feature vector under the two categories; Based on the entropy value of each input feature vector, an empirical cumulative distribution function is constructed; The quantile threshold is determined based on the empirical cumulative distribution function and a preset selection ratio; and Input feature vectors whose entropy values ​​are less than or equal to the quantile threshold are added to the extended feature set as extended feature vectors. The input feature vectors added to the extended feature set have pseudo-labels indicating the class with the highest prediction probability.

2. The training method as described in claim 1, characterized in that, By extracting features from the facial information included in each of the plurality of videos, a feature set including feature vectors corresponding to each video is obtained, including: Based on the N feature vectors corresponding to each video obtained by performing the following operations on each video, a feature set is constructed: Divide the video into N video segments, where N is a positive integer; For each of the N video segments, the following operations are performed: selecting multiple consecutive frames within the video segment; performing face detection on each of the multiple frames to extract the corresponding face image; and obtaining a spatiotemporal plane image of the video segment by arranging the multiple face images corresponding to the multiple frames in spatial order according to time; and By inputting N spatiotemporal plane images obtained from the video into the feature extraction model and normalizing the output of the feature extraction model, N feature vectors corresponding to the video are obtained.

3. The training method as described in claim 1, characterized in that, Based on the similarity between the feature vector and each proxy vector in each of the two categories, the predicted probability of the feature vector in each category is obtained, including the following in each category: Calculate the similarity between the feature vector and each proxy vector in that category; Based on the similarity, the weight corresponding to each agent vector under that category is obtained; The weighted similarity score for that category is determined by weighting and aggregating the similarity between the feature vector and each proxy vector in that category using the weights. as well as Based on the weighted similarity score within the category, the predicted probability of the feature vector within that category is obtained.

4. The training method as described in claim 1, characterized in that, Also includes: Using the multiple proxy vectors included in the category to which each feature vector belongs, a reconstructed feature vector corresponding to each feature vector is obtained through mask reconstruction. Based on each feature vector and its corresponding reconstructed feature vector, calculate the proxy refining loss; and The parameters of the fake face detection model are adjusted based on the cross-entropy loss, including: The parameters of the fake face detection model are adjusted based on the cross-entropy loss and the surrogate refinement loss.

5. The training method as described in claim 4, characterized in that, Using the multiple proxy vectors included in the category to which each feature vector belongs, a reconstructed feature vector corresponding to each feature vector is obtained through mask reconstruction, including for each feature vector: By using the feature mask ratio to mask the feature vector, the mask feature corresponding to the feature vector can be obtained. By inputting the mask features corresponding to the feature vector into a preset encoder and normalizing the output of the encoder, a preliminary feature representation corresponding to the feature vector is obtained. Calculate the similarity between the preliminary feature representation corresponding to the feature vector and each proxy vector under the category to which the feature vector belongs; Based on the similarity, the weights corresponding to each proxy vector in the category to which the feature vector belongs are obtained; We perform weighted aggregation on each proxy vector in the category to which the feature vector belongs, using the weights corresponding to each proxy vector in the category to which the feature vector belongs. as well as The weighted aggregation result is input into a preset decoder, which outputs a reconstructed feature vector corresponding to the feature vector.

6. The training method as described in claim 4, characterized in that, Also includes: The agent anchor loss is calculated by using each agent vector under the two categories as an agent anchor. and Specifically, the parameters of the fake face detection model are adjusted through training based on cross-entropy loss and surrogate refinement loss, including: The parameters of the fake face detection model are adjusted based on cross-entropy loss, proxy refinement loss, and proxy anchor point loss.

7. The training method as described in claim 1, characterized in that, Building an extended feature set also includes: For each forged feature vector in the extended feature set, a synthetic feature vector is obtained by interpolating the forged feature vector with another selected forged feature vector, where the forged feature vector is an extended feature vector with a pseudo-label indicating the category of the forged face; and The synthetic feature vectors obtained from all the fake feature vectors in the extended feature set are added to the extended feature set as extended feature vectors, where the synthetic feature vectors have pseudo-labels indicating the fake face category.

8. The method as described in claim 1, characterized in that, Based on the extended feature set, a supervised contrastive learning method is used to obtain the contrastive loss, including: The contrastive loss is obtained by summing the losses obtained for each extended feature vector in the extended feature set by performing the following operations: By calculating the first similarity between the extended feature vector and multiple positive extended feature vectors in the extended feature set, multiple first similarities corresponding to the multiple positive extended feature vectors are obtained, wherein the multiple positive extended feature vectors have the same pseudo-label as the extended feature vector, and the first similarity is the similarity after temperature scaling and exponential transformation. The first similarity between the extended feature vector and each of the other extended feature vectors in the extended feature set is summed. The loss for the extended feature vector is obtained by summing the multiple first similarities.

9. A method for detecting fake faces, characterized in that, The method includes: Obtain the video to be inspected, including facial information; By extracting features from the facial information included in the video to be detected, a feature vector corresponding to the video to be detected is obtained. Obtain a fake face detection model trained by the training method as described in any one of claims 1-8; and Based on the feature vector corresponding to the video to be detected, the fake face detection model is used to obtain the predicted probability of the feature vector corresponding to the video to be detected in each of the two categories.

10. An electronic device, characterized in that, include: processor; Memory used to store the processor's executable instructions; The processor is configured to execute the instructions to implement the training method for the fake face detection model as described in any one of claims 1 to 8, or to implement the fake face detection method as described in claim 9.

11. A computer-readable storage medium, characterized in that, When the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device is able to perform the training method of the fake face detection model as described in any one of claims 1 to 8, or the fake face detection method as described in claim 9.

Images