Face authentic identification method and device

Abstract

The invention provides a face authentic identification method and device. The method comprises the following steps: acquiring a multi-task face authentic identification model; wherein the multi-task face authentic identification model is generated according to an image face authentic identification model and a video face authentic identification model; acquiring facial visual data to be detected; wherein the face visual data comprises image face data or video face data; and inputting the face visual data into the multi-task face authentic identification model to obtain a face authentic identification result of the to-be-detected face in the face visual data output by the multi-task face authentic identification model, so that a single model can process the image face data and the video face data in a compatible manner, the system deployment complexity and the computing resource consumption are reduced, and the user experience is improved. Moreover, the method can achieve the refined recognition of counterfeit categories, improves the interpretability and practicability of an authentication result, facilitates the adoption of different defense strategies for different face counterfeit categories, and remarkably improves the safety protection capability of the system.

Description

Technical Field

[0001] This disclosure relates to the field of computer vision technology, and in particular to a method and apparatus for facial authentication. Background Technology

[0002] With the rapid development of biometrics and artificial intelligence, facial recognition technology has been widely applied in various fields such as financial payments, security monitoring, and smart terminal unlocking, becoming one of the core means of identity authentication. However, deep learning-based facial spoofing technology has emerged rapidly, capable of creating visually highly realistic fake facial images or videos by replacing, editing, or generating facial features. When such technology is maliciously exploited, it may lead to serious security problems such as identity theft, financial fraud, and the spread of false information, severely threatening the reliability and security of related facial recognition systems. Therefore, to improve the security protection capabilities of the system, how to effectively distinguish between real and fake faces has become an urgent technical problem to be solved. Summary of the Invention

[0003] This disclosure provides a facial authentication method and apparatus to at least partially solve one of the technical problems in related technologies. The technical solution of this disclosure is as follows:

[0004] According to a first aspect of the present disclosure, a facial authentication method is provided, comprising: acquiring a multi-task facial authentication model; wherein the multi-task facial authentication model is generated based on an image facial authentication model and a video facial authentication model; acquiring facial visual data to be detected; wherein the facial visual data includes image facial data or video facial data; inputting the facial visual data into the multi-task facial authentication model to obtain a facial authentication result of the face to be detected in the facial visual data output by the multi-task facial authentication model; wherein the facial authentication result includes at least one of the following: a first probability that the face to be detected belongs to a real face, a second probability that the face to be detected belongs to a fake face, and a third probability that the face to be detected belongs to multiple candidate fake categories when the second probability is greater than the first probability.

[0005] According to a second aspect of the present disclosure, a face authentication device is provided, comprising: a first acquisition module for acquiring a multi-task face authentication model; wherein the multi-task face authentication model is generated based on an image face authentication model and a video face authentication model; a second acquisition module for acquiring facial visual data to be detected; wherein the facial visual data includes image facial data or video facial data; and an authentication module for inputting the facial visual data into the multi-task face authentication model to obtain a face authentication result of the face to be detected in the facial visual data output by the multi-task face authentication model; wherein the face authentication result includes at least one of the following: a first probability that the face to be detected belongs to a real face, a second probability that the face to be detected belongs to a fake face, and a third probability that the face to be detected belongs to multiple candidate fake categories when the second probability is greater than the first probability.

[0006] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to execute the instructions to implement the face authentication method as described in the first aspect of the present disclosure.

[0007] According to a fourth aspect of the present disclosure, a computer-readable storage medium is provided that, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to perform the facial authentication method as described in the first aspect of the present disclosure.

[0008] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising: a computer program that, when executed by a processor, implements the facial authentication method as described in the first aspect of the present disclosure.

[0009] The technical solutions provided by the embodiments of this disclosure have at least the following beneficial effects:

[0010] In this technical solution, a unified multi-task face authentication model is constructed, integrating the discriminative capabilities of image-based and video-based face authentication models. This allows a single model to handle both image and video face data, significantly reducing system deployment complexity and computational resource consumption, while improving operational efficiency and maintainability. Furthermore, at the output level, the model not only provides two probabilities (first and second probabilities) for whether the detected face is real or fake, enabling accurate identification of face authenticity, but also outputs a third probability in cases where the face is determined to be fake—the probability that the detected face belongs to multiple candidate fake categories—achieving refined identification of fake categories. This mechanism not only enhances the interpretability and practicality of the authentication results but also helps the system adopt differentiated defense strategies for different face fake categories, significantly improving the system's security capabilities. Specifically, during the generation of the multi-task face authentication model, an image-based face authentication model pre-trained on large-scale image data is first utilized. The algorithm acquires robust static facial feature representation capabilities. Then, it freezes the image feature extraction network to prevent damage to its learned image feature extraction capabilities during subsequent training. Single-frame features from video frames are extracted using the image feature extraction network and used to train the feature transfer network and video face authentication network. This allows training only the feature transfer network and video face authentication network without updating the parameters of the image feature extraction network, reducing computational resource consumption and training time, and improving model convergence speed. Finally, the trained image feature extraction network, image face authentication network, feature transfer network, and video face authentication network are integrated to generate a unified multi-task face authentication model. This enables the multi-task face authentication model to simultaneously possess image and video dual-modal processing capabilities, supports multi-task output, and improves the applicability of the multi-task face authentication model. Specifically, by introducing the feature transfer network, a fusion feature with temporal context awareness is constructed based on category vectors, significantly enhancing the ability to determine the authenticity of faces in videos.

[0011] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure, and are not intended to unduly limit this disclosure.

[0013] Figure 1 This is a schematic flowchart of the facial authentication method shown in the first embodiment of this disclosure;

[0014] Figure 2 This is a schematic flowchart of the facial authentication method shown in the second embodiment of this disclosure;

[0015] Figure 3 This is a flowchart illustrating the facial authentication method shown in the third embodiment of this disclosure;

[0016] Figure 4 This is a schematic flowchart of the facial authentication method shown in the fourth embodiment of this disclosure;

[0017] Figure 5 This is a schematic flowchart of the facial authentication method shown in the fifth embodiment of this disclosure;

[0018] Figure 6 This is a schematic diagram of the structure of the image face authentication network shown in the embodiments of this disclosure;

[0019] Figure 7 This is a schematic diagram of the structure of the nonlinear layer shown in the embodiments of this disclosure;

[0020] Figure 8 This is a schematic diagram illustrating the principle of the video face authentication model shown in the embodiments of this disclosure;

[0021] Figure 9 This is a schematic diagram illustrating the principle of the multi-task face authentication model shown in the embodiments of this disclosure;

[0022] Figure 10 This is a schematic diagram of the structure of the facial authentication device shown in the sixth embodiment of this disclosure;

[0023] Figure 11 This is a schematic diagram of the structure of an electronic device shown in an exemplary embodiment of the present disclosure. Detailed Implementation

[0024] To enable those skilled in the art to better understand the technical solutions of this disclosure, the technical solutions in the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings.

[0025] It should be noted that the terms "first," "second," etc., used in the specification, claims, and accompanying drawings of this disclosure are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this disclosure described herein can be implemented in orders other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0026] It should be noted that the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution disclosed herein are all carried out with the consent of the user, and all comply with the provisions of relevant laws and regulations, and do not violate public order and good morals.

[0027] With the rapid development of Artificial Intelligence Generated Content (AIGC) technology, the distinction between real faces (e.g., human faces) and AIGC-generated faces is becoming increasingly blurred. If this technology is misused, it could seriously impact human property and portrait security. To differentiate between real faces and AIGC-generated forged faces, corresponding face authentication techniques have been proposed, including image face authentication, video face authentication, and multimodal authentication. Image face authentication can only focus on the spatial features of an image and relies primarily on the identification of local forgery traces, resulting in a significant decrease in its ability to detect high-fidelity forged face images. Multimodal authentication requires the video to simultaneously contain the image and audio information of the object, limiting its applicability in silent videos or non-audio scenarios. Early video face authentication relied on image face authentication models and multi-frame result fusion techniques; these methods can only learn the forgery information of a single frame image, ignoring the inconsistencies between multiple frames in a video. Recent video face authentication techniques not only focus on the spatial information of a single frame but also on the inconsistencies between multiple frames, i.e., temporal information, demonstrating great potential on video face test sets. However, these methods cannot be applied to image-based face authentication tasks, requiring multiple authentication models for multiple tasks.

[0028] In response to at least one of the aforementioned problems, this disclosure proposes a facial authentication method and apparatus.

[0029] The facial authentication method and apparatus of this disclosure are described below with reference to the accompanying drawings.

[0030] Figure 1 This is a schematic flowchart of the facial authentication method shown in the first embodiment of this disclosure.

[0031] like Figure 1 As shown, this facial authentication method includes the following steps:

[0032] Step 101: Obtain the multi-task face authentication model.

[0033] The multi-task face authentication model is generated based on the image face authentication model and the video face authentication model.

[0034] To reduce deployment complexity, in this embodiment of the disclosure, a multi-task face authentication model is constructed based on an image face authentication model and a video face authentication model. The image face authentication model can handle face authentication tasks in the image modality, the video face authentication model can handle face authentication tasks in the video modality, and the multi-task face authentication model can handle face authentication tasks in either the image or the video modality.

[0035] Step 102: Obtain the visual data of the face to be detected.

[0036] Facial visual data includes image facial data or video facial data.

[0037] To enhance the flexibility and applicability of facial authentication, in this embodiment of the disclosure, the input to the multi-task facial authentication model can be a single image or video clip. That is, the facial visual data to be detected can include image facial data or video facial data. For example, image facial data is a selfie, and video facial data is a 3-second video of a face.

[0038] Step 103: Input the facial visual data into the multi-task facial authentication model to obtain the facial authentication result of the face to be detected in the facial visual data output by the multi-task facial authentication model.

[0039] The facial authentication result includes at least one of the following: a first probability that the face to be detected is a real face, a second probability that the face to be detected is a fake face, and a third probability that the face to be detected belongs to multiple candidate fake categories when the second probability is greater than the first probability.

[0040] Furthermore, facial visual data is input into a multi-task facial authentication model. This model comprehensively judges the authenticity of the face to be detected in the image or video, and finally outputs the facial authentication result. The facial authentication result includes at least: a first probability that the detected face is a real face, reflecting the confidence level that it is a real face; a second probability that the detected face is a fake face, indicating the possibility that it is not a living person or synthetic content; furthermore, when the second probability is greater than the first probability, the model will also output a third probability that the detected face belongs to multiple candidate fake categories (such as artificial intelligence (AI) face swapping, face reconstruction, lip-syncing, etc.), to indicate the most likely fake category. Thus, this output mechanism can not only accurately determine the authenticity of faces, but also provide fine-grained fake category identification, enhancing the interpretability and security of facial authentication.

[0041] In summary, by constructing a unified multi-task face authentication model that integrates the discriminative capabilities of image-based and video-based face authentication models, a single model can be compatible with both image and video face data. This significantly reduces system deployment complexity and computational resource consumption, while improving operational efficiency and maintainability. Furthermore, at the output level, the model not only provides two probabilities (i.e., the first probability and the second probability) for the detected face to be real or fake, achieving accurate discrimination of face authenticity, but also outputs a third probability in cases where the face is determined to be fake, i.e., the probability that the detected face belongs to multiple candidate fake categories, achieving refined identification of fake categories. This mechanism not only enhances the interpretability and practicality of the authentication results but also helps the system adopt differentiated defense strategies for different face fake categories, significantly improving the system's security protection capabilities.

[0042] To clearly illustrate how the multi-task face authentication model is generated in the above embodiments, this disclosure proposes another face authentication method.

[0043] Figure 2 This is a schematic flowchart of the facial authentication method shown in the second embodiment of this disclosure.

[0044] like Figure 2 As shown, based on any of the above embodiments, the multi-task face authentication model can be generated using the following steps:

[0045] Step 201: Obtain the trained image face authentication model.

[0046] The image face authentication model includes an image feature extraction network and an image face authentication network.

[0047] To reduce the training time and resources of the multi-task face authentication model, in this embodiment of the disclosure, a multi-task face authentication model is constructed based on the trained image face authentication model. Therefore, in the process of constructing the multi-task face authentication model, the trained image face authentication model can be obtained first, which can determine whether a face in an image is real or fake.

[0048] It should be noted that the image face authentication model includes an image feature extraction network and an image face authentication network. The image feature extraction network is used to extract key visual features (such as texture, edges, micro-expression cues, etc.) from the input image, while the image face authentication network is used to classify based on the extracted image features and output a judgment result on whether the face is fake.

[0049] It should be noted that building a multi-task face authentication model based on a trained image face authentication model can fully utilize the robust feature representation capabilities learned by the image face authentication model on large-scale image data, thereby significantly improving the initial recognition performance and training stability of the subsequent multi-task face authentication model in image modality. In addition, this design not only avoids the high cost problem caused by training from scratch, but also lays a high-quality feature foundation for realizing multi-task authentication capabilities that fuse image and video modalities.

[0050] Step 202: Freeze the image feature extraction network in the trained image face authentication model to train the feature transfer network and the video face authentication network in the video face authentication model.

[0051] Among them, the feature transfer network is used to fuse image features based on the category vector.

[0052] To further reduce the construction cost of the multi-task face authentication model while retaining the feature extraction capability of the image face authentication model, in this embodiment of the disclosure, the image feature extraction network in the trained image face authentication model can be frozen, that is, the weight parameters of the image feature extraction network in the trained image face authentication model are fixed and not updated, and it is only used as the feature extractor of the multi-task face authentication model.

[0053] Furthermore, the feature extraction capabilities of this image feature extraction network are transferred to video tasks. A frozen feature extractor is used to extract static visual features from each frame of the video. These single-frame features are then used as input to train the feature transfer network and the video face authentication network in the video face authentication model. Specifically, the feature transfer network fuses image features based on category vectors, and the video face authentication network determines whether a face in the video is fake based on the fused features.

[0054] It's important to understand that image features are learned under static, single-frame conditions, primarily capturing spatial information such as texture, edges, and lighting consistency. Video data, however, contains not only spatial information but also dynamic features in the temporal dimension, such as blink frequency, mouth movement trajectories, and micro-expression changes—physiological rhythms. Therefore, directly using image-level features is insufficient for effectively modeling the temporal dynamics of video.

[0055] Therefore, a feature transfer network is introduced to fuse static features in the image domain across time based on category vectors (e.g., through temporal modeling mechanisms such as LSTM, 3D convolution, or Transformer), constructing fused features with temporal context awareness. This process significantly enhances the video face authentication model's ability to perceive dynamic consistency. For example, real faces typically exhibit natural and coherent movement patterns in video sequences, while fake videos (such as AI face swapping and screen playback) often exhibit temporal anomalies such as frame discontinuities, stiff movements, and missing physiological signals. By modeling and identifying such dynamic inconsistencies, the video face authentication model can more accurately distinguish between real and fake faces, thereby effectively improving the accuracy of face forgery detection.

[0056] Step 203: Generate a multi-task face authentication model based on the trained image feature extraction network, the trained image face authentication network, the trained feature transfer network, and the trained video face authentication network.

[0057] Furthermore, the trained image feature extraction network, the trained image face authentication network, the trained feature transfer network, and the trained video face authentication network are combined to generate a multi-task face authentication model.

[0058] It should be noted that by integrating four trained networks—image feature extraction network, image face authentication network, feature transfer network, and video face authentication network—a multi-task face authentication model is constructed. This not only achieves unified processing across modalities (images / videos), but also improves the ability to identify complex face forgeries through the collaborative analysis of spatial features and temporal dynamics.

[0059] In summary, firstly, a robust static facial feature representation capability is obtained by pre-training an image face authentication model on large-scale image data. Next, the image feature extraction network is frozen to prevent damage to its learned image feature extraction capabilities during subsequent training. Single-frame features from video frames are then extracted using the image feature extraction network and used to train the feature transfer network and the video face authentication network. This allows training only the feature transfer network and the video face authentication network without updating the parameters of the image feature extraction network, reducing computational resource consumption and training time, and improving model convergence speed. Finally, the trained image feature extraction network, image face authentication network, feature transfer network, and video face authentication network are integrated to generate a unified multi-task face authentication model. This enables the multi-task face authentication model to simultaneously possess image and video dual-modal processing capabilities, supports multi-task output, and improves the applicability of the multi-task face authentication model.

[0060] To clearly illustrate how the image face authentication model is trained in the above embodiments, this disclosure proposes another face authentication method.

[0061] Figure 3 This is a schematic flowchart of the facial authentication method shown in the third embodiment of this disclosure.

[0062] like Figure 3 As shown, based on any of the above embodiments, the image face authentication model can be trained using the following steps:

[0063] Step 301: Use an image feature extraction network to extract semantic features from the face in the sample image to obtain the first facial semantic features.

[0064] To improve the model's feature representation capability in complex scenes, in this embodiment, an image feature extraction network is used to extract semantic features from the sample images to obtain first facial semantic features. These semantic features may include features such as skin texture regularity, edge continuity, illumination consistency, and micro-expression distribution.

[0065] It should be noted that the sample image can be a single facial image from the training dataset, and the sample image can be obtained using the following steps:

[0066] (1) Obtain the original facial image; wherein the original facial image includes a real face or a fake face;

[0067] To enable the model to comprehensively learn various forgery features, this embodiment of the disclosure constructs a diverse dataset covering both real and forged scenarios, ensuring the model's discriminative ability. It should be noted that original facial images refer to images containing faces obtained from sources such as cameras, databases, and user uploads. Each original facial image contains one type of face, such as a real face (live) and a forged face (e.g., printed photos, mobile phone screen displays, 3D masks, AI-generated faces, etc.).

[0068] (2) Perform facial key point detection on the original facial image to obtain the initial facial detection box;

[0069] To improve the accuracy of the model's focus area, in this embodiment, a facial key point detection algorithm is used to locate key facial points (eyes, nose tip, corners of mouth) in the original facial image. Based on the spatial distribution of these key points, a rectangular box that surrounds the main facial region is fitted, i.e., the initial facial detection box. The initial facial detection box can be used to locate the position and pose of the face in the image.

[0070] (3) Adjust the size of the initial face detection box in the original face image to obtain the target face detection box;

[0071] Since the original coordinate box may be inaccurate, resulting in the loss of facial areas, in order to retain more facial areas and thus more facial details, in this embodiment of the disclosure, after obtaining the initial facial detection box, it is uniformly scaled or cropped to a fixed size according to the model input requirements to obtain a standardized facial rectangle, i.e., the target facial detection box.

[0072] (4) Based on the target face detection box, crop the sample image from the original face image.

[0073] In order to enable the model to focus on key regions and improve the training efficiency of the model, in this embodiment of the disclosure, the pixel data within the rectangular region of the target face detection box is extracted from the original face image to generate a sample image.

[0074] Step 302: Using an image face authentication network based on the first facial semantic features, the faces in the sample image are classified as real or fake to obtain the first predicted label.

[0075] The first prediction label is used to indicate the probability that a face in the sample image is a real face or a fake face.

[0076] To improve the interpretability of the model, in this embodiment, the image face authentication network serves as the task head of the model. It receives semantic features extracted from the deep network (such as skin texture regularity, edge continuity, illumination consistency, and other discriminative features closely related to authenticity) and finally outputs a first prediction label. The first prediction label is used to indicate the probability that the face in the sample image belongs to a real face or the probability that it belongs to a fake face (e.g., the real probability is 96%, and the fake probability is 4%).

[0077] It should be noted that the image face authentication network can be a multilayer perceptron (MLP) network. The MLP network adopts a non-linear layer stacking structure consisting of linear layers, batch normalization layers and ReLU activation layers. Combined with the final linear layer, it not only enhances the non-linear fitting ability of the MLP model, but also accelerates training convergence and improves stability through batch normalization. The overall structure balances expressiveness and training efficiency.

[0078] Step 303: Train the image face authentication model based on the difference between the first predicted label and the first labeled label corresponding to the sample image.

[0079] The first label is used to indicate whether the face in the sample image is real or fake.

[0080] To achieve effective training of the image face authentication model, in this embodiment, during the training process, the first predicted label output by the model is compared with the real label (i.e., the first annotation label) corresponding to the sample image. By calculating the difference between the two (usually using a loss function, such as cross-entropy loss), the model parameters are optimized. The first predicted label is the probability predicted by the model based on the facial semantic features of the input image as either real or fake, while the first annotation label is pre-annotated manually and reflects the true state of the image (e.g., 1 for real, 0 for fake). By minimizing the error between the prediction result and the real label, the parameters of the image feature extraction network and the image face authentication network are adjusted layer by layer using the backpropagation algorithm, enabling the model to gradually learn discriminative features that can accurately distinguish between real and fake faces (such as photos, screen captures, AI face swaps, etc.).

[0081] In summary, semantic features of faces in sample images are extracted using an image feature extraction network to obtain the first facial semantic features. The image face authentication network then classifies faces based on these features, outputting a first predicted label indicating the probability of a face being real or fake. By comparing this predicted result with the first labeled label reflecting the true state, the difference between the two is calculated, and the model parameters are optimized through backpropagation. This achieves end-to-end training of the image face authentication model. Thus, the model can effectively learn the deep differences in semantic features between real faces and various fake faces (such as printed photos, screen playback, AI face swaps, etc.), constructing a face authenticity recognition model with strong discriminative power, significantly improving the model's recognition accuracy and generalization ability. Simultaneously, based on the loss calculation between the probability output and the true label, the model is continuously iterated and optimized through large-scale data, improving convergence and stability.

[0082] To clearly illustrate how the video face authentication model is trained in the above embodiments, this disclosure proposes another face authentication method.

[0083] Figure 4 This is a schematic flowchart of the face authentication method shown in the fourth embodiment of this disclosure. It should be noted that the video face authentication model includes an image feature extraction network, a feature transfer network, and a video face authentication network from a trained image face authentication model.

[0084] like Figure 4 As shown, this facial authentication method includes the following steps:

[0085] Step 401: Use a frozen image feature extraction network to extract semantic features from the faces in multiple video frames of the sample video to obtain multiple second facial semantic features.

[0086] In order to utilize the discriminative power of the pre-trained model to stably output high-quality facial features while reducing training complexity, in this embodiment of the present disclosure, an image feature extraction network trained and with fixed parameters in the image face authentication model is used to extract semantic features of faces in multiple video frames in the sample video to obtain high-level semantic features of each video frame.

[0087] The determination of multiple video frames can be achieved through the following steps:

[0088] (1) Sample video frames from the sample video;

[0089] (2) Perform face detection on each sampled video frame to identify multiple video frames containing faces from the sampled video frames.

[0090] Step 402: A feature transfer network is used to fuse multiple second facial semantic features based on learnable category vectors to obtain sample fusion features.

[0091] Among them, the sample fusion feature is used to indicate the inconsistency between multiple second facial semantic features.

[0092] It is important to understand that the second facial semantic features are spatial discrimination information such as texture and edge learned under static image conditions. In order to improve the spatial discrimination and temporal perception capabilities of the video face fake detection model, a feature transfer network is introduced to map the semantic features that originally belong to the image domain to the video domain representation space suitable for video understanding, while learning the inconsistencies between multiple frames of facial images in the video.

[0093] As an example, a feature transfer network receives multiple second facial semantic features arranged chronologically. Then, based on a learnable class vector (also known as cls_token), these features are fused to obtain sample fusion features. This process not only integrates global information from the entire video segment but also explicitly models the consistency or inconsistency between frames. For instance, real faces typically exhibit natural and coherent movement patterns, while fake videos (such as AI face-swapping) often show anomalies such as skipped expressions, misaligned mouth shapes, and flickering skin textures. Therefore, sample fusion features can effectively capture such dynamic anomalies, becoming a key basis for judging the authenticity of videos.

[0094] Step 403: The first sub-network of the video face authentication network is used to classify the faces in the sample video as real or fake based on the sample fusion features to obtain the second predicted label.

[0095] The second prediction label is used to indicate the probability that a face in the sample video is a real face or a fake face; the second annotation label is used to indicate the authenticity of a face in the sample video.

[0096] Furthermore, after obtaining the sample fusion features, the first sub-network (e.g., an MLP network) in the video face authentication network is used to analyze these features and output a second predicted label, which is the probability value of whether the face in the sample video is real or fake (e.g., real: 12%, fake: 88%). This probability reflects the model's comprehensive judgment on the authenticity of the entire video segment.

[0097] Step 404: Train the video face authentication model based on the difference between the second predicted label and the second labeled label corresponding to the sample video.

[0098] Then, the second predicted label is compared with the second manually labeled label (i.e., the true category of the video, such as "real" or "fake"), the classification loss (such as cross-entropy loss) is calculated, and the model parameters are optimized through backpropagation to minimize the classification loss.

[0099] To improve the model's ability to identify fake categories, in this embodiment of the disclosure, during the video face authentication process, when the model determines that the face in the sample video belongs to a fake face based on the second predicted label, the second sub-network in the video face authentication network is further used to identify the fake category based on the generated sample fusion features, and outputs a third predicted label, which is the probability distribution of the fake face belonging to multiple candidate fake categories. In addition, during the training phase, a joint loss function can also be used for optimization, that is, based on the difference between the second predicted label and the second labeled label (such as the true / false classification loss), and based on the difference between the third predicted label and the third labeled label (i.e., the true / false category) (such as the category recognition loss), the parameters of the feature transfer network and the video face authentication network are updated by backpropagation.

[0100] In summary, by employing a frozen image feature extraction network to extract semantic features from multiple frames of sample videos, redundant training is avoided, significantly reducing computational overhead. A feature transfer network adapts static image features to the video domain, and the transferred features from multiple frames are fused, enabling the fused features to explicitly reflect dynamic inconsistencies between frames, thereby enhancing the ability to detect facial forgery. Furthermore, the first sub-network of the video face authentication network is used for authenticity classification, outputting a probabilistic second predicted label to achieve accurate judgment of video-level facial authenticity. Supervised training is conducted by comparing the predicted label with the second predicted label, driving continuous model optimization. This achieves knowledge transfer and optimization from image to video, improving the model's detection accuracy and robustness in complex scenarios.

[0101] To clearly illustrate that the above embodiments involve inputting facial visual data into a multi-task facial authentication model to obtain the facial authentication result of the face to be detected in the facial visual data output by the multi-task facial authentication model, this disclosure proposes another facial authentication method.

[0102] Figure 5 This is a schematic flowchart of the facial authentication method shown in the fifth embodiment of this disclosure.

[0103] like Figure 5 As shown, this facial authentication method includes the following steps:

[0104] Step 501: Obtain the multi-task face authentication model.

[0105] The multi-task face authentication model is generated based on the image face authentication model and the video face authentication model.

[0106] Step 502: Obtain the visual data of the face to be detected.

[0107] Facial visual data includes image facial data or video facial data.

[0108] Step 503: In response to the facial visual data including image facial data, the image feature extraction network in the multi-task facial authentication model is used to extract semantic features of the face to be detected in the image facial data to obtain the third facial semantic features.

[0109] As an example, when the facial visual data input to the multi-task face authentication model includes image facial data, the image feature extraction network of the multi-task face authentication model is invoked to extract the semantic features of the face to be detected in the image, thus obtaining the third facial semantic features.

[0110] As an example, when the received facial visual data is image facial data (such as static face photos or screenshots), the image processing branch in the multi-task face authentication model will be automatically triggered.

[0111] In this embodiment of the disclosure, the image feature extraction network of the multi-task face authentication model is first invoked to extract high-dimensional semantic features from the facial region to be detected located in the image, thereby obtaining the third facial semantic features.

[0112] Step 504: Using the image face authentication network in the multi-task face authentication model, the face to be detected is classified as real or fake based on the third facial semantic features, so as to obtain the first probability that the face to be detected belongs to the real face and / or the second probability that the face to be detected belongs to the fake face.

[0113] Furthermore, after obtaining the third facial semantic features of the face to be detected, the image face authentication network in the multi-task face authentication model is used to classify the authenticity based on the third facial semantic features, and outputs the first probability that the face to be detected in the image face data belongs to the real face and / or the second probability that the face to be detected belongs to the fake face.

[0114] Step 505: In response to the facial visual data, including video facial data, the image feature extraction network in the multi-task facial authentication model is used to extract semantic features of the face to be detected in multiple video frames of the video facial data to obtain multiple fourth facial semantic features.

[0115] As another example, when the input facial visual data is video facial data, its modality is automatically identified and the video processing branch in the multi-task facial deception model is enabled.

[0116] In this embodiment of the disclosure, the image feature extraction network integrated in the model is first used to extract high-dimensional semantic features of the detected face located in the video image, thereby obtaining multiple fourth facial semantic features.

[0117] It should be noted that the semantic features of each frame are high-dimensional vectors, which can contain discriminative information related to the realism of the face in that frame, such as texture, structure, and lighting. Since the image feature extraction network has been fully trained and frozen on large-scale image data, it can output high-quality facial features.

[0118] Step 506: The feature transfer network in the multi-task face authentication model is used to fuse multiple fourth facial semantic features based on the learned category vectors to obtain video fusion features.

[0119] Among them, video fusion features are used to indicate inconsistencies between multiple video migration features.

[0120] It is important to understand that since image features mainly focus on the spatial structure information of a single frame, while video authentication needs to perceive the changing patterns in the time dimension, in order to accurately determine the authenticity of faces in a video, a feature transfer network in a multi-task face authentication model is used when processing video face data. Based on the learned category vectors, multiple fourth facial semantic features extracted from multiple video frames are fused into a unified video fusion feature. This fusion process not only preserves the key appearance information of each frame, but also models the differences or inconsistencies between frames. For example, real faces present a smooth transition in continuous movements, while fake videos (such as AI face swapping, screen playback) show abnormalities such as boundary jitter, texture flickering, and stiff expressions, resulting in unnatural feature changes between frames.

[0121] Step 507: The first sub-network of the video face authentication network in the multi-task face authentication model is used to classify the face to be detected in the video face data as real or fake based on video fusion features, so as to obtain the first probability that the face to be detected in the video face data belongs to a real face and / or the second probability that the face to be detected belongs to a fake face.

[0122] In this embodiment of the disclosure, after obtaining the video fusion features, the first sub-network of the video face authentication network in the multi-task face authentication model is invoked. Based on the features, the face to be detected in the video is classified as real or fake. The first sub-network can be used for video-level authenticity judgment and outputs the first probability that the face to be detected in the video face data belongs to a real face and / or the second probability that the face to be detected belongs to a fake face (the sum of the first probability and the second probability is 1).

[0123] Furthermore, in response to the fact that the second probability of the detected face belonging to a fake face is greater than the first probability of the detected face belonging to a real face, the second sub-network in the video face authentication network is used to classify the detected face into a fake category based on video fusion features, so as to obtain the third probability that the detected face belongs to multiple candidate fake categories.

[0124] In other words, when the second probability of determining that the face to be detected belongs to a fake face is greater than the first probability that the face to be detected belongs to a real face, the face to be detected in the video face data is determined to be a fake face. At this time, the second sub-network in the video face fakeness detection network is further used to identify the fakeness category and generate a third probability, that is, the probability distribution of the fake face belonging to each candidate fakeness category.

[0125] In summary, by constructing a unified multi-task facial authentication model, adaptive processing and high-precision authenticity recognition of facial data in both image and video modalities are achieved. When the input is an image, a third facial semantic feature is extracted using an image feature extraction network, and the image facial authentication network outputs the classification probability of authenticity and forgery, achieving fast and accurate static authentication. When the input is a video, the same image feature extraction network is reused to extract features from multiple frames, and then a feature transfer network is used to fuse the static images to obtain video fusion features containing inter-frame inconsistencies. The first sub-network of the video facial authentication network outputs the video-level authenticity probability, which reduces model redundancy and deployment costs while ensuring feature consistency between image and video branches. At the same time, the introduction of a feature transfer network for video data significantly improves the accuracy and robustness of video facial forgery detection, making it suitable for identity authentication and security protection needs in multiple scenarios.

[0126] Based on the above embodiments, taking a human face as an example, the face authentication method of this disclosure can also be implemented based on the following steps. It should be noted that the face authentication method proposed in this disclosure combines the spatial information of the image and the temporal information of multiple frames to authenticate video faces. In addition, the face authentication method proposed in this disclosure can be applied to both image face authentication tasks and video face authentication tasks.

[0127] The specific steps are as follows:

[0128] The first step is to train the image facial authentication model.

[0129] (1) Constructing image face training and validation sets

[0130] 1. The training and validation sets contain real faces as well as fake faces generated by multiple forgery methods. The ratio of real faces to fake faces in the training and validation sets should be close to 1:1, and it is recommended not to exceed 1:2.

[0131] 2. Based on the face detection model, extract the face from the original face image to obtain the initial face detection box with coordinates (x1, y1, x2, y2);

[0132] 3. Enlarge the length and width of the extracted initial face detection box by a factor of 1.3 to obtain the target face detection box with coordinates (x1′, y1′, x′2, y′2);

[0133] 4. Based on the coordinates of the target face detection box, the face is cropped from the original face image to obtain the face training set and validation set.

[0134] (2) Constructing an image face authentication model

[0135] 1. Select the vit-b16 (a feature extraction network) network as the backbone network of the image face authentication model to extract high-dimensional semantic information of the face;

[0136] 2. An MLP network is selected as the task head (i.e., the image face authentication network) for the image face authentication model. The specific structure is as follows: Figure 6 As shown; the MLP network consists of two nonlinear layers and one linear layer, where, as Figure 7 As shown, the nonlinear layer consists of a linear layer, a batch normalization layer (BN), and an activation layer (ReLU).

[0137] (3) Training of Image Face Authentication Model

[0138] In this embodiment of the disclosure, cross-entropy loss is used to train the image face authentication model based on the training set, and the effectiveness of the hyperparameters selected when training the model and the degree of fit of the model to the training set are verified based on the validation set.

[0139] The second step is to train the image face authentication model.

[0140] (1) Constructing video face training and validation sets

[0141] 1. The training and validation sets contain real face videos as well as fake face videos generated by multiple forgery methods. The ratio of real face videos to fake face videos in the training and validation sets should be close to 1:1, and it is recommended not to exceed 1:2.

[0142] 2. For each video in the training and validation sets, 16 frames of images are sampled from the sample video (also known as the original video) using a sampling method (sampling methods are diverse and are not limited here). The sampling of the sample video is mainly to improve the processing speed of the fake detection model.

[0143] 3. For each frame of each video, the face can be extracted from the original image using the methods described in the steps above;

[0144] 4. The training set is ultimately abbreviated as: Where s i This represents the face image corresponding to each image frame sampled from the video, where n represents the number of videos in the training set.

[0145] (2) Constructing a video face authentication model

[0146] The principle of video face authentication model is as follows: Figure 8 As shown, the details are as follows:

[0147] 1. Image Feature Extraction Network: The image feature extraction network (e.g., vit-b16 backbone network) in the image face authentication model is used as the image feature extraction network in the video face authentication model. It extracts high-level semantic information from single frames of images in the video. The parameters of the image feature extraction network are frozen during the training of the video face authentication model and no parameter updates are made. The input of the image feature extraction network is the face image, and the output is the high-level semantic features corresponding to the face image.

[0148] 2. Feature Transfer Network (also known as adapter network): Since the parameters of the image feature extractor are frozen during the training of a video face authentication model, a feature transfer network was proposed to transfer the extracted facial features to a video dataset. The feature transfer network consists of two transformer blocks, which not only transfer the features extracted by the image feature extraction network but also learn the inconsistencies between multiple frames of face images in a video. The input to the feature transfer network is the high-level semantic features corresponding to multiple frames of face images, denoted as f1, f2, ... f 16 And learnable category tokens (denoted as f) cls The output is the global feature vector f. global (Also known as sample fusion features), it integrates high-level semantic information from multiple frames of face images and learns the inconsistencies of multiple frames of face images;

[0149] 3. Task Head Construction: The task head of the video fake-detection model (also known as the video face fake-detection network) includes a binary classification task head (also known as the first sub-network) and a multi-classification task head (also known as the second sub-network). The binary classification task head is used to determine whether the input video is real, and the multi-classification head is used to determine which fake category the input video belongs to when it is fake. Both the binary classification head and the multi-classification head are MLP models, and their specific structures are as follows: Figure 6 As shown; the input to the binary classification task head is the global feature vector (referring to f). global The input of the multi-class task head is a global feature vector, and the output is a multi-dimensional vector. After softmax encoding, it represents the probability that the input video is real or fake. The input of the multi-class task head is a global feature vector, and the output is a multi-dimensional vector. After sigmoid activation, it represents the probability that the input video is a certain type of fake. The output dimension here depends on how many fake categories are included in the training set, such as AI face swapping, face reconstruction, lip editing, etc.

[0150] (3) Training of video authentication model

[0151] In this embodiment of the disclosure, the adapter network and the task head network in the video authentication model are trained using a video training set based on cross-entropy loss.

[0152] The third step involves reasoning using a combined image and video facial authentication model.

[0153] In the embodiments disclosed herein, such as Figure 9 As shown, the multi-task face authentication model includes: image feature extraction network, image face authentication network, feature transfer network, and video face authentication network.

[0154] Among them, the image feature extraction network: Since the backbone network of the image face authentication model is frozen when training the video authentication model, the same backbone network can be used for high-level feature semantic extraction for both image faces and video faces.

[0155] The main function of the feature transfer network is to transfer the high-level semantic features extracted by the pre-trained image feature extraction network to the video domain and learn the inconsistencies between multiple frames of face images obtained by video sampling.

[0156] Image face authentication network: Based on the output features of an image feature extraction network, it determines whether the input face image is real;

[0157] Video face authentication network: The video face authentication network includes a binary classification task head and a multi-classification task head;

[0158] Furthermore, when the input data (facial visual data to be detected) is an image of a face, it passes through an image feature extraction network and an image face authentication network to output the probability that the face is real or fake. When the input data is a video, the original video is first downsampled to obtain video frames. Then, it passes through an image feature extractor to obtain high-level semantic information corresponding to the video frames. Next, it passes through an adapter module to obtain fused high-level semantic features. Finally, it passes through a binary classification task head and a multi-classification task head to obtain the probability that the original video is real and the probability that the video belongs to a certain fake category when it is fake.

[0159] Corresponding to the facial authentication method provided in the above embodiments, this disclosure also provides a facial authentication device. Since the facial authentication device provided in this disclosure corresponds to the facial authentication method provided in the above embodiments, the implementation of the facial authentication method is also applicable to the facial authentication device provided in this disclosure, and will not be described in detail in this disclosure.

[0160] Figure 10 This is a schematic diagram of the structure of the facial authentication device shown in the sixth embodiment of this disclosure.

[0161] like Figure 10 As shown, the facial authentication device 1000 includes: a first acquisition module 1010, a second acquisition module 1020, and an authentication module 1030.

[0162] The first acquisition module 1010 is used to acquire a multi-task face authentication model, which is generated based on an image face authentication model and a video face authentication model. The second acquisition module 1020 is used to acquire the facial visual data to be detected, which includes image facial data or video facial data. The authentication module 1030 is used to input the facial visual data into the multi-task face authentication model to obtain the face authentication result of the face to be detected in the facial visual data output by the multi-task face authentication model. The face authentication result includes at least one of the following: a first probability that the face to be detected belongs to a real face, a second probability that the face to be detected belongs to a fake face, and a third probability that the face to be detected belongs to multiple candidate fake categories when the second probability is greater than the first probability.

[0163] As one possible implementation of this disclosure, the multi-task face authentication model is generated using the following modules: a third acquisition module, a first training module, and a generation module.

[0164] The third acquisition module is used to acquire a trained image face authentication model, which includes an image feature extraction network and an image face authentication network. The first training module is used to freeze the image feature extraction network in the trained image face authentication model to train the feature transfer network and the video face authentication network in the video face authentication model. The feature transfer network is used to fuse image features based on category vectors. The generation module is used to generate a multi-task face authentication model based on the trained image feature extraction network, the trained image face authentication network, the trained feature transfer network, and the trained video face authentication network.

[0165] As one possible implementation of this disclosure, the image face authentication model is trained using the following modules: a first extraction module, a first classification module, and a second training module.

[0166] The system comprises: a first extraction module for extracting semantic features from faces in a sample image using the image feature extraction network to obtain first facial semantic features; a first classification module for classifying faces in a sample image as real or fake based on the first facial semantic features using the image face authentication network to obtain a first predicted label; wherein the first predicted label indicates the probability that a face in the sample image is a real face or a fake face; and a second training module for training an image face authentication model based on the difference between the first predicted label and the first annotation label corresponding to the sample image; wherein the first annotation label indicates the authenticity of a face in the sample image.

[0167] As one possible implementation of this disclosure, the face authentication device 1000 further includes a fourth acquisition module.

[0168] The fourth acquisition module is used to acquire the original facial image, which includes a real face or a fake face; to perform facial key point detection on the original facial image to obtain an initial facial detection box; to resize the initial facial detection box in the original facial image to obtain a target facial detection box; and to crop a sample image from the original facial image based on the target facial detection box.

[0169] As one possible implementation of this disclosure, the video face authentication model includes an image feature extraction network, a feature transfer network, and a video face authentication network in a trained image face authentication model. The video face authentication model is trained using the following modules: a second extraction module, a transfer module, a second classification module, and a third training module.

[0170] The system comprises the following modules: a second extraction module, which uses a frozen image feature extraction network to extract semantic features from faces in multiple frames of the sample video to obtain multiple second facial semantic features; a transfer module, which uses a feature transfer network to fuse the multiple second facial semantic features based on learnable category vectors to obtain sample fusion features, wherein the sample fusion features are used to indicate inconsistencies among the multiple second facial semantic features; a second classification module, which uses the first sub-network of the video face authentication network to classify faces in the sample video as real or fake based on the sample fusion features to obtain a second predicted label, wherein the second predicted label is used to indicate the probability that a face in the sample video is a real face or a fake face; and a second annotation label is used to indicate the authenticity of a face in the sample video; and a third training module, which trains the video face authentication model based on the difference between the second predicted label and the second annotation label corresponding to the sample video.

[0171] As a possible implementation of this disclosure, the third training module is further configured to, when determining that a face in a sample video belongs to a fake face based on the second predicted label, use the second sub-network of the video face authentication network to classify the face in the sample video into a fake category based on sample fusion features to obtain a third predicted label; wherein, the third predicted label is used to indicate the probability of belonging to multiple candidate fake categories; the video face authentication model is trained based on the difference between the second predicted label and the second labeled label, and the difference between the third predicted label and the third labeled label corresponding to the sample video; wherein, the third labeled label is used to indicate the fake category to which the face belongs when it is a fake face.

[0172] As one possible implementation of this disclosure, the anti-counterfeiting module 1030 is used to, in response to facial visual data including image facial data, use an image feature extraction network in a multi-task facial anti-counterfeiting model to extract semantic features of the face to be detected in the image facial data to obtain a third facial semantic feature; and use the image facial anti-counterfeiting network in the multi-task facial anti-counterfeiting model to classify the face to be detected as real or fake based on the third facial semantic feature to obtain a first probability that the face to be detected belongs to a real face and / or a second probability that the face to be detected belongs to a fake face.

[0173] As one possible implementation of this disclosure, the anti-spoofing module 1030 is configured to, in response to facial visual data including video facial data, use an image feature extraction network in a multi-task facial anti-spoofing model to extract semantic features of the face to be detected in multiple video frames of the video facial data to obtain multiple fourth facial semantic features; use a feature transfer network in the multi-task facial anti-spoofing model to fuse the multiple fourth facial semantic features based on learned category vectors to obtain video fusion features; wherein, the video fusion features are used to indicate inconsistencies among the multiple fourth facial semantic features; and use a first sub-network in the video facial anti-spoofing network of the multi-task facial anti-spoofing model to classify the face to be detected in the video facial data as real or fake based on the video fusion features to obtain a first probability that the face to be detected in the video facial data belongs to a real face and / or a second probability that the face to be detected belongs to a fake face.

[0174] As one possible implementation of this disclosure, the anti-spoofing module 1030 is further configured to, in response to the second probability that the detected face belongs to a fake face being greater than the first probability that the detected face belongs to a real face, use the second sub-network in the video face anti-spoofing network to classify the detected face into a forgery category based on video fusion features, so as to obtain a third probability that the detected face belongs to multiple candidate forgery categories.

[0175] The facial authentication device of this disclosure constructs a unified multi-task facial authentication model, integrating the discrimination capabilities of image facial authentication models and video facial authentication models. This allows a single model to process both image and video facial data, significantly reducing system deployment complexity and computational resource consumption, and improving operational efficiency and maintainability. Furthermore, at the output level, the model not only provides two probabilities (i.e., a first probability and a second probability) for whether the detected face is real or fake, achieving accurate discrimination of facial authenticity, but also outputs a third probability in cases where the face is determined to be fake, i.e., the probability that the detected face belongs to multiple candidate fake categories, achieving refined identification of fake types. This mechanism not only enhances the interpretability and practicality of the authentication results but also helps the system adopt differentiated defense strategies for different types of facial fakes, significantly improving the system's security protection capabilities.

[0176] In an exemplary embodiment, an electronic device is also proposed.

[0177] The electronic devices include:

[0178] processor;

[0179] Memory used to store processor-executable instructions;

[0180] The processor is configured to execute instructions to implement the face authentication method as proposed in any of the foregoing embodiments.

[0181] As an example, Figure 11 This is a schematic diagram of the structure of an electronic device 1100 as shown in an exemplary embodiment of this disclosure, as follows: Figure 11 As shown, the above-mentioned electronic device 1100 may further include:

[0182] The memory 1110 and processor 1120 are connected by a bus 1130, which connects different components (including the memory 1110 and the processor 1120). The memory 1110 stores a computer program, which implements the face authentication method described in this embodiment when the processor 1120 executes the program.

[0183] Bus 1130 represents one or more of several bus architectures, including a memory bus or memory controller, a peripheral bus, a graphics acceleration port, a processor, or a local bus using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0184] Electronic device 1100 typically includes a variety of electronic device readable media. These media can be any available media that can be accessed by electronic device 1100, including volatile and non-volatile media, removable and non-removable media.

[0185] Memory 1110 may also include computer system readable media in the form of volatile memory, such as random access memory (RAM) 1140 and / or cache memory 1150. Electronic device 1100 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 1160 may be used to read and write non-removable, non-volatile magnetic media (… Figure 11 Not shown; usually referred to as a "hard drive"). Although Figure 11 Not shown, a disk drive for reading and writing to a removable non-volatile disk (e.g., a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (e.g., a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 1130 via one or more data media interfaces. Memory 1110 may include at least one program product having a set (e.g., at least one) of program modules configured to perform the functions of the embodiments of this disclosure.

[0186] A program / utility 1180 having a set (at least one) of program modules 1170 may be stored, for example, in memory 1110. Such program modules 1170 include, but are not limited to, an operating system, one or more application programs, other program modules, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 1170 typically perform the functions and / or methods described in the embodiments of this disclosure.

[0187] Electronic device 1100 can also communicate with one or more external devices 1190 (e.g., keyboard, pointing device, display 1191, etc.), and with one or more devices that enable a user to interact with electronic device 1100, and / or with any device that enables electronic device 1100 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed via input / output (I / O) interface 1192. Furthermore, electronic device 1100 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1193. As shown, network adapter 1193 communicates with other modules of electronic device 1100 via bus 1130. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 1100, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0188] The processor 1120 performs various functional applications and data processing by running programs stored in the memory 1110.

[0189] It should be noted that the implementation process and technical principles of the electronic device in this embodiment are explained in the foregoing description of the facial authentication method of this disclosure, and will not be repeated here.

[0190] In an exemplary embodiment, a computer-readable storage medium including instructions is also provided, such as a memory including instructions, which can be executed by a processor of an electronic device to perform the facial authentication method proposed in any of the above embodiments. Optionally, the computer-readable storage medium may be a ROM, random access memory (RAM), CD-ROM, magnetic tape, floppy disk, and optical data storage device, etc.

[0191] In an exemplary embodiment, a computer program product is also provided, including a computer program / instructions, characterized in that the computer program / instructions, when executed by a processor, implement the facial authentication method proposed in any of the above embodiments.

[0192] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

[0193] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is limited only by the appended claims.

Claims

1. A facial authentication method, characterized in that, include: A multi-task face authentication model is obtained; wherein the multi-task face authentication model is generated based on the image face authentication model and the video face authentication model. Acquire the facial visual data to be detected; wherein, the facial visual data includes image facial data or video facial data; The facial visual data is input into a multi-task facial authentication model to obtain the facial authentication result of the face to be detected in the facial visual data output by the multi-task facial authentication model; wherein, the facial authentication result includes at least one of the following: a first probability that the face to be detected belongs to a real face, a second probability that the face to be detected belongs to a fake face, and a third probability that the face to be detected belongs to multiple candidate fake categories when the second probability is greater than the first probability.

2. The method according to claim 1, characterized in that, The multi-task face authentication model is generated using the following steps: Obtain a trained image face authentication model; wherein the image face authentication model includes an image feature extraction network and an image face authentication network; The image feature extraction network in the trained image face authentication model is frozen to train the feature transfer network and the video face authentication network in the video face authentication model; wherein, the feature transfer network is used to perform feature fusion of image features based on the category vector; The multi-task face authentication model is generated based on the trained image feature extraction network, the trained image face authentication network, the trained feature transfer network, and the trained video face authentication network.

3. The method according to claim 2, characterized in that, The image facial authentication model is trained using the following steps: The image feature extraction network is used to extract semantic features from the face in the sample image to obtain the first facial semantic features; The image face authentication network uses the first facial semantic features to classify the faces in the sample image as real or fake to obtain a first predicted label; wherein, the first predicted label is used to indicate the probability that the face in the sample image belongs to a real face or the probability that it belongs to a fake face. The image face authentication model is trained based on the difference between the first predicted label and the first annotation label corresponding to the sample image; wherein, the first annotation label is used to indicate the authenticity of the face in the sample image.

4. The method according to claim 3, characterized in that, Before using the image feature extraction network to extract semantic features from the face in the sample image to obtain the first facial semantic features, the method further includes: Obtain the original facial image; wherein the original facial image includes a real face or a fake face; Facial key point detection is performed on the original facial image to obtain an initial facial detection box; The initial face detection bounding box in the original face image is resized to obtain the target face detection bounding box; The sample image is cropped from the original facial image based on the target face detection bounding box.

5. The method according to claim 2, characterized in that, The video face authentication model includes an image feature extraction network, a feature transfer network, and a video face authentication network from a trained image face authentication model. The video face authentication model is trained using the following steps: A frozen image feature extraction network was used to extract semantic features from the faces in multiple frames of the sample video to obtain multiple second facial semantic features; The feature transfer network is used to fuse the multiple second facial semantic features based on learnable category vectors to obtain sample fusion features; wherein, the sample fusion features are used to indicate the inconsistencies among the multiple second facial semantic features; The first sub-network of the video face authentication network classifies faces in the sample video as real or fake based on the sample fusion features to obtain a second predicted label; wherein, the second predicted label is used to indicate the probability that a face in the sample video is a real face or the probability that it is a fake face. The video face authentication model is trained based on the difference between the second predicted label and the second annotation label corresponding to the sample video; wherein the second annotation label is used to indicate the authenticity of the face in the sample video.

6. The method according to claim 5, characterized in that, The step of training the video face authentication model based on the difference between the second predicted label and the second annotation label corresponding to the sample video includes: If the face in the sample video is determined to be a fake face based on the second predicted label, the second sub-network of the video face authentication network is used to classify the face in the sample video into a fake category based on the sample fusion features to obtain a third predicted label; wherein, the third predicted label is used to indicate the probability of belonging to multiple candidate fake categories; The video face authentication model is trained based on the difference between the second predicted label and the second annotation label, and the difference between the third predicted label and the third annotation label corresponding to the sample video; wherein the third annotation label is used to indicate the forgery category to which the face belongs when it is a forged face.

7. The method according to claim 1, characterized in that, The step of inputting the facial visual data into a multi-task facial authentication model to obtain the facial authentication result of the face to be detected in the facial visual data output by the multi-task facial authentication model includes: In response to the fact that the facial visual data includes image facial data, the image feature extraction network in the multi-task facial authentication model is used to extract semantic features of the face to be detected in the image facial data to obtain a third facial semantic feature. The image face authentication network in the multi-task face authentication model uses the third facial semantic features to classify the face to be detected as real or fake, so as to obtain a first probability that the face to be detected belongs to a real face and / or a second probability that the face to be detected belongs to a fake face.

8. The method according to claim 1, characterized in that, The step of inputting the facial visual data into a multi-task facial authentication model to obtain the facial authentication result of the face to be detected in the facial visual data output by the multi-task facial authentication model includes: In response to the fact that the facial visual data includes video facial data, the image feature extraction network in the multi-task facial authentication model is used to extract semantic features of the face to be detected in multiple video frames of the video facial data to obtain multiple fourth facial semantic features. The feature transfer network in the multi-task face authentication model is used to fuse the multiple fourth facial semantic features based on the learned category vectors to obtain video fusion features; wherein, the video fusion features are used to indicate the inconsistencies among the multiple fourth facial semantic features. The first sub-network in the video face authentication network of the multi-task face authentication model is used to classify the face to be detected in the video face data as real or fake based on the video fusion features, so as to obtain a first probability that the face to be detected in the video face data belongs to a real face and / or a second probability that the face to be detected belongs to a fake face.

9. The method according to claim 8, characterized in that, The method further includes: In response to the fact that the second probability of the detected face being a fake face is greater than the first probability of the detected face being a real face, the second sub-network in the video face authentication network is used to classify the detected face into a fake category based on the video fusion features, so as to obtain a third probability that the detected face belongs to multiple candidate fake categories.

10. A facial authentication device, characterized in that, include: The first acquisition module is used to acquire a multi-task face authentication model; wherein the multi-task face authentication model is generated based on the image face authentication model and the video face authentication model. The second acquisition module is used to acquire the facial visual data to be detected; wherein, the facial visual data includes image facial data or video facial data; The anti-spoofing module is used to input the facial visual data into a multi-task facial anti-spoofing model to obtain the facial anti-spoofing result of the face to be detected in the facial visual data output by the multi-task facial anti-spoofing model; wherein, the facial anti-spoofing result includes at least one of the following: a first probability that the face to be detected belongs to a real face, a second probability that the face to be detected belongs to a fake face, and a third probability that the face to be detected belongs to multiple candidate fake categories when the second probability is greater than the first probability.

Images