Model training method, live face detection method, device, equipment and medium

Abstract

Embodiments of the present disclosure provide a model training method, a live face detection method, device, equipment and medium. The model training method comprises: performing self-supervised training on an initial face recognition model by using an unlabeled image dataset, so that the first extractor parameters of the first face recognition model obtained by training have the ability to extract general image data features; performing self-supervised training on the first face recognition model by using a first real live face dataset, so that the second extractor parameters of the second face recognition model obtained by training have the ability to extract face features; performing asymmetric metric learning on the second face recognition model by using a labeled real and fake face mixed dataset, so that the live face recognition model has the ability to judge real and fake faces. The method of the present disclosure significantly improves the ability of the live face recognition model to extract face features and the recognition ability, and can identify fake faces of unknown attack types.

Description

Technical Field

[0001] The embodiments of this disclosure relate to the field of image processing technology, and more specifically, the embodiments of this disclosure relate to model training methods, live face detection methods, apparatus, devices, and media. Background Technology

[0002] This section is intended to provide background or context for the embodiments of this disclosure as set forth in the claims. The description herein is not intended to be a prior art simply because it is included in this section.

[0003] In recent years, facial recognition technology has been widely used in security, finance, and other scenarios requiring identity verification, such as electronic payments, access control, and account opening. However, because faces are easily copied using images, videos, models, or masks, incorporating liveness detection as a step in identity authentication is essential to prevent malicious actors from forging and stealing others' biometric features to impersonate them, effectively protecting users' information and property security.

[0004] Currently, labeled real face data and labeled dummy face data are typically used to supervise the training of deep learning networks to obtain liveness detection models. These models are then used to identify faces in images and achieve the goal of liveness detection. However, existing liveness detection models have weak recognition capabilities and cannot identify dummy faces with unknown attack types. Summary of the Invention

[0005] This disclosure provides a model training method, a live face detection method, an apparatus, a device, and a medium to address the problem that existing live face recognition models have weak recognition capabilities and cannot identify fake people with unknown attack types.

[0006] In a first aspect of this disclosure, a model training method is provided, comprising:

[0007] An unlabeled image dataset is used to perform self-supervised training on an initial face recognition model until the initial face recognition model reaches the training cutoff condition to obtain a corresponding first face recognition model. The initial face recognition model includes a feature extractor, and the model parameters of the first face recognition model are first parameters, which include first extractor parameters corresponding to the feature extractor.

[0008] Using a first real live face dataset, and with the first extractor parameters as the initial parameters of the feature extractor, the first face recognition model is subjected to self-supervised training until the first face recognition model reaches the training cutoff condition, so as to obtain a corresponding second face recognition model. The model parameters of the second face recognition model are second parameters, and the second parameters include the second extractor parameters corresponding to the feature extractor.

[0009] A labeled dataset of real and fake faces is used. The parameters of the second extractor are used as the initial parameters of the feature extractor. The second face recognition model is subjected to asymmetric metric learning until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model. The model parameters of the live face recognition model are the third parameters, which include the third extractor parameters corresponding to the feature extractor.

[0010] In one embodiment of this disclosure, the step of using a labeled mixed dataset of real and fake faces, using the parameters of the second extractor as the initial parameters of the feature extractor, and performing asymmetric metric learning on the second face recognition model until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model includes:

[0011] Using the labeled real and fake face mixed dataset, and taking the parameters of the second extractor as the initial parameters of the feature extractor, asymmetric metric learning is performed on the second face recognition model containing the classifier and the feature extractor until the second face recognition model reaches the training cutoff condition, so as to obtain the live face recognition model.

[0012] In another embodiment of this disclosure, the second face recognition model further includes a feature clusterer, the feature extractor is connected to the classifier and the feature clusterer respectively, and the loss function of the second face recognition model includes a first sub-loss function corresponding to the classifier and a second sub-loss function corresponding to the feature clusterer.

[0013] In yet another embodiment of this disclosure, during the asymmetric metric learning process of the second face recognition model, the loss function of the second face recognition model can be dynamically determined according to the following process:

[0014] The first preset weight value is determined as the weight value of the first sub-loss function;

[0015] Calculate the ratio of the first sub-loss function value to the second sub-loss function value;

[0016] The smaller of the ratio and the second preset weight value is determined as the weight value of the second sub-loss function;

[0017] Multiply the first sub-loss function by the corresponding weight value to obtain the first product;

[0018] Multiply the second sub-loss function by the corresponding weight value to obtain the second product;

[0019] The sum of the first product and the second product is determined as the loss function.

[0020] In another embodiment of this disclosure, before the first face recognition model is trained using a first real live face dataset and the first extractor parameters as the initial parameters of the feature extractor, until the first face recognition model reaches the training cutoff condition to obtain the corresponding second face recognition model, the method further includes:

[0021] Using a face detection algorithm, the real face regions in the initial real live face dataset are cropped, and the cropped real face region images are determined as the first real live face dataset.

[0022] In another embodiment of this disclosure, before using a labeled mixed dataset of real and fake faces, and using the parameters of the second extractor as the initial parameters of the feature extractor to perform asymmetric metric learning on the second face recognition model until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model, the method further includes:

[0023] Using a face detection algorithm, the real face regions in the initial real live face dataset are cropped, and corresponding labels are generated for the cropped real face region images.

[0024] The face detection algorithm is used to crop out the fake face regions in the initial fake face dataset and generate corresponding labels for the cropped fake face region images.

[0025] Data augmentation processing is performed on labeled real human face region images and labeled fake human face region images to obtain the labeled real and fake face hybrid dataset.

[0026] In a second aspect of this disclosure, a live face detection method is provided, comprising:

[0027] Acquire the image of the face to be identified;

[0028] The face image to be identified is input into a live face recognition model to obtain a recognition result. The live face recognition model is trained according to the method described in the first aspect. The recognition result is used to indicate whether the face image to be identified is a real face.

[0029] In one embodiment of this disclosure, before inputting the face image to be identified into a liveness detection model to obtain the recognition result, the method further includes:

[0030] Using a face detection algorithm, the real face regions in the initial real live face dataset are cropped, and the cropped real face region images are determined as the second real live face dataset.

[0031] Based on the feature extractor of the live face recognition model, extract the features of each image in the second real live face dataset;

[0032] The features of each image in the second real live face dataset are calculated to determine the real face baseline features.

[0033] In another embodiment of this disclosure, the step of calculating the features of each image in the second real live face dataset to determine the real face baseline features includes:

[0034] Calculate the average feature value of each image in the second real live face dataset, and determine the average feature value as the real face baseline feature.

[0035] In another embodiment of this disclosure, the step of inputting the face image to be identified into a liveness detection model to obtain the recognition result includes:

[0036] The features of the face image to be identified are extracted using the feature extractor of the live face recognition model.

[0037] Calculate the feature distance between the features of the face image to be identified and the reference features of the real face;

[0038] The recognition result is generated based on the feature distance.

[0039] In yet another embodiment of this disclosure, generating the recognition result based on the feature distance includes:

[0040] When the feature distance is greater than a preset distance, a recognition result is generated to indicate that the face image to be identified is a fake face;

[0041] When the feature distance is less than or equal to the preset distance, a recognition result is generated to indicate that the face image to be identified is a real human face.

[0042] In another embodiment of this disclosure, calculating the feature distance between the features of the face image to be identified and the reference features of the real face includes:

[0043] The feature distance is obtained by calculating the Euclidean distance between the features of the face image to be identified and the reference features of the real face.

[0044] In yet another embodiment of this disclosure, the second real live face dataset is a subset of the first real live face dataset, which is a training set used for model training to obtain the live face recognition model.

[0045] In a third aspect of this disclosure, a model training apparatus is provided, comprising:

[0046] The first training module is used to perform self-supervised training on an initial face recognition model using an unlabeled image dataset until the initial face recognition model reaches the training cutoff condition to obtain a corresponding first face recognition model. The initial face recognition model includes a feature extractor, and the model parameters of the first face recognition model are first parameters, which include first extractor parameters corresponding to the feature extractor.

[0047] The second training module is used to perform self-supervised training on the first face recognition model using the first real live face dataset and the first extractor parameters as the initial parameters of the feature extractor, until the first face recognition model reaches the training cutoff condition, so as to obtain the corresponding second face recognition model. The model parameters of the second face recognition model are second parameters, and the second parameters include the second extractor parameters corresponding to the feature extractor.

[0048] The third training module is used to perform asymmetric metric learning on the second face recognition model using a labeled mixed dataset of real and fake faces, with the second extractor parameters as the initial parameters of the feature extractor, until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model. The model parameters of the live face recognition model are the third parameters, which include the third extractor parameters corresponding to the feature extractor.

[0049] In one embodiment of this disclosure, the third training module is specifically used for:

[0050] Using the labeled real and fake face mixed dataset, and taking the parameters of the second extractor as the initial parameters of the feature extractor, asymmetric metric learning is performed on the second face recognition model containing the classifier and the feature extractor until the second face recognition model reaches the training cutoff condition, so as to obtain the live face recognition model.

[0051] In another embodiment of this disclosure, the second face recognition model further includes a feature clusterer, the feature extractor is connected to the classifier and the feature clusterer respectively, and the loss function of the second face recognition model includes a first sub-loss function corresponding to the classifier and a second sub-loss function corresponding to the feature clusterer.

[0052] In another embodiment of this disclosure, the apparatus further includes a determining module, wherein during the asymmetric metric learning process of the second face recognition model, the loss function of the second face recognition model can be dynamically determined according to the determining module:

[0053] The first preset weight value is determined as the weight value of the first sub-loss function;

[0054] Calculate the ratio of the first sub-loss function value to the second sub-loss function value;

[0055] The smaller of the ratio and the second preset weight value is determined as the weight value of the second sub-loss function;

[0056] Multiply the first sub-loss function by the corresponding weight value to obtain the first product;

[0057] Multiply the second sub-loss function by the corresponding weight value to obtain the second product;

[0058] The sum of the first product and the second product is determined as the loss function.

[0059] In another embodiment of this disclosure, before the first face recognition model is trained using a first real live face dataset and the first extractor parameters as the initial parameters of the feature extractor, until the first face recognition model reaches the training cutoff condition to obtain the corresponding second face recognition model, the apparatus further includes:

[0060] The first cropping module is used to crop the real face regions in the initial real live face dataset using a face detection algorithm, and to determine the cropped real face region image as the first real live face dataset.

[0061] In another embodiment of this disclosure, before the second face recognition model is subjected to asymmetric metric learning using a labeled mixed dataset of real and fake faces, with the parameters of the second extractor as the initial parameters of the feature extractor, until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model, the apparatus further includes:

[0062] The second cropping module is used to crop the real face regions in the initial real live face dataset using a face detection algorithm, and generate corresponding labels for the cropped real face region images.

[0063] The third cropping module is used to crop the fake face region in the initial fake face dataset using the face detection algorithm, and generate corresponding labels for the cropped fake face region image;

[0064] The processing module is used to perform data augmentation processing on labeled real human face region images and labeled fake human face region images to obtain the labeled real and fake face mixed dataset.

[0065] In a fourth aspect of this disclosure, a live face detection device is provided, comprising:

[0066] The acquisition module is used to acquire the face image to be identified;

[0067] The input module is used to input the face image to be identified into the live face recognition model and obtain the recognition result. The live face recognition model is trained according to the method described in the first aspect. The recognition result is used to indicate whether the face image to be identified is a real face.

[0068] In one embodiment of this disclosure, before inputting the face image to be identified into the liveness detection model to obtain the recognition result, the device further includes:

[0069] The cropping module is used to crop the real face regions in the initial real live face dataset using a face detection algorithm, and to determine the cropped real face region image as the second real live face dataset.

[0070] The extraction module is used to extract features from each image in the second real live face dataset based on the feature extractor of the live face recognition model.

[0071] The calculation module is used to calculate the features of each image in the second real live face dataset to determine the real face baseline features.

[0072] In another embodiment of this disclosure, the computing module is specifically used for:

[0073] Calculate the average feature value of each image in the second real live face dataset, and determine the average feature value as the real face baseline feature.

[0074] In yet another embodiment of this disclosure, the acquisition module is specifically used for:

[0075] The features of the face image to be identified are extracted using the feature extractor of the live face recognition model.

[0076] Calculate the feature distance between the features of the face image to be identified and the reference features of the real face;

[0077] The recognition result is generated based on the feature distance.

[0078] In yet another embodiment of this disclosure, the acquisition module is specifically used for:

[0079] When the feature distance is greater than a preset distance, a recognition result is generated to indicate that the face image to be identified is a fake face;

[0080] When the feature distance is less than or equal to the preset distance, a recognition result is generated to indicate that the face image to be identified is a real human face.

[0081] In yet another embodiment of this disclosure, the acquisition module is specifically used for:

[0082] The feature distance is obtained by calculating the Euclidean distance between the features of the face image to be identified and the reference features of the real face.

[0083] In yet another embodiment of this disclosure, the second real live face dataset is a subset of the first real live face dataset, which is a training set used for model training to obtain the live face recognition model.

[0084] Fifthly, embodiments of this disclosure provide a computing device, including: a processor, and a memory communicatively connected to the processor;

[0085] The memory stores instructions that the computer executes;

[0086] The processor executes computer execution instructions stored in memory to implement the methods described in the first and second aspects of this disclosure.

[0087] In a sixth aspect, embodiments of this disclosure provide a storage medium storing computer program instructions, which, when executed, implement the methods described in the first and second aspects of this disclosure.

[0088] The model training method, live face detection method, apparatus, device, and medium disclosed in this embodiment include: a model training method comprising: self-supervised training of an initial face recognition model using an unlabeled image dataset, enabling the first extractor parameters of the trained first face recognition model to extract general image data features; further, self-supervised training of the first face recognition model using a first real live face dataset, enabling the second extractor parameters of the trained second face recognition model to extract face features; and finally, asymmetric metric learning of the second face recognition model using a labeled real / fake face mixed dataset, enabling the live face recognition model to distinguish between real and fake faces. This technical solution addresses the issue from two directions: self-supervised learning and asymmetric metric learning. By fully utilizing the ImageNet dataset and the first real live face dataset for self-supervised training, it significantly improves the ability of the second face recognition model to extract face features and its recognition capabilities, enabling the live face recognition model to identify fake faces of unknown attack types. The model training method provided by this technical solution can be transferred to face research-related fields such as liveness detection, person re-identification, and action recognition to improve the ability of deep models to extract facial features. Attached Figure Description

[0089] The above and other objects, features, and advantages of this disclosure will become readily apparent from the following detailed description of exemplary embodiments, taken in conjunction with the accompanying drawings. Several embodiments of this disclosure are illustrated in the drawings by way of example and not limitation, in which:

[0090] Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present disclosure;

[0091] Figure 2 A flowchart of a model training method provided in an embodiment of this disclosure;

[0092] Figure 3 This is a schematic diagram showing the feature distribution results of real human face data and fake human face data;

[0093] Figure 4 A flowchart of a model training method provided in another embodiment of this disclosure;

[0094] Figure 5 A flowchart of a model training method provided in yet another embodiment of this disclosure;

[0095] Figure 6 A flowchart of a model training method provided in yet another embodiment of this disclosure;

[0096] Figure 7 A flowchart of a live face detection method provided in an embodiment of this disclosure;

[0097] Figure 8 A flowchart of a live face detection method provided in another embodiment of this disclosure;

[0098] Figure 9 A flowchart of a live face detection method provided in yet another embodiment of this disclosure;

[0099] Figure 10 A flowchart of a live face detection method provided in yet another embodiment of this disclosure;

[0100] Figure 11 A schematic diagram of a storage medium provided in an embodiment of this disclosure;

[0101] Figure 12 This is a schematic diagram of the structure of a model training device provided in an embodiment of the present disclosure;

[0102] Figure 13 This is a schematic diagram of the structure of a live face detection device provided in an embodiment of the present disclosure;

[0103] Figure 14 This is a schematic diagram of the structure of a computing device provided in an embodiment of the present disclosure.

[0104] In the accompanying drawings, the same or corresponding reference numerals indicate the same or corresponding parts. Detailed Implementation

[0105] The principles and spirit of this disclosure will now be described with reference to several exemplary embodiments. It should be understood that these embodiments are given merely to enable those skilled in the art to better understand and implement this disclosure, and are not intended to limit the scope of this disclosure in any way. Rather, these embodiments are provided to make this disclosure more thorough and complete, and to fully convey the scope of this disclosure to those skilled in the art.

[0106] Those skilled in the art will recognize that embodiments of this disclosure can be implemented as a system, apparatus, device, method, or computer program product. Therefore, this disclosure can be specifically implemented in the following forms: entirely hardware, entirely software (including firmware, resident software, microcode, etc.), or a combination of hardware and software.

[0107] According to embodiments of this disclosure, a model training method, a live face detection method, an apparatus, a device, and a medium are proposed.

[0108] In this article, it is important to understand the following terms:

[0109] Self-supervised face recognition: assists in learning face-related tasks by mining supervisory information from large-scale face data to achieve specific tasks.

[0110] Asymmetric metric learning: For scenarios with multiple object categories, it measures the distance between some categories of objects, so that under the learned distance metric, the distance between similar objects is small and the distance between dissimilar objects is large.

[0111] Liveness detection: Determines whether the people in the data are real, living, and present.

[0112] Furthermore, the number of any elements in the accompanying drawings is for illustrative purposes only and not for limitation, and any naming is for distinction only and has no limiting meaning.

[0113] In addition, the data involved in this disclosure may be data authorized by the user or fully authorized by all parties. The collection, dissemination and use of the data shall comply with the requirements of relevant national laws and regulations. The implementation methods / executives of this disclosure may be combined with each other.

[0114] The principles and spirit of this disclosure will be explained in detail below with reference to several representative embodiments. Invention Overview

[0116] Liveness detection is a crucial part of identity authentication, primarily used in high-security systems such as electronic payments, access control, and account opening. It verifies that the face acquired by the device is a real, living, and present biological face, rather than a fake face, such as a face in a media outlet like an image, newspaper, or computer.

[0117] The inventors have discovered that in the field of liveness detection, a common method is to use labeled data, such as real faces and fake faces, for supervised learning to obtain a liveness detection model. This model is then used to identify the face image to be detected, achieving the goal of liveness detection. However, face recognition technology can obtain a large amount of real face data (millions or even billions of data points), while the amount of fake face data is relatively small. Given this imbalance between real and fake face data, the limited types of available fake face data, and the uneven distribution of different types of fake faces, even if the imbalance can be addressed using methods commonly employed in supervised learning, such as undersampling and oversampling, the poor generalization ability of existing deep learning networks results in weak ability to identify fake faces with known attack types. Furthermore, with the continuous development of attack techniques, attackers can quickly develop new attack types. Since the recognition ability of existing liveness detection models relies on the training set, they lack the ability to identify unknown attack types because the training set does not contain corresponding samples.

[0118] To address the aforementioned issues, this disclosure provides a model training method, a live face detection method, an apparatus, a device, and a medium. It employs an unlabeled image dataset to perform self-supervised training on an initial face recognition model to obtain a corresponding first face recognition model. Subsequently, it uses a first real live face dataset to perform self-supervised training on the first face recognition model to obtain a corresponding second face recognition model. Finally, it uses a labeled mixed dataset of real and fake faces to perform asymmetric metric learning on the second face recognition model to obtain a live face recognition model, thereby enabling the live face recognition model to have a strong ability to extract facial features. In practical applications, the image of the face to be identified can be input into the live face recognition model. The live face recognition model can obtain the recognition result indicating whether the image of the face to be identified is a real face by calculating the feature distance between the features of the image of the face to be identified and the reference features of the real face. The reference features of the real face are calculated based on the features of each image in the second real live face dataset. This solves the problems of imbalance between the amount of real face and fake face data, limited types of available fake face data, and imbalance in the number of different types of fake faces, thus improving the generalization of fake faces.

[0119] After introducing the basic principles of this disclosure, various non-limiting embodiments of this disclosure will be described in detail below.

[0120] Application Scenarios Overview

[0121] First refer to Figure 1 Examples of application scenarios for the solutions provided in this disclosure are given. Figure 1 This is a schematic diagram of an application scenario provided by an embodiment of the present disclosure, such as... Figure 1 As shown, in this application scenario, terminal device 101 can provide various services to users through a browser or application. When the service involves user authentication, terminal device 101 can acquire a face image to be recognized in the acquisition area through an image acquisition device and send the face image to be recognized to server 102. Server 102 receives the face image to be recognized sent by terminal device 101, inputs the face image to be recognized into a pre-acquired liveness detection model, thereby obtaining a recognition result indicating whether the face image to be recognized is a real face, and returns the recognition result to terminal device 101.

[0122] Optionally, the live face recognition model can be obtained by pre-training the model on the server 102, or it can be obtained from other data storage devices that store the live face recognition model. This disclosure does not impose specific restrictions on this.

[0123] It should be noted that, Figure 1This is merely a schematic diagram illustrating one application scenario provided by an embodiment of this disclosure; this embodiment does not necessarily represent an application scenario. Figure 1 The included equipment is not limited, nor is it restricted. Figure 1 The positional relationships between devices are defined. For example, in Figure 1 The application scenario shown may also include a data storage device, which may be an external storage device relative to the terminal device 101 or the server 102, or an internal storage device integrated in the terminal device 101 or the server 102.

[0124] Exemplary methods

[0125] The following is combined Figure 1 Application scenarios, refer to Figure 1 This document describes a model training method and a live face detection method according to exemplary embodiments of the present disclosure. It should be noted that the above application scenarios are shown only to facilitate understanding of the spirit and principles of the present disclosure, and the embodiments of the present disclosure are not limited in any way. Rather, the embodiments of the present disclosure can be applied to any applicable scenario.

[0126] First, the model training method will be introduced through specific examples.

[0127] Figure 2 This is a flowchart illustrating a model training method provided in one embodiment of this disclosure. The method of this embodiment can be applied to a computing device, which may be a server or a server cluster, etc. Figure 2 As shown, the method in this embodiment includes:

[0128] S201. Use an unlabeled image dataset to perform self-supervised training on the initial face recognition model until the initial face recognition model reaches the training cutoff condition, so as to obtain the corresponding first face recognition model.

[0129] The initial face recognition model can be a specific network architecture N, such as ViT-B / 16, which includes a feature extractor. The unlabeled image dataset can be the ImageNet dataset, which is in the tens of millions and contains a large number of categories.

[0130] Optionally, the initial face recognition model may also include a decoder.

[0131] Optionally, the initial face recognition model can be trained in a self-supervised manner based on the image pixel reconstruction self-supervised (masked autoencoders, MAE) algorithm.

[0132] After determining the network structure N of the initial face recognition model, self-supervised training can be performed on ImageNet using the MAE algorithm based on this network structure N until the initial face recognition model reaches the training cutoff condition, thus obtaining the corresponding first face recognition model.

[0133] Among them, the model parameters of the first face recognition model are the first parameters, which include the first extractor parameters corresponding to the feature extractor. The first extractor parameters have the ability to extract general image data features.

[0134] S202. Using the first real live face dataset, and with the parameters of the first extractor as the initial parameters of the feature extractor, the first face recognition model is trained in a self-supervised manner until the first face recognition model reaches the training cutoff condition, so as to obtain the corresponding second face recognition model.

[0135] Optionally, the first face recognition model trained in S201 can be reused, and the parameters of the first extractor can be used as the initial parameters of the feature extractor. The MAE algorithm can be used again to perform self-supervised training on the first real live face dataset until the first face recognition model reaches the training cutoff condition, so as to obtain the corresponding second face recognition model.

[0136] Among them, the model parameters of the second face recognition model are the second parameters, which include the second extractor parameters corresponding to the feature extractor.

[0137] Among them, since there are obvious differences in object types and scenes between the unlabeled image dataset and the first real live face dataset, i.e., domain differences, the second extractor parameters are able to extract facial features.

[0138] Optionally, before S202, a first real live face dataset can be determined from the initial real live face dataset. The specific determination method will be described in detail in the following embodiments and will not be repeated here.

[0139] Optionally, the initial real live face dataset can be obtained from real human face data, general face data, or pedestrian data, depending on different needs.

[0140] Figure 3 This is a schematic diagram illustrating the feature distribution results of real and fake face data. For example... Figure 3 As shown, Figure 3This includes feature distributions A, B, and C. Feature distribution A is obtained by processing the CASIA-CUR liveness dataset (CASIA-CUR fake face data, gray) and real-scene face data (black) using a third face recognition model. This third face recognition model is obtained through supervised training of an initial face recognition model using an unlabeled image dataset. Feature distribution B is obtained by processing the CASIA-CUR liveness dataset (CASIA-CUR fake face data and real-scene face data) using a first face recognition model. Feature distribution C is obtained by processing the CASIA-CUR liveness dataset (CASIA-CUR fake face data and real-scene face data) using a second face recognition model. Figure 3 It can be seen that the concentration of features in the real face data, from weakest to strongest, is feature distribution A, feature distribution B, and feature distribution C, respectively. In other words, the second extractor parameters of the second face recognition model have a stronger ability to extract face features.

[0141] It should be understood that Figure 3 None of the data in this dataset was used for model training. The dummy faces can be faces from media such as images, newspapers, and computers, or they can be the faces of real dummy figures.

[0142] S203. Using a labeled mixed dataset of real and fake faces, and taking the parameters of the second extractor as the initial parameters of the feature extractor, perform asymmetric metric learning on the second face recognition model until the second face recognition model reaches the training cutoff condition, so as to obtain a live face recognition model.

[0143] Since the second extractor parameters already possess the ability to extract facial features, they lack the ability to distinguish between real and fake faces. Therefore, asymmetric metric learning is required for the second face recognition model to enable the liveness detection model to distinguish between real and fake faces.

[0144] Optionally, the labeled real-fake face mixed dataset includes labeled real human face data and labeled fake human face data. Before S203, the labeled real-fake face mixed dataset can also be determined from the initial real live human face dataset and the initial fake human face dataset. The specific determination method will be described in detail in the following embodiments and will not be repeated here.

[0145] Among them, the model parameter of the live face recognition model is the third parameter, which includes the third extractor parameter corresponding to the feature extractor.

[0146] It should be understood that the first real live face dataset does not include labeled mixed datasets of real and fake faces.

[0147] The model training method provided in this disclosure employs an unlabeled image dataset for self-supervised training of an initial face recognition model. This enables the first extractor parameters of the trained first face recognition model to extract general image data features. Furthermore, the first face recognition model is self-supervised trained using a first real live face dataset, enabling the second extractor parameters of the trained second face recognition model to extract facial features. Finally, a labeled mixed dataset of real and fake faces is used to perform asymmetric metric learning on the second face recognition model, enabling the live face recognition model to distinguish between real and fake faces. This technical solution addresses the issue from two directions: self-supervised learning and asymmetric metric learning. By fully utilizing the ImageNet dataset and the first real live face dataset for self-supervised training, it significantly improves the facial feature extraction and recognition capabilities of the second face recognition model, allowing it to identify fake faces of unknown attack types. The model training method provided in this technical solution can be transferred to face research-related fields such as liveness detection, person re-identification, and action recognition to enhance the facial feature extraction capabilities of deep models.

[0148] This disclosure effectively combines self-supervised training on unlabeled image datasets with self-supervised training for specific task scenarios to improve the feature extraction capability of the live face recognition model for specific task data. The self-supervised training itself is task-independent and can assist the model in extracting features for any task. This disclosure uses it as the initialization parameters for the live face recognition task (i.e., the second extractor parameters), introducing more supervisory information into the live face recognition task.

[0149] In contrast, unlabeled image datasets in real-world scenarios are readily available and exist in large quantities, ranging from millions to tens of millions. In contrast, fake face datasets are difficult to obtain and limited in both quantity and variety, including those based on paper images, electronic device screens, and composite images. This disclosure utilizes a first real live face dataset for self-supervised training, enabling the live face recognition model to extract facial features from various scenarios.

[0150] In one possible implementation, S203 can be achieved through the following steps:

[0151] Using a labeled mixed dataset of real and fake faces, and taking the parameters of the second extractor as the initial parameters of the feature extractor, asymmetric metric learning is performed on the second face recognition model containing the classifier and the feature extractor until the second face recognition model reaches the training cutoff condition, so as to obtain a live face recognition model.

[0152] After obtaining the second face recognition model, a classifier can be added to the second face recognition model. The parameters of the second extractor can be used as the initial parameters of the feature extractor. Asymmetric metric learning can be performed on the second face recognition model containing the classifier and the feature extractor to obtain a live face recognition model.

[0153] Optionally, the second face recognition model also includes a feature clusterer, with the feature extractor connected to both the classifier and the feature clusterer. Since the data domain of real faces shows relatively small differences, while the data domain of fake faces shows significant differences—for example, fake face data can be face data from different attack types such as images and electronic identification—the distribution of features on real faces is more clustered, while the distribution of features on fake faces is more dispersed. Therefore, when performing asymmetric metric learning on the second face recognition model, a first sub-loss function can be introduced for the classifier, and a second sub-loss function can be introduced for the real face features. In other words, the loss function of the second face recognition model includes the first sub-loss function corresponding to the classifier and the second sub-loss function corresponding to the feature clusterer.

[0154] Optionally, the first sub-loss function can be the real / fake face recognition loss function (Adaface), which makes the features of the two categories of real and fake faces distinguishable. The second sub-loss function can be feature clustering analysis (Center Loss), i.e., asymmetric metric learning, which makes the distribution of real face features concentrated.

[0155] In summary, the embodiments of this disclosure use the second extractor parameters as the initial parameters of the feature extractor, and the classifier introduces a first sub-loss function to constrain the feature distribution of each category, so that the model parameters of the live face recognition model have the ability to distinguish between real and fake faces, and to eliminate as much as possible the potential domain differences between the second extractor parameters of the second face recognition model and the second face recognition model.

[0156] Meanwhile, since the differences in the face domain of real people are not significant, while the differences in the face domain of fake people are significant, fake faces can be paper-based, computer-generated, cartoon-like, and other attack-type faces. Therefore, using a parameter-learnable Center Loss to constrain the distribution of real person features makes the distribution of real person face features more concentrated, thereby improving the generalization ability of the liveness detection model parameters to fake people.

[0157] Optionally, since the first and second sub-loss functions differ significantly in numerical value, dynamic weighting can be used to balance the two sub-loss functions during the asymmetric metric learning process of the second face recognition model. This allows the live face recognition model to simultaneously possess the ability to distinguish between real and fake faces and the ability to set real face features. In other words, the loss function of the second face recognition model can be dynamically determined. Figure 4A flowchart illustrating a model training method provided in another embodiment of this disclosure. (See attached flowchart.) Figure 4 As shown, the loss function of the second face recognition model can be dynamically determined through the following steps:

[0158] S401. The first preset weight value is determined as the weight value of the first sub-loss function.

[0159] Optionally, the first preset weight value can be 1. It should be understood that the first preset weight value can also be other values, which can be limited according to the actual situation, and the embodiments of this disclosure do not impose specific limitations in this regard.

[0160] S402. Calculate the ratio of the first sub-loss function value to the second sub-loss function value.

[0161] S403. Determine the smaller of the ratio and the second preset weight value as the weight value of the second sub-loss function.

[0162] S404. Multiply the first sub-loss function by the corresponding weight value to obtain the first product.

[0163] S405. Multiply the second sub-loss function by the corresponding weight value to obtain the second product.

[0164] S406. The sum of the first product and the second product is used as the loss function.

[0165] In other words, the loss function of the second face recognition model can be dynamically determined using the following formula:

[0166]

[0167] Where L is the loss function, L adaface For the first sub-loss function, L center The second sub-loss function is γ, which is the second preset weight value, which can be 0.05, and the first preset weight value is 1.

[0168] Figure 5 A flowchart illustrating a model training method provided in yet another embodiment of this disclosure. For example... Figure 5 As shown, asymmetric metric learning of the second face recognition model can be achieved through the following process:

[0169] A labeled mixed dataset of real and fake faces is input into the feature extractor of the second face recognition model. This feature extractor then feeds the labeled mixed dataset to the classifier, which is trained using the first sub-loss function (Adaface) to enable the trained liveness recognition model to distinguish between real and fake faces. Simultaneously, the feature extractor feeds the labeled real face data from the mixed dataset to the feature clusterer, which is then trained using the second sub-loss function (Center Loss) to enable the trained liveness recognition model to concentrate the distribution of real face features, thereby improving its generalization ability against fake faces.

[0170] Because the differences in the facial features of real people are small, while the types of attacks by fake people are varied, the differences in the facial features of fake people are significant. Therefore, in this embodiment of the disclosure, based on the characteristics that the distribution of real facial features is concentrated, while the distribution of fake facial features of different attack types is relatively dispersed, a Center Loss is introduced for real facial features, and a Dadaface is introduced for both real and fake facial features. Dynamic weighted fusion of Center Loss and Dadaface is used to enable the liveness detection model to have the ability to concentrate the distribution of real facial features and to distinguish between real and fake faces, thereby improving the liveness detection model's ability to identify fake people of various attack types.

[0171] Optionally, in some embodiments, before S202, it is also necessary to obtain a first real live face dataset. That is, before S202, the method of this disclosure embodiment further includes:

[0172] Using a face detection algorithm, the real face regions in the initial real live face dataset are cropped, and the cropped real face region images are determined as the first real live face dataset.

[0173] Optionally, the face detection algorithm can be Yolov5s, Retinaface, or FaceNet, etc. It should be understood that the face detection algorithm can also be other existing algorithms for face detection, which can be determined according to the actual situation, and this disclosure does not impose specific limitations in this regard.

[0174] This embodiment of the disclosure uses a face detection algorithm to crop the real face regions in the initial real live face dataset to obtain a first real live face dataset. This reduces unnecessary noise information such as the surrounding environment information introduced into the face features, and improves the accuracy of subsequent self-supervised training based on the first real live face dataset.

[0175] Optionally, in some embodiments, a labeled dataset of mixed real and fake faces needs to be obtained before S203. Figure 6 A flowchart illustrating a model training method provided in yet another embodiment of this disclosure. For example... Figure 6 As shown, prior to S203, the method of this embodiment further includes:

[0176] S601. Using a face detection algorithm, crop the real face regions from the initial real live face dataset and generate corresponding labels for the cropped real face region images.

[0177] Optionally, the face detection algorithm can be Yolov5s.

[0178] S602. Using a face detection algorithm, crop the fake face regions in the initial fake face dataset and generate corresponding labels for the cropped fake face region images.

[0179] Optionally, the face detection algorithm can be Yolov5s.

[0180] S603. Perform data augmentation processing on labeled real human face region images and labeled fake human face region images to obtain a labeled real and fake face mixed dataset.

[0181] This embodiment of the disclosure obtains a labeled mixed dataset of real and fake faces by cropping and augmenting the initial dataset of real and fake faces. This ensures that the labeled mixed dataset of real and fake faces contains sufficient data, laying the foundation for subsequent asymmetric metric learning of the second face recognition model based on the labeled mixed dataset of real and fake faces.

[0182] After obtaining the above-mentioned live face recognition model, it can be used to identify the face image to determine whether it is a real face. The following detailed description, in conjunction with specific embodiments, illustrates the use of this live face recognition model to identify the face image. These specific embodiments can be combined with each other; similar or identical concepts or processes may not be repeated in some embodiments.

[0183] In practice, the entity executing the liveness detection method can also be a computing device such as a server or server cluster. It should be understood that the computing device executing the liveness detection method and the computing device executing the aforementioned model training method can be the same device or different devices.

[0184] Next, we will introduce the live face detection method through specific examples.

[0185] Figure 7This is a flowchart illustrating a live face detection method according to an embodiment of this disclosure. The method of this embodiment can be applied to a computing device, such as a server or server cluster. Figure 7 As shown, the method in this embodiment includes:

[0186] S701. Obtain the face image to be identified.

[0187] Because some services involve users' personal information or assets, user authentication is required before processing any transactions to ensure information and asset security. In other words, when user authentication is required, the computing device can acquire a facial image to be recognized sent by the user's terminal device, which is obtained by the terminal device through an image acquisition device.

[0188] S702. Input the face image to be recognized into the live face recognition model and obtain the recognition result.

[0189] The live face recognition model is trained according to the method of any of the above embodiments, and the recognition result is used to indicate whether the face image to be recognized is a real face.

[0190] In one possible implementation, S702 can be achieved through steps (1) to (3):

[0191] Step (1): Extract the features of the face image to be recognized using the feature extractor of the live face recognition model.

[0192] Because existing live face recognition models rely on training data, while fake face data is scarce and attack types are limited, overfitting is a common problem, resulting in poor generalization of existing live face recognition models to fake faces. Since real face data has a concentrated feature distribution, this embodiment uses a feature extractor from the live face recognition model to extract features from the face image to be recognized, allowing for subsequent comparison with real face baseline features to generate a recognition result.

[0193] The real face baseline features are determined in advance based on the initial real live face dataset. The specific determination process will be explained in detail in the following embodiments, and will not be repeated here.

[0194] Step (2): Calculate the feature distance between the features of the face image to be identified and the reference features of the real face.

[0195] In one possible implementation, the Euclidean distance between the features of the face image to be identified and the reference features of the real face can be calculated to obtain the feature distance.

[0196] Optionally, the Euclidean distance between the features of the face image to be identified and the reference features of the real face can be calculated using the following formula:

[0197]

[0198] Where, f(x) i，j ) represents the face image to be identified x i The j-th eigenvalue, F j Let j represent the j-th feature value of the real face baseline features, where j = 0, 1, 2, ..., K-1, and K = 768.

[0199] Step (3): Generate recognition results based on feature distance.

[0200] In one possible implementation, when the feature distance is greater than a preset distance, a recognition result is generated to indicate that the face image to be identified is a fake face; when the feature distance is less than or equal to the preset distance, a recognition result is generated to indicate that the face image to be identified is a real face. When facing fake faces of unknown attack types, the recognition result of the face image to be identified is determined by calculating the feature distance between the features of the face image to be identified and the reference features of the real face, which can effectively identify fake faces of unknown attack types.

[0201] Optionally, the preset distance can be set in advance according to the actual situation, and this disclosure does not impose specific restrictions on it.

[0202] Optionally, after generating the recognition result, it can be further determined whether the face image to be recognized is the face image of the user and an identity authentication result can be generated.

[0203] The live face detection method provided in this disclosure involves a computing device acquiring an image of a face to be identified and inputting it into a live face recognition model to obtain a recognition result. This disclosure leverages the concentrated distribution of features in real human faces, using a feature extractor in the live face recognition model to extract features from the face image to be identified. These features are then compared with baseline features of a real face to generate a recognition result. This effectively improves the generalization ability of the live face recognition model to fake faces of unknown attack types, as well as its ability to identify fake faces of known attack types.

[0204] based on Figure 7 The following is a detailed description of the process for determining the baseline features of a real human face, based on the illustrated embodiment. Figure 8 A flowchart illustrating a live face detection method provided in another embodiment of this disclosure. Figure 8 As shown, prior to S702, the method of this embodiment further includes:

[0205] S801. Using a face detection algorithm, crop the real face regions in the initial real live face dataset, and determine the cropped real face region image as the second real live face dataset.

[0206] Optionally, the second real live face dataset is a subset of the first real live face dataset, which is the training set used for model training to obtain a live face recognition model.

[0207] Optionally, the face detection algorithm can be YOLOv5.

[0208] S802. Based on the feature extractor of the live face recognition model, extract the features of each image in the second real live face dataset.

[0209] To prevent overfitting of the classifier, features from each image in a second real live face dataset can be extracted using the feature extractor of the live face recognition model to construct a feature library. It should be understood that the parameters of the feature extractor in the live face recognition model are the parameters of the second extractor.

[0210] S803. Calculate the features of each image in the second real live face dataset to determine the real face baseline features.

[0211] In one possible implementation, all features in the feature library can be fused together as the baseline features for the real face. Specifically, the average feature value of each image in the second real live face dataset can be calculated, and this average feature value can be determined as the baseline feature for the real face.

[0212] Optionally, the aforementioned real facial reference features can be achieved using the following formula:

[0213]

[0214] Where i = 1, 2, ..., n, n is the data size of the second real live face dataset, x i Let f(x) represent the i-th real face data in the second real live face dataset. i x is the feature extractor obtained through the live face recognition model. i Features, f(x) i Let F be a K-dimensional eigenvector. In one possible implementation, n = 2000000, K = 768, and F is a 768-dimensional eigenvector.

[0215] In the above embodiments, the construction method of real face reference features can be transferred to feature references in other computer vision tasks, such as pedestrian re-identification, action recognition, object recognition and other fields; similarly, the matching evaluation method of calculating the Euclidean distance between the features of the face image to be identified and the real face reference features to obtain the feature distance, and generating the recognition result based on the feature distance can also be flexibly applied to other fields of vision tasks.

[0216] Based on the above embodiments, the method will be described in detail below through two specific examples.

[0217] Figure 9 This is a flowchart of a live face detection method provided in another embodiment of this disclosure. Figure 8 As shown, the method in this embodiment includes:

[0218] Based on the feature extractor of the live face recognition model, features are extracted from each image in the second real live face dataset to construct a feature library. All features in the feature library are then fused as the baseline features for real faces. Simultaneously, an image of the face to be recognized is acquired. Using the feature extractor of the live face recognition model, features of the image are extracted. The feature distance between the features of the image to be recognized and the baseline features of the real face is calculated. When the feature distance is greater than a preset distance, a recognition result indicating that the image to be recognized is a fake face is generated; when the feature distance is less than or equal to the preset distance, a recognition result indicating that the image to be recognized is a real face is generated.

[0219] Existing live face recognition models use classifiers to classify the faces to be recognized. However, these classifiers rely heavily on the variety and quantity of the training dataset, making them prone to overfitting. This results in weak recognition capabilities against fake faces with known attack types and poor generalization against fake faces with unknown attack types. Based on the characteristic that real face features are concentrated while fake face features are dispersed, this embodiment constructs real face baseline features, calculates the feature distance between the features of the face image to be recognized and the real face baseline features, and uses this feature distance to determine whether a face is real or fake, further improving the model's generalization ability against fake faces.

[0220] Figure 10 This is a flowchart of a live face detection method provided in another embodiment of this disclosure. Figure 10 As shown, the method in this embodiment includes:

[0221] Step 1: Use an unlabeled image dataset to perform self-supervised training on the initial face recognition model to obtain the first face recognition model.

[0222] Step 2: Using the first real live face dataset, perform self-supervised training on the first face recognition model to obtain the second face recognition model.

[0223] Step 3: Using a labeled mixed dataset of real and fake faces, perform asymmetric metric learning on the second face recognition model to obtain a live face recognition model.

[0224] Step 4: Construct real face baseline features based on the initial real live face dataset.

[0225] Step 5: Perform feature matching between the features of the face image to be identified and the baseline features of the real face to generate the recognition result.

[0226] Exemplary media

[0227] After introducing the methods of exemplary embodiments of this disclosure, the following references are made. Figure 11 The storage medium of the exemplary embodiments of this disclosure will be described.

[0228] refer to Figure 11 As shown, the storage medium 110 stores a program product for implementing the above-described methods according to embodiments of the present disclosure. This program product may be a portable compact disc read-only memory (CD-ROM) and includes computer-executable instructions for causing a computing device to execute the model training method and the live face detection method provided in this disclosure. However, the program product of this disclosure is not limited thereto.

[0229] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0230] A readable signal medium may include data signals propagated in baseband or as part of a carrier wave, carrying computer-executed instructions. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium.

[0231] Computer-executable instructions for performing the operations disclosed herein can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The computer-executable instructions can be executed entirely on the user's computing device, partially on the user's computing device, partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing devices can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN).

[0232] Exemplary device

[0233] Having introduced the medium of exemplary embodiments of this disclosure, the following references are made to... Figure 12 and Figure 13 The model training apparatus and the live face detection apparatus of the exemplary embodiments of this disclosure will be described to implement the model training method or the live face detection method in any of the above method embodiments. Their implementation principles and technical effects are similar, and will not be repeated here.

[0234] Figure 12 This is a schematic diagram of the structure of a model training device provided in an embodiment of this disclosure. Figure 12 As shown, the model training device 1200 includes:

[0235] The first training module 1201 is used to perform self-supervised training on the initial face recognition model using an unlabeled image dataset until the initial face recognition model reaches the training cutoff condition to obtain the corresponding first face recognition model. The initial face recognition model includes a feature extractor, and the model parameters of the first face recognition model are first parameters, which include first extractor parameters corresponding to the feature extractor.

[0236] The second training module 1202 is used to perform self-supervised training on the first face recognition model using the first real live face dataset and the first extractor parameters as the initial parameters of the feature extractor, until the first face recognition model reaches the training cutoff condition, so as to obtain the corresponding second face recognition model. The model parameters of the second face recognition model are the second parameters, which include the second extractor parameters corresponding to the feature extractor.

[0237] The third training module 1203 is used to perform asymmetric metric learning on the second face recognition model using a labeled mixed dataset of real and fake faces, with the parameters of the second extractor as the initial parameters of the feature extractor, until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model. The model parameters of the live face recognition model are the third parameters, which include the third extractor parameters corresponding to the feature extractor.

[0238] In some embodiments, the third training module 1203 is specifically used for:

[0239] Using a labeled mixed dataset of real and fake faces, and taking the parameters of the second extractor as the initial parameters of the feature extractor, asymmetric metric learning is performed on the second face recognition model containing the classifier and the feature extractor until the second face recognition model reaches the training cutoff condition, so as to obtain a live face recognition model.

[0240] In some embodiments, the second face recognition model further includes a feature clusterer, and the feature extractor is connected to the classifier and the feature clusterer respectively. The loss function of the second face recognition model includes a first sub-loss function corresponding to the classifier and a second sub-loss function corresponding to the feature clusterer.

[0241] In some embodiments, the model training apparatus 1200 further includes a determination module, wherein during the asymmetric metric learning process of the second face recognition model, the loss function of the second face recognition model can be dynamically determined according to the determination module:

[0242] The first preset weight value is determined as the weight value of the first sub-loss function.

[0243] Calculate the ratio of the first sub-loss function value to the second sub-loss function value.

[0244] The smaller of the ratio and the second preset weight value is determined as the weight value of the second sub-loss function.

[0245] Multiply the first sub-loss function by the corresponding weight value to obtain the first product.

[0246] The second sub-loss function is multiplied by the corresponding weight value to obtain the second product.

[0247] The sum of the first and second products is used to determine the loss function.

[0248] In some embodiments, before the first face recognition model is trained in a self-supervised manner using a first real live face dataset and the first extractor parameters as the initial parameters of the feature extractor, until the first face recognition model reaches the training cutoff condition to obtain the corresponding second face recognition model, the model training device 1200 further includes:

[0249] The first cropping module is used to crop the real face regions in the initial real live face dataset using a face detection algorithm, and to determine the cropped real face region image as the first real live face dataset.

[0250] In some embodiments, when using a labeled mixed dataset of real and fake faces, and using the parameters of the second extractor as the initial parameters of the feature extractor, the second face recognition model is subjected to asymmetric metric learning until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model. Before this, the model training device 1200 further includes:

[0251] The second cropping module is used to crop the real face regions in the initial real live face dataset using a face detection algorithm, and to generate corresponding labels for the cropped real face region images.

[0252] The third cropping module is used to crop the fake face regions in the initial fake face dataset using a face detection algorithm, and to generate corresponding labels for the cropped fake face region images.

[0253] The processing module is used to perform data augmentation processing on labeled real human face region images and labeled fake human face region images to obtain a labeled real and fake face mixed dataset.

[0254] The model training apparatus provided in this application embodiment can be used to execute the model training method in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0255] Figure 13 This is a schematic diagram of the structure of a live face detection device provided in an embodiment of this disclosure. Figure 13 As shown, the live face detection device 1300 includes:

[0256] The acquisition module 1301 is used to acquire the face image to be identified.

[0257] The input module 1302 is used to input the face image to be recognized into the live face recognition model and obtain the recognition result. The live face recognition model is trained according to the first method, and the recognition result is used to indicate whether the face image to be recognized is a real face.

[0258] In some embodiments, before inputting the image of the face to be identified into the liveness detection model and obtaining the recognition result, the liveness detection device 1300 further includes:

[0259] The cropping module is used to crop the real face regions in the initial real live face dataset using a face detection algorithm, and to determine the cropped real face region image as the second real live face dataset.

[0260] The extraction module is used to extract features from each image in the second real live face dataset based on the feature extractor of the live face recognition model.

[0261] The calculation module is used to calculate the features of each image in the second real live face dataset to determine the real face baseline features.

[0262] In some embodiments, the computing module is specifically used for:

[0263] Calculate the average feature value of each image in the second real live face dataset, and determine the average feature value as the benchmark feature of the real face.

[0264] In some embodiments, the acquisition module 1301 is specifically used for:

[0265] The feature extractor of the live face recognition model extracts the features of the face image to be recognized.

[0266] Calculate the feature distance between the features of the face image to be identified and the baseline features of the real face.

[0267] The recognition result is generated based on the feature distance.

[0268] In some embodiments, the acquisition module 1301 is specifically used for:

[0269] When the feature distance is greater than a preset distance, a recognition result is generated to indicate that the face image to be identified is a fake face.

[0270] When the feature distance is less than or equal to a preset distance, a recognition result is generated to indicate that the face image to be identified is a real human face.

[0271] In some embodiments, the acquisition module 1301 is specifically used for:

[0272] Calculate the Euclidean distance between the features of the face image to be identified and the reference features of the real face to obtain the feature distance.

[0273] In some embodiments, the second real live face dataset is a subset of the first real live face dataset, which is a training set used for model training to obtain a live face recognition model.

[0274] The live face detection device provided in this application embodiment can be used to execute the live face detection method in any of the above embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0275] Exemplary computing device

[0276] Having described the methods, media, and apparatus of exemplary embodiments of this disclosure, the following references... Figure 14 A computing device according to an exemplary embodiment of the present disclosure will be described.

[0277] Figure 14 The computing device 1400 shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments disclosed herein.

[0278] like Figure 4 As shown, the computing device 1400 is presented in the form of a general-purpose computing device. Components of the computing device 1400 may include, but are not limited to: at least one processing unit 1401, at least one storage unit 1402, and a bus 1403 connecting different system components (including the processing unit 1401 and the storage unit 1402). The at least one storage unit 1402 stores computer-executable instructions; the at least one processing unit 1401 includes a processor that executes the computer-executable instructions to implement the methods described above.

[0279] Bus 1403 includes a data bus, a control bus, and an address bus.

[0280] Storage unit 1402 may include readable media in the form of volatile memory, such as random access memory (RAM) 14021 and / or cache memory 14022, and may further include readable media in the form of non-volatile memory, such as read-only memory (ROM) 14023.

[0281] Storage unit 1402 may also include a program / utility 14025 having a set (at least one) of program modules 14024, such program modules 14024 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0282] The computing device 1400 can also communicate with one or more external devices 1404 (e.g., keyboard, pointing device, etc.). This communication can be performed via input / output (I / O) interface 1405. Furthermore, the computing device 1400 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 1406. Figure 4 As shown, network adapter 1406 communicates with other modules of computing device 1400 via bus 1403. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with computing device 1400, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0283] Optionally, the computing device 1400 can be implemented as a server 102.

[0284] It should be noted that although several units / modules or sub-units / modules of the model training device and the live face detection device are mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more units / modules described above can be embodied in one unit / module. Conversely, the features and functions of one unit / module described above can be further divided and embodied by multiple units / modules.

[0285] Furthermore, although the operations of the methods disclosed herein are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0286] While the spirit and principles of this disclosure have been described with reference to several specific embodiments, it should be understood that this disclosure is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for convenience of expression. This disclosure is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

Claims

1. A model training method, comprising: An unlabeled image dataset is used to perform self-supervised training on an initial face recognition model until the initial face recognition model reaches the training cutoff condition to obtain a corresponding first face recognition model. The initial face recognition model includes a feature extractor, and the model parameters of the first face recognition model are first parameters, which include first extractor parameters corresponding to the feature extractor. Using a first real live face dataset, and with the parameters of the first extractor as the initial parameters of the feature extractor, the first face recognition model is subjected to self-supervised training until the first face recognition model reaches the training cutoff condition, thereby obtaining a corresponding second face recognition model. The model parameters of the second face recognition model are second parameters, which include second extractor parameters corresponding to the feature extractor. The second face recognition model includes a classifier, the feature extractor, and a feature clusterer. The feature extractor is connected to both the classifier and the feature clusterer. The loss function of the second face recognition model includes a first sub-loss function corresponding to the classifier and a second sub-loss function corresponding to the feature clusterer. A labeled dataset of real and fake faces is used. The parameters of the second extractor are used as the initial parameters of the feature extractor. The second face recognition model is subjected to asymmetric metric learning until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model. The model parameters of the live face recognition model are the third parameters, which include the third extractor parameters corresponding to the feature extractor. The asymmetric metric learning of the second face recognition model is achieved through the following process: A labeled mixed dataset of real and fake faces is input into the feature extractor. The feature extractor then transmits the labeled mixed dataset of real and fake faces to the classifier. The classifier is trained using the first sub-loss function so that the trained live face recognition model has the ability to distinguish between real and fake faces. Simultaneously, the feature extractor transmits labeled real face data from the labeled mixed dataset of real and fake faces to the feature clusterer so that the feature clusterer is trained using the second sub-loss function so that the trained live face recognition model has the ability to concentrate the distribution of real face features.

2. According to the method described in claim 1, during the asymmetric metric learning process of the second face recognition model, the loss function of the second face recognition model can be dynamically determined according to the following process: The first preset weight value is determined as the weight value of the first sub-loss function; Calculate the ratio of the first sub-loss function value to the second sub-loss function value; The smaller of the ratio and the second preset weight value is determined as the weight value of the second sub-loss function; Multiply the first sub-loss function by the corresponding weight value to obtain the first product; Multiply the second sub-loss function by the corresponding weight value to obtain the second product; The sum of the first product and the second product is determined as the loss function.

3. The method according to claim 1, before the first face recognition model is trained by self-supervised training using a first real live face dataset and the first extractor parameters as the initial parameters of the feature extractor until the first face recognition model reaches the training cutoff condition to obtain the corresponding second face recognition model, the method further includes: Using a face detection algorithm, the real face regions in the initial real live face dataset are cropped, and the cropped real face region images are determined as the first real live face dataset.

4. The method according to claim 1, wherein before using a labeled mixed dataset of real and fake faces, and using the parameters of the second extractor as the initial parameters of the feature extractor to perform asymmetric metric learning on the second face recognition model until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model, the method further includes: Using a face detection algorithm, the real face regions in the initial real live face dataset are cropped, and corresponding labels are generated for the cropped real face region images. The face detection algorithm is used to crop out the fake face regions in the initial fake face dataset and generate corresponding labels for the cropped fake face region images. Data augmentation processing is performed on labeled real human face region images and labeled fake human face region images to obtain the labeled real and fake face hybrid dataset.

5. A method for detecting a live human face, comprising: Acquire the image of the face to be identified; The face image to be identified is input into a live face recognition model to obtain a recognition result. The live face recognition model is trained according to the method described in any one of claims 1-4. The recognition result is used to indicate whether the face image to be identified is a real face.

6. The method according to claim 5, before inputting the face image to be identified into the liveness detection model to obtain the recognition result, the method further includes: Using a face detection algorithm, the real face regions in the initial real live face dataset are cropped, and the cropped real face region images are determined as the second real live face dataset. Based on the feature extractor of the live face recognition model, extract the features of each image in the second real live face dataset; The features of each image in the second real live face dataset are calculated to determine the real face baseline features.

7. The method according to claim 6, wherein calculating the features of each image in the second real live face dataset to determine the real face baseline features includes: Calculate the average feature value of each image in the second real live face dataset, and determine the average feature value as the real face baseline feature.

8. The method according to claim 6 or 7, wherein inputting the face image to be identified into a liveness detection model to obtain the recognition result includes: The features of the face image to be identified are extracted using the feature extractor of the live face recognition model. Calculate the feature distance between the features of the face image to be identified and the reference features of the real face; The recognition result is generated based on the feature distance.

9. The method according to claim 8, wherein generating the recognition result based on the feature distance comprises: When the feature distance is greater than a preset distance, a recognition result is generated to indicate that the face image to be identified is a fake face; When the feature distance is less than or equal to the preset distance, a recognition result is generated to indicate that the face image to be identified is a real human face.

10. The method according to claim 8, wherein calculating the feature distance between the features of the face image to be identified and the reference features of the real face includes: The feature distance is obtained by calculating the Euclidean distance between the features of the face image to be identified and the reference features of the real face.

11. The method according to claim 6, wherein the second real live face dataset is a subset of the first real live face dataset, and the first real live face dataset is a training set used for model training to obtain the live face recognition model.

12. A model training device, comprising: The first training module is used to perform self-supervised training on an initial face recognition model using an unlabeled image dataset until the initial face recognition model reaches the training cutoff condition to obtain a corresponding first face recognition model. The initial face recognition model includes a feature extractor, and the model parameters of the first face recognition model are first parameters, which include first extractor parameters corresponding to the feature extractor. The second training module is used to perform self-supervised training on the first face recognition model using the first real live face dataset and the first extractor parameters as the initial parameters of the feature extractor, until the first face recognition model reaches the training cutoff condition, so as to obtain the corresponding second face recognition model. The model parameters of the second face recognition model are second parameters, and the second parameters include the second extractor parameters corresponding to the feature extractor. The third training module is used to perform asymmetric metric learning on the second face recognition model using a labeled real and fake face mixed dataset, with the second extractor parameters as the initial parameters of the feature extractor, until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model. The model parameters of the live face recognition model are the third parameters, which include the third extractor parameters corresponding to the feature extractor. The second face recognition model includes a classifier, a feature extractor, and a feature clusterer. The feature extractor is connected to the classifier and the feature clusterer, respectively. The loss function of the second face recognition model includes a first sub-loss function corresponding to the classifier and a second sub-loss function corresponding to the feature clusterer. The asymmetric metric learning of the second face recognition model is achieved through the following process: A labeled mixed dataset of real and fake faces is input into the feature extractor. The feature extractor then transmits the labeled mixed dataset of real and fake faces to the classifier. The classifier is trained using the first sub-loss function so that the trained live face recognition model has the ability to distinguish between real and fake faces. Simultaneously, the feature extractor transmits labeled real face data from the labeled mixed dataset of real and fake faces to the feature clusterer so that the feature clusterer is trained using the second sub-loss function so that the trained live face recognition model has the ability to concentrate the distribution of real face features.

13. The apparatus according to claim 12, further comprising a determining module, wherein during the asymmetric metric learning process of the second face recognition model, the loss function of the second face recognition model can be dynamically determined according to the determining module: The first preset weight value is determined as the weight value of the first sub-loss function; Calculate the ratio of the first sub-loss function value to the second sub-loss function value; The smaller of the ratio and the second preset weight value is determined as the weight value of the second sub-loss function; Multiply the first sub-loss function by the corresponding weight value to obtain the first product; Multiply the second sub-loss function by the corresponding weight value to obtain the second product; The sum of the first product and the second product is determined as the loss function.

14. The apparatus according to claim 12, wherein before the first face recognition model is self-supervised trained using a first real live face dataset and the first extractor parameters as initial parameters of the feature extractor until the first face recognition model reaches the training cutoff condition to obtain a corresponding second face recognition model, the apparatus further comprises: The first cropping module is used to crop the real face regions in the initial real live face dataset using a face detection algorithm, and to determine the cropped real face region image as the first real live face dataset.

15. The apparatus of claim 12, wherein before the second face recognition model is subjected to asymmetric metric learning using a labeled mixed dataset of real and fake faces, with the second extractor parameters as the initial parameters of the feature extractor, until the second face recognition model reaches the training cutoff condition to obtain a live face recognition model, the apparatus further comprises: The second cropping module is used to crop the real face regions in the initial real live face dataset using a face detection algorithm, and generate corresponding labels for the cropped real face region images. The third cropping module is used to crop the fake face region in the initial fake face dataset using the face detection algorithm, and generate corresponding labels for the cropped fake face region image; The processing module is used to perform data augmentation processing on labeled real human face region images and labeled fake human face region images to obtain the labeled real and fake face mixed dataset.

16. A live face detection device, comprising: The acquisition module is used to acquire the face image to be identified; The input module is used to input the face image to be identified into the live face recognition model and obtain the recognition result. The live face recognition model is trained by the method according to any one of claims 1-4. The recognition result is used to indicate whether the face image to be identified is a real face.

17. The apparatus according to claim 16, wherein before inputting the face image to be identified into the liveness detection model and obtaining the recognition result, the apparatus further comprises: The cropping module is used to crop the real face regions in the initial real live face dataset using a face detection algorithm, and to determine the cropped real face region image as the second real live face dataset. The extraction module is used to extract features from each image in the second real live face dataset based on the feature extractor of the live face recognition model. The calculation module is used to calculate the features of each image in the second real live face dataset to determine the real face baseline features.

18. The apparatus according to claim 17, wherein the computing module is specifically used for: Calculate the average feature value of each image in the second real live face dataset, and determine the average feature value as the real face baseline feature.

19. The apparatus according to claim 17 or 18, wherein the acquisition module is specifically used for: The features of the face image to be identified are extracted using the feature extractor of the live face recognition model. Calculate the feature distance between the features of the face image to be identified and the reference features of the real face; The recognition result is generated based on the feature distance.

20. The apparatus according to claim 19, wherein the acquisition module is specifically used for: When the feature distance is greater than a preset distance, a recognition result is generated to indicate that the face image to be identified is a fake face; When the feature distance is less than or equal to the preset distance, a recognition result is generated to indicate that the face image to be identified is a real human face.

21. The apparatus according to claim 19, wherein the acquisition module is specifically used for: The feature distance is obtained by calculating the Euclidean distance between the features of the face image to be identified and the reference features of the real face.

22. The apparatus of claim 17, wherein the second real live face dataset is a subset of the first real live face dataset, and the first real live face dataset is a training set used for model training to obtain the live face recognition model.

23. A computing device, comprising: A processor, and a memory communicatively connected to the processor; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory to implement the method as described in any one of claims 1 to 11.

24. A storage medium storing computer program instructions that, when executed, implement the method as described in any one of claims 1 to 11.

Images