A method, apparatus, and equipment for generating a liveness detection model.

Abstract

This specification discloses a method, apparatus, and device for generating a liveness detection model. The scheme may include: training a first liveness classification sub-model and a second liveness classification sub-model in an initial model using color image samples and depth image samples or reflective image samples determined based on the color image samples, respectively, to obtain a trained model; and then extracting the first liveness classification sub-model trained using color image samples from the trained model as the liveness detection model.

Description

Technical Field

[0001] This application relates to the field of liveness detection technology, and in particular to a method, apparatus and equipment for generating a liveness detection model. Background Technology

[0002] With the development of computer technology and optical imaging technology, image processing technology is increasingly being used in various scenarios for user identification. For example, in user authentication scenarios, attackers might attempt to impersonate legitimate users by placing photos, recorded videos, or even wax figures of legitimate users in front of the camera capturing user images. Therefore, liveness detection is necessary for the captured user images to reduce the risks in the user identification process. Currently, researchers are increasingly using deep learning technology to train liveness detection models for this purpose.

[0003] Therefore, improving the accuracy of liveness detection results generated by liveness detection models has become an urgent technical problem to be solved. Summary of the Invention

[0004] The embodiments of this specification provide a method, apparatus, and device for generating a liveness detection model, which can improve the accuracy of liveness detection results generated by the trained liveness detection model.

[0005] To solve the above-mentioned technical problems, the embodiments in this specification are implemented as follows:

[0006] This specification provides an embodiment of a method for generating a liveness detection model, comprising:

[0007] Obtain an initial model; the initial model includes a first liveness classification sub-model and a second liveness classification sub-model;

[0008] Based on a multi-task learning approach, the first liveness classification sub-model is trained using a first sample set, and the second liveness classification sub-model is trained using a second sample set to obtain a trained model; the first sample set contains color image samples, and the second sample set contains depth image samples or reflective image samples determined based on the color image samples.

[0009] The first liveness classification sub-model trained after training is extracted from the trained model to obtain the liveness detection model.

[0010] This specification provides an embodiment of a liveness detection model generation device, comprising:

[0011] An initial model acquisition module is used to acquire an initial model; the initial model includes a first liveness classification sub-model and a second liveness classification sub-model.

[0012] The initial model training module is used to train the first liveness classification sub-model using a first sample set and the second liveness classification sub-model using a second sample set, respectively, based on a multi-task learning approach, to obtain the trained model; the first sample set contains color image samples, and the second sample set contains depth image samples or reflective image samples determined based on the color image samples.

[0013] The detection model acquisition module is used to extract the trained first liveness classification sub-model from the trained model to obtain a liveness detection model.

[0014] This specification provides an embodiment of a device for generating a liveness detection model, comprising:

[0015] At least one processor; and,

[0016] A memory communicatively connected to the at least one processor; wherein,

[0017] The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to:

[0018] Obtain an initial model; the initial model includes a first liveness classification sub-model and a second liveness classification sub-model;

[0019] Based on a multi-task learning approach, the first liveness classification sub-model is trained using a first sample set, and the second liveness classification sub-model is trained using a second sample set to obtain a trained model; the first sample set contains color image samples, and the second sample set contains depth image samples or reflective image samples determined based on the color image samples.

[0020] The first liveness classification sub-model trained after training is extracted from the trained model to obtain the liveness detection model.

[0021] At least one embodiment provided in this specification can achieve the following beneficial effects:

[0022] By employing a multi-task learning approach, the first and second liveness classification sub-models in the initial model are trained using color image samples and depth or reflective image samples determined from the color image samples, respectively. This results in a trained model, from which the first liveness classification sub-model trained using color image samples can be extracted as the liveness detection model. This scheme leverages the fact that depth and reflective image samples typically carry significant differences between live and non-live objects. Therefore, by introducing a liveness classification sub-task based on depth and / or reflective image samples, the optimization of the main liveness detection task based on color image samples can be aided. This improves the stability and accuracy of the trained liveness detection model for color image recognition, thereby ensuring the stability and accuracy of the liveness detection results generated by the trained model. Attached Figure Description

[0023] To more clearly illustrate the technical solutions in the embodiments or prior art of this specification, the drawings used in the description of the embodiments or prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0024] Figure 1 This diagram illustrates an application scenario of a method for generating a liveness detection model provided in the embodiments of this specification.

[0025] Figure 2 A schematic flowchart illustrating a method for generating a liveness detection model provided in an embodiment of this specification;

[0026] Figure 3 The embodiments provided in this specification correspond to Figure 2 A swimlane flowchart illustrating the method for generating a liveness detection model in the image.

[0027] Figure 4 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of a device for generating a liveness detection model;

[0028] Figure 5 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of the structure of a device for generating a liveness detection model. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of one or more embodiments of this specification clearer, the technical solutions of one or more embodiments of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the protection scope of one or more embodiments of this specification.

[0030] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0031] With the widespread application of facial recognition technology, risks have also emerged. For example, criminals can use printed photos or videos to cheat or attack the facial recognition process. To address these issues, liveness detection technology has been developed.

[0032] Currently, common liveness detection techniques typically use facial image samples combined with facial tag data to train a liveness detection model. This model aims to identify whether the person performing the facial recognition operation is a genuine, living individual or a malicious attacker using printed photos, silicone masks, or played videos. However, the facial tag data currently used, such as gender, age, and whether a mask is being worn, cannot accurately reflect the differences between live and malicious individuals, thus affecting the accuracy and stability of the trained liveness detection model.

[0033] To address the shortcomings of existing technologies, this solution provides the following embodiments:

[0034] Figure 1 This diagram illustrates an application scenario of a method for generating a liveness detection model provided in the embodiments of this specification. Figure 1 As shown, the liveness detection model generation method provided in the embodiments of this specification can be executed using server 103. Specifically, server 103 can determine depth image samples or reflective image samples using color image samples, and then, based on multi-task learning, train the first liveness classification sub-model and the second liveness classification sub-model in the initial model using color image samples and depth image samples or reflective image samples respectively, to obtain a trained model. The trained first liveness classification sub-model is extracted from the trained model as a liveness detection model, which can be used to generate liveness detection results for color images.

[0035] Subsequently, when user 101 performs facial recognition at device 102, device 102 can acquire a color image of user 101 to obtain a color image to be detected. In practical applications, the liveness detection model on device 102 can be used directly to perform liveness detection on the color image to be detected. Alternatively, device 102 can send the color image to be detected to server 103, so that the liveness detection model on server 103 can perform liveness detection on the color image to obtain the liveness detection result for user 101. No specific limitation is made in this regard. Device 102 and server 103 can be the client and server of the application used for liveness detection, respectively.

[0036] Figure 1 The proposed solution addresses this issue by recognizing that depth images obtained from processing live objects typically reflect the relative depth features of various parts of the human body and face, while depth images obtained from processing non-live objects usually fail to do so. Furthermore, reflective images obtained from processing live objects typically reflect diffuse lighting characteristics, while reflective images obtained from processing non-live objects typically reflect reflected lighting characteristics. Therefore, depth and reflective images often carry significant differences between live and non-live objects. Based on this, by incorporating this significant difference between live and non-live objects from depth and reflective image samples into a liveness classification subtask based on depth and / or reflective image samples, the main task of the liveness detection model based on color image samples can be optimized. This improves the stability and accuracy of the trained liveness detection model for color images.

[0037] Next, a method for generating a liveness detection model provided in the embodiments of the specification will be described in detail with reference to the accompanying drawings:

[0038] Figure 2 This is a flowchart illustrating a method for generating a liveness detection model according to an embodiment of this specification. From a programming perspective, the entity executing this process can be a device used for model training, or it can be an application program installed on the aforementioned device. Figure 2 As shown, the process may include the following steps:

[0039] Step 202: Obtain the initial model; the initial model includes a first live classifier sub-model and a second live classifier sub-model.

[0040] In the embodiments of this specification, the first liveness classification sub-model can be used to perform liveness detection and recognition on color images, generating detection results that indicate whether the user performing the liveness detection is a real live user or an object exhibiting liveness attack behavior; the second liveness classification sub-model can be used to perform liveness detection and recognition on depth images or reflective images generated based on color images, generating detection results that indicate whether the user performing the liveness detection is a real live user or an object exhibiting liveness attack behavior.

[0041] In the embodiments of this specification, the first liveness classification sub-model and the second liveness classification sub-model can be binary or multi-class detection models built on convolutional neural networks, or they can be models based on machine learning, such as models built on at least one of logistic regression, decision trees, random forests, gradient boosting trees, multilayer perceptrons, linear SVMs, Naive Bayes, etc.; no specific limitation is made in this regard.

[0042] In the embodiments of this specification, the first liveness classification sub-model and the second liveness classification sub-model may not have shared network layers, but the hidden layers of the first liveness classification sub-model and the second liveness classification sub-model can be connected so that at least some features between the hidden layers of the two sub-models can be fused, and the fused at least some features can be passed to the subsequent network layers of the two sub-models, thereby realizing feature parameter sharing between the two sub-models.

[0043] Step 204: Based on multi-task learning, the first liveness classification sub-model is trained using the first sample set, and the second liveness classification sub-model is trained using the second sample set to obtain the trained model; the first sample set contains color image samples, and the second sample set contains depth image samples or reflective image samples determined based on the color image samples.

[0044] In the embodiments described in this specification, the multi-task learning is a joint learning method in which multiple tasks are learned in parallel and their results influence each other. Specifically, training a first liveness classification sub-model using color image samples can be taken as the main task, while training a second liveness classification sub-model using depth image samples or reflective image samples can be taken as an auxiliary task.

[0045] The training process of the first and second liveness classification sub-models may include: inputting color image samples into the first liveness classification sub-model and performing reverse optimization on the first liveness classification sub-model using the loss function corresponding to the first liveness classification sub-model, thereby training the first liveness classification sub-model; similarly, depth image samples or reflective image samples may be input into the second liveness classification sub-model and performing reverse optimization on the second liveness classification sub-model using the loss function corresponding to the second liveness classification sub-model, thereby training the second liveness classification sub-model to obtain the trained model.

[0046] In the embodiments of this specification, based on a multi-task learning approach, feature parameters are shared between the first and second liveness classification sub-models. This allows the information contained in depth image samples or reflective image samples to be referenced during the training and optimization process of the first liveness classification sub-model, which serves as the main task. This enables them to share the information they have learned, increasing the data dimensionality of the data that the first liveness classification sub-model can use. Consequently, this helps to improve the accuracy of the training of the first liveness classification sub-model, allowing the trained first liveness classification sub-model to achieve better generalization results.

[0047] In the embodiments described in this specification, the color image samples included in the first sample set can be images captured by the device for the user during face recognition or liveness detection operations. In practical applications, the device can be a facial recognition payment machine in a store or a user terminal device. Of course, the color image samples included in the first sample set can also be images extracted from an existing face database, or images obtained by staff through deliberate image capture; no specific limitation is made in this regard.

[0048] In the embodiments of this specification, the depth image sample may refer to an image sample in which the distance (depth) from the image acquisition device to the sample object is used as the pixel value. It directly reflects the geometry of the visible surface of the sample object (e.g., a human face). When the sample object is inanimate, the pixel values ​​of each pixel in the depth image sample tend to be consistent. The reflective image sample may refer to an image used to reflect the reflected light information of the sample object. When the sample object is inanimate, the reflectivity of each pixel in the reflective image generally conforms to specular reflection characteristics. When the sample object is alive, the reflectivity of each pixel in the reflective image generally conforms to diffuse reflection characteristics.

[0049] Step 206: Extract the first liveness classification sub-model after training from the trained model to obtain the liveness detection model.

[0050] In the embodiments of this specification, since in scenarios requiring liveness detection (e.g., facial payment, facial recognition), devices typically acquire color images as the images to be detected, a first liveness classification sub-model trained based on color image samples can be extracted as a liveness detection model. Based on this, the trained first liveness classification sub-model can be used to perform liveness detection on the color images acquired by the device in scenarios requiring liveness detection (e.g., facial payment, facial recognition).

[0051] Figure 2 The proposed scheme trains the first and second liveness classification sub-models in the initial model using color image samples and depth or reflective image samples determined from the color image samples, respectively, to obtain the trained model. Since the depth and reflective image samples carry significant differential representation information between live and non-live objects, a liveness classification sub-task based on depth and / or reflective image samples is introduced to assist in optimizing the main liveness detection task based on color image samples. This improves the accuracy of training the first liveness classification sub-model for liveness recognition using color images, and is convenient and fast.

[0052] In addition, before generating the liveness detection result, it is usually unknown whether the target object corresponding to the acquired color image to be detected is a live object. However, regardless of whether the target object is a live or non-live object, there is usually a certain depth difference between the pixels in the depth image extracted from the color image to be detected, and some pixels in the reflection image extracted from the color image to be detected usually also conform to the diffuse reflection characteristics. This makes the depth image and the reflection image biased in indicating that the target object is a live object. As a result, the detection result generated by using the second liveness classification sub-model to detect the depth image or reflection image determined based on the color image to be detected is usually inaccurate. Therefore, extracting the trained first liveness classification sub-model from the trained model as the liveness detection model is also beneficial to further ensure the accuracy and stability of the liveness detection model.

[0053] based on Figure 2 In addition to the method described in the embodiments of this specification, some specific implementation schemes of the method are also provided, which will be described below.

[0054] In the embodiments of this specification, since the depth image samples and the reflection image samples carry obvious differential representation information between living and non-living objects, the depth sample set and the reflection sample set can be used simultaneously to assist in training the first living object classification sub-model in the initial model, so as to improve the accuracy of training the first living object classification sub-model.

[0055] Based on this, the second sample set may include a depth sample set and a reflective sample set.

[0056] Before training the second liveness classification sub-model using the second sample set, the method further includes:

[0057] If the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is a living object, then the depth estimation model is used to perform depth estimation processing on the color image sample to obtain the depth image sample corresponding to the color image sample.

[0058] The depth sample set is generated based on the depth image samples corresponding to the color image samples.

[0059] If the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is not a living organism, then the color image sample is processed by illumination feature analysis using an illumination model to obtain the reflected image sample corresponding to the color image sample.

[0060] The reflective sample set is generated based on the reflective image samples corresponding to the color image samples.

[0061] In the embodiments of this specification, the second sample set may include multiple sample sets, wherein the samples in one of the multiple sample sets are samples of the same type, that is, the samples in a sample set can only be depth image samples or reflection image samples. For example, for the second sample set {N1, N2, N3...N... m}, N1, N2, N3...N m Each of the N samples is a sample set in the second sample set. m The samples in the image can only be either depth image samples or reflection image samples, and N1 and N m The samples in the samples can be different.

[0062] In practical applications, the classification labels carried by color image samples are used to indicate whether the sample objects in the color image samples are living or non-living. The classification labels can be generated manually to ensure the accuracy of the labels.

[0063] In practical applications, the depth image samples can be obtained by performing depth estimation processing on the color image samples using a depth estimation model. The depth estimation model can be a model obtained by training an existing MiDaS model.

[0064] The reflective image sample can be obtained by performing illumination feature analysis on the color image sample using an illumination model. This illumination feature analysis may include the illumination model (or illumination model) analyzing whether the reflectance of each pixel in the color image sample exhibits specular or diffuse reflection characteristics, thereby generating a reflective image containing reflectance information and the analysis results.

[0065] In the embodiments of this specification, since both the depth sample set and the reflective sample set are used as training data to assist in multi-task learning of the first liveness classification sub-model in the initial model, the dimensionality of the training data is increased, thereby improving the accuracy of the liveness detection model training.

[0066] In practical applications, in order to ensure the difference between positive and negative samples in the second sample set, depth image samples corresponding to non-live subjects and reflective image samples corresponding to live subjects can be manually set to ensure the accuracy of training the first live subject classification sub-model in the main task.

[0067] Based on this, before training the second liveness classification sub-model using the second sample set, the process may further include:

[0068] If the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is not a living organism, then the first solid color image is determined as the depth image sample corresponding to the color image sample.

[0069] If the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is a living body, then the second solid color image is determined as the reflective image sample corresponding to the color image sample.

[0070] In the embodiments of this specification, when the sample object is not a living person, for example, when the sample object is used for liveness detection using a printed color face image, the difference in pixel values ​​of each pixel in the depth image sample determined based on the color image sample acquired from the color face image should be relatively small, indicating that the sample object does not have the depth features of a human body or face. However, in practical applications, if the sample object is bent to a certain extent in relation to the printed color face image, it may coincidentally lead to the depth image sample determined based on the color image sample matching a small number of human body or face depth features, thus making the depth image sample biased in indicating that the sample object is a living person. Obviously, this is inconsistent with the fact that the sample object is not a living person.

[0071] Therefore, if the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is not a living organism, then a first solid color image can be used instead of the depth image sample determined based on the color image sample. In this first solid color image, the pixel values ​​of each pixel are consistent, and the pixel values ​​of each pixel in the first solid color image can be used to reflect the depth value of the sample object.

[0072] In the embodiments of this specification, when the sample object is a living body, due to interference from factors such as facial makeup and sweating, the reflective image sample determined based on the color image sample collected from the sample object may have some pixels whose pixel values ​​conform to specular reflection characteristics. This makes the reflective image sample biased in indicating that the sample object is not a living body, which is obviously inconsistent with the fact that the sample object is a living body.

[0073] Therefore, if the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is a living body, then a second solid color image can be used instead of the reflective image sample determined based on the color image sample. In this second solid color image, the pixel values ​​of each pixel are consistent, and the pixel values ​​of each pixel in the second solid color image can be used to reflect the reflectivity of the sample object.

[0074] In the embodiments of this specification, depth image samples corresponding to non-living objects and reflective image samples corresponding to living objects are manually set to ensure the difference between positive and negative samples in the second sample set and to ensure the accuracy of training samples in the second sample set, thereby improving the accuracy of training the first living object classification sub-model.

[0075] In the embodiments of this specification, in order to make full use of the depth image samples and reflection image samples, the second liveness classification sub-model may include a depth classification model and a reflection classification model, so that each training task corresponding to the depth classification model and the reflection classification model is used as an auxiliary task to assist the training of the first liveness classification sub-model corresponding to the main task, thereby improving the accuracy of the training results of the main task.

[0076] Based on this, the second liveness classification sub-model may include: a depth classification model and a reflectivity classification model.

[0077] The multi-task learning approach trains the first liveness classification sub-model using a first sample set and the second liveness classification sub-model using a second sample set to obtain the trained model, specifically including:

[0078] Based on a multi-task learning approach, the first liveness classification sub-model is trained using the first sample set, the depth classification model is trained using the depth sample set, and the reflective classification model is trained using the reflective sample set, thereby obtaining the trained model.

[0079] In the embodiments described in this specification, theoretically, the second liveness classification sub-model can include only a depth classification model, or only a reflection classification model, or both a depth classification model and a reflection classification model. When both a depth classification model and a reflection classification model are included, the hidden layers of the three models can be connected, allowing the feature information of color image samples, depth image samples, and reflection image samples in the hidden layers of the three models to be fused and passed on. This helps to further increase the feature dimension of the feature data that can be obtained at the first liveness classification sub-model, thereby improving the accuracy of training the first liveness classification sub-model.

[0080] In the embodiments of this specification, since the background area other than the face area in the color image sample can also reflect whether the sample object is a live attack, for example, since criminals may continue to cheat or attack in the same environment, the environmental information contained in the background area of ​​the color image sample can be combined to determine the attack risk of this face scan.

[0081] Based on this, the first sample set may include: a global color image sample set and a local color image sample set; the global color image sample set includes global color image samples carrying facial information and background information, and the local color image sample set includes local color image samples carrying the facial information.

[0082] The depth sample set includes a global depth sample set and a local depth sample set; the global depth sample set contains depth image samples corresponding to the global color image samples, and the local depth sample set contains depth image samples corresponding to the local color image samples.

[0083] The reflective sample set includes a global reflective sample set and a local reflective sample set; the global reflective sample set contains reflective image samples corresponding to the global color image samples, and the local reflective sample set contains reflective image samples corresponding to the local color image samples.

[0084] In the embodiments of this specification, the background information may typically include environmental information of the sample object, such as windows, doors, shelves, signs, etc. around the sample object.

[0085] In the embodiments of this specification, a depth estimation model can be used to generate a global depth image sample based on global color image samples, and an illumination model can be used to generate a global reflection sample based on the global color image samples. Similarly, a depth estimation model can be used to generate a local depth image sample based on local color image samples, and an illumination model can be used to generate a local reflection sample based on the local color image samples. The global color image sample, the global depth image sample, and the global reflection sample constitute a global image sample; the local color image sample, the local depth sample, and the local reflection sample constitute a local image sample.

[0086] In the embodiments of this specification, the local image samples carrying facial information and the global image samples carrying both facial and background information increase the data dimension of the sample data compared to the local image samples carrying facial information, thereby improving the accuracy of training the first liveness classification sub-model corresponding to the main task.

[0087] To facilitate understanding, the implementation methods for generating global color image samples and local color image samples are explained here. Specifically, before training the first liveness classification sub-model using the first sample set, the following may also be included:

[0088] Acquire color images captured by the device.

[0089] Face recognition processing is performed on the color image to obtain the predicted face region in the color image.

[0090] The image within the predicted face region is extracted from the color image to obtain the local color image sample.

[0091] The global color image sample is obtained by extracting an image from a designated region, including the predicted face region, from the color image; the area of ​​the designated region is larger than the area of ​​the predicted face region.

[0092] In the embodiments of this specification, a trained face detection model can be used to identify and process the color image of the user collected by the device to obtain a predicted face region carrying face information. The predicted face region can be the region in the prediction box output by the trained face detection model, thereby allowing the extraction of local color image samples from the image within the predicted face region.

[0093] Subsequently, the predicted face region can be expanded outward according to a certain proportion or size to obtain a global color image sample carrying face and background information; of course, the color image collected by the device can also be directly used as a global color image sample, which is convenient and quick, and no specific limitation is made on this.

[0094] To facilitate understanding of how to apply the first sample set, depth sample set, and reflection sample set to train the initial model, the embodiments in this specification are described in detail here:

[0095] Specifically, the first liveness classification sub-model includes a first classification model and a second classification model, the depth classification model includes a third classification model and a fourth classification model, and the reflectivity classification model includes a fifth classification model and a sixth classification model.

[0096] The multi-task learning approach trains the first liveness classification sub-model using the first sample set, the deep classification model using the deep sample set, and the reflective classification model using the reflective sample set to obtain the trained model. Specifically, this includes:

[0097] Based on a multi-task learning approach, the first classification model is trained using the global color image sample set, the third classification model is trained using the global depth sample set, and the fifth classification model is trained using the global reflectivity sample set, resulting in a first trained model; and...

[0098] Based on a multi-task learning approach, the second classification model is trained using the local color image sample set, the fourth classification model is trained using the local depth sample set, and the sixth classification model is trained using the local reflective sample set, thereby obtaining the second trained model.

[0099] Correspondingly, step 206: Extract the trained first liveness classification sub-model from the trained model to obtain the liveness detection model, which may specifically include:

[0100] The first classification model after training is extracted from the first trained model, and the second classification model after training is extracted from the second trained model to obtain the liveness detection model.

[0101] In the embodiments of this specification, the first classification model and the second classification model may have the same or different structures, the third classification model and the fourth classification model may have the same or different structures, and the fifth classification model and the sixth classification model may have the same or different structures; no specific limitations are made in this regard.

[0102] In the embodiments of this specification, the training of the overall model can be divided into two parts. The first part is to train the first classification model, the third classification model, and the fifth classification model using global image samples in a multi-task manner to obtain the first trained model. The second part is to train the second classification model, the fourth classification model, and the sixth classification model using local image samples in a multi-task manner to obtain the second trained model.

[0103] In practical applications, the connection between the hidden layers of the first classification model and the third and fifth classification models in the first trained model can be disconnected to obtain the first classification model after training. Similarly, the connection between the hidden layers of the second classification model and the fourth and sixth classification models in the second trained model can be disconnected to obtain the second classification model after training.

[0104] In the embodiments of this specification, the first classification model can be subsequently used to perform liveness detection and recognition on a global color image to obtain a liveness detection and recognition result for the global color image; the second classification model can be subsequently used to perform liveness detection and recognition on a local color image to obtain a liveness detection and recognition result for the local color image. The first classification model and the second classification model are two independent models, that is, they may not have connected network layers or parameter interaction based on feature fusion.

[0105] In the embodiments of this specification, the connection relationships between the various classification models are explained in detail below:

[0106] Specifically, the first classification model, the third classification model, and the fifth classification model do not share a network layer, and the first hidden layer in the first classification model, the second hidden layer in the third classification model, and the third hidden layer in the fifth classification model have a first connection relationship. The first connection relationship is used to perform feature fusion processing on some feature vectors at the first hidden layer, the second hidden layer, and the third hidden layer.

[0107] The second classification model, the fourth classification model, and the sixth classification model do not share a network layer, and the fourth hidden layer in the second classification model, the fifth hidden layer in the fourth classification model, and the sixth hidden layer in the sixth classification model have a second connection relationship. The second connection relationship is used to perform feature fusion processing on some feature vectors at the fourth hidden layer, the fifth hidden layer, and the sixth hidden layer.

[0108] Furthermore, the network layers at the first classification model, the third classification model, and the fifth classification model are not connected to the network layers at the second classification model, the fourth classification model, and the sixth classification model.

[0109] In the embodiments of this specification, the feature fusion can use the ADD algorithm in a neural network to fuse the features output by the hidden layer. The feature vector dimensions of the first hidden layer, the second hidden layer, and the third hidden layer that are used for feature fusion should be the same to ensure the feasibility of fusion. Similarly, the feature vector dimensions of the fourth hidden layer, the fifth hidden layer, and the sixth hidden layer that are used for feature fusion should also be the same.

[0110] In the embodiments of this specification, each classification model may include multiple hidden layers, which are used to extract features from their input image samples. The first connection relationship indicates that the first hidden layer in the first classification model, the second hidden layer in the third classification model, and the third hidden layer in the fifth classification model can be connected. Since the information carried by the reflective image and the depth image is very helpful in assisting the training of the liveness detection model, the first connection relationship can be used to fuse some feature vectors extracted from the first hidden layer, the second hidden layer, and the third hidden layer, and the fused feature can replace the fused feature vector. This allows the fused feature data to be used to continue training the above multiple models, realizing parameter sharing among the models and improving the accuracy of the model training results.

[0111] Similarly, based on the second connection relationship, feature fusion processing can be performed on some feature vectors in the fourth hidden layer of the second classification model, the fifth hidden layer of the fourth classification model, and the sixth hidden layer of the sixth classification model. The fused features can then replace the fused feature vectors, allowing the fused feature data to be used to continue training the above multiple models, achieving parameter sharing among models and improving the accuracy of model training results.

[0112] In the embodiments described in this specification, typically only some features at the first hidden layer, the second hidden layer, and the third hidden layer are fused and replaced, while the features at the aforementioned hidden layers are retained without fusion and replacement. This helps to ensure the independence and stability of the trained model.

[0113] It is worth noting that the first classification model, the third classification model, and the fifth classification model typically have their own unique network layers and do not directly share any network layer. Similarly, the second classification model, the fourth classification model, and the sixth classification model typically have their own unique network layers and do not directly share any network layer. This is to ensure the accuracy of the subsequently extracted liveness detection model.

[0114] In practical applications, the training of liveness detection models is usually done to make the models more accurate, so that more accurate models can be used for liveness detection in the face recognition process.

[0115] Based on this, after extracting the trained first classification model from the first trained model and the trained second classification model from the second trained model to obtain the liveness detection model, the process may further include:

[0116] Acquire the color image to be detected.

[0117] Based on the color image to be detected, a local color image to be detected and a global color image to be detected are generated; the local color image to be detected contains the face information to be detected, and the global color image to be detected contains the face information to be detected and the background information to be detected.

[0118] The trained first classification model generates a first liveness detection result based on the global color image to be detected.

[0119] The trained second classification model is used to generate a second liveness detection result based on the local color image to be detected.

[0120] Based on the first liveness detection result and the second liveness detection result, a liveness detection result is generated for the color image to be detected.

[0121] In the embodiments of this specification, the color image to be detected may be a facial recognition image of the target object to be detected acquired by the device.

[0122] In the embodiments of this specification, the principle of generating local and global color images to be detected based on the color image to be detected is the same as the principle of determining the local and global color samples based on the color image samples described above, and will not be repeated here.

[0123] In the embodiments of this specification, the first classification model can be used to detect and identify the global color image to be detected, so as to obtain a first liveness detection result that can be used to indicate whether the target object to be detected is a live body; the second classification model can be used to detect and identify the local color image to be detected, so as to obtain a second liveness detection result that can be used to indicate whether the target object to be detected is a live body; so that the liveness detection result of the color image to be detected of the target object can be determined based on the first liveness detection result and the second liveness detection result.

[0124] In the embodiments of this specification, the first liveness detection result may include: a first liveness probability value and a first non-liveness probability value; the second liveness detection result may include: a second liveness probability value and a second non-liveness probability value.

[0125] The step of generating a liveness detection result for the color image to be detected based on the first liveness detection result and the second liveness detection result may specifically include:

[0126] Based on the first liveness probability value and the second liveness probability value, a comprehensive liveness probability value is generated for the color image to be detected;

[0127] If the overall liveness probability value is greater than the first threshold, a liveness detection result is generated to indicate that the object corresponding to the detected face information is a live object; or,

[0128] Based on the first non-inactive probability value and the second non-inactive probability value, a comprehensive non-inactive probability value is generated for the color image to be detected;

[0129] If the overall non-living probability value is greater than the second threshold, a liveness detection result is generated to indicate that the object corresponding to the face information to be detected is a non-living object.

[0130] In the embodiments of this specification, the weights of the first liveness probability value and the second liveness probability value can be set as a first preset weight and a second preset weight, respectively. The first preset weight and the second preset weight can be set according to actual needs, but the first preset weight and the second preset weight are usually positive numbers; for example, they can be set to 0.6 and 0.4 or 0.7 and 0.3, respectively, without specific limitation. Based on this, the sum of the product of the first liveness probability value and the weight of the first liveness probability value and the product of the second liveness probability value and the weight of the second liveness probability value can be determined as the comprehensive liveness probability value for the color image to be detected. Similarly, the comprehensive non-liveness probability value of the color image to be detected can also be generated using the same method, which will not be elaborated here.

[0131] In the embodiments described in this specification, the first threshold and the second threshold can be set according to actual needs, and no specific limitation is made thereto.

[0132] Of course, in practical applications, it is also possible to directly determine whether the sum of the first live probability value and the second live probability value is greater than the sum of the first non-live probability value and the second non-live probability value; if so, a live detection result can be generated to indicate that the object corresponding to the face information to be detected is a live person; if not, a live detection result can be generated to indicate that the object corresponding to the face information to be detected is a non-live person.

[0133] In this embodiment, the first liveness probability value and the first non-liveness probability value are liveness detection results predicted by combining background information in the color image to be detected, while the second liveness probability value and the second non-liveness probability value are liveness detection results predicted by focusing on facial texture details in the color image to be detected. This allows the scheme to fully combine background information and facial information in the image to be detected to generate liveness detection results, which is beneficial to improving the accuracy and effectiveness of the generated liveness detection results.

[0134] Figure 3 The embodiments provided in this specification correspond to Figure 2 The swimlane flowchart illustrates the method for generating the liveness detection model in [the dataset]. (For example...) Figure 3 As shown, the process of generating the liveness detection model may involve execution entities such as the image acquisition device to be detected and the liveness detection model generation device. In this specification, the liveness detection model generation device and the liveness detection model application device are described as the same device. It can be understood that in actual applications, the liveness detection model generation device and the liveness detection model application device may not be the same device.

[0135] In the liveness detection model generation stage, the liveness detection model generation device can first generate a first initial model including a first classification model, a third classification model, and a fifth classification model, and a second initial model including a second classification model, a fourth classification model, and a sixth classification model. Simultaneously, it can acquire color image samples collected by the device. Then, based on the color image samples, it determines global color image samples and local color image samples. Finally, based on the global color image samples, it determines global depth samples and global reflection samples, and based on the local color image samples, it determines local depth samples and local reflection samples. This allows for multi-task learning, where the first classification model, the third classification model, and the fifth classification model are trained using the global color image samples, global depth samples, and global reflection samples, respectively. Simultaneously, based on multi-task learning, the second classification model, the fourth classification model, and the sixth classification model are trained using the local color image samples, local depth samples, and local reflection samples, respectively. Thus, the trained first classification model and the trained second classification model can be extracted as the liveness detection model.

[0136] In the liveness detection model application stage, the image acquisition device can acquire a color image of the user to be detected and send the acquired color image to the liveness detection model generation device. This allows the liveness detection model generation device to determine the global color image and the local color image to be detected based on the color image. The trained first classification model can then be used to perform liveness detection on the global color image to generate a first liveness detection result, and the trained second classification model can be used to perform liveness detection on the local color image to generate a second liveness detection result. Finally, based on the first and second liveness detection results, a liveness detection result for the color image to be detected is generated.

[0137] Based on the same idea, embodiments of this specification also provide apparatus corresponding to the above methods. Figure 4 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of a device for generating a liveness detection model. (See attached diagram.) Figure 4 As shown, the device may include:

[0138] The initial model acquisition module 402 is used to acquire an initial model; the initial model includes a first live classifier sub-model and a second live classifier sub-model.

[0139] The initial model training module 404 is used to train the first liveness classification sub-model using a first sample set and the second liveness classification sub-model using a second sample set, respectively, based on a multi-task learning approach, to obtain a trained model; the first sample set contains color image samples, and the second sample set contains depth image samples or reflective image samples determined based on the color image samples.

[0140] The detection model acquisition module 406 is used to extract the trained first liveness classification sub-model from the trained model to obtain a liveness detection model.

[0141] based on Figure 4 The embodiments of this specification also provide some specific implementations of the device, which will be described below.

[0142] Optional, Figure 4 The device may further include:

[0143] The first color image processing module is used to perform depth estimation processing on the color image sample using a depth estimation model if the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is a living body, so as to obtain the depth image sample corresponding to the color image sample.

[0144] The depth sample set generation module is used to generate the depth sample set based on the depth image samples corresponding to the color image samples.

[0145] The second color image processing module is used to perform illumination feature analysis on the color image sample using an illumination model if the classification label data carried by the color image sample indicates that the sample object in the color image sample is not a living body, thereby obtaining the reflective image sample corresponding to the color image sample.

[0146] The reflective sample set generation module is used to generate the reflective sample set based on the reflective image samples corresponding to the color image samples.

[0147] Optional, Figure 4 The device may further include:

[0148] The first solid color image determination module is used to determine the first solid color image as the depth image sample corresponding to the color image sample if the classification label data carried by the color image sample indicates that the sample object in the color image sample is a non-living object.

[0149] The second solid color image determination module is used to determine the second solid color image as the reflective image sample corresponding to the color image sample if the classification label data carried by the color image sample indicates that the sample object in the color image sample is a living body.

[0150] Optionally, the second liveness classification sub-model may include: a depth classification model and a reflectivity classification model.

[0151] The initial model training module can be specifically used for:

[0152] Based on a multi-task learning approach, the first liveness classification sub-model is trained using the first sample set, the depth classification model is trained using the depth sample set, and the reflective classification model is trained using the reflective sample set, thereby obtaining the trained model.

[0153] Optionally, the first sample set may include: a global color image sample set and a local color image sample set; the global color image sample set includes global color image samples carrying facial information and background information, and the local color image sample set includes local color image samples carrying the facial information.

[0154] The depth sample set may include: a global depth sample set and a local depth sample set; the global depth sample set contains depth image samples corresponding to the global color image samples, and the local depth sample set contains depth image samples corresponding to the local color image samples.

[0155] The reflective sample set may include: a global reflective sample set and a local reflective sample set; the global reflective sample set contains reflective image samples corresponding to the global color image samples, and the local reflective sample set contains reflective image samples corresponding to the local color image samples.

[0156] Optional, Figure 4 The device may further include:

[0157] The color image acquisition module is used to acquire color images captured by the device.

[0158] The predicted face region acquisition module is used to perform face recognition processing on the color image to obtain the predicted face region in the color image.

[0159] The local color image sample acquisition module is used to extract the image within the predicted face region from the color image to obtain the local color image sample.

[0160] A global color image sample acquisition module is used to extract an image from a specified region, including the predicted face region, from the color image to obtain the global color image sample; the area of ​​the specified region is larger than the area of ​​the predicted face region.

[0161] Optionally, the first liveness classification sub-model may include a first classification model and a second classification model, the depth classification model may include a third classification model and a fourth classification model, and the reflectivity classification model may include a fifth classification model and a sixth classification model.

[0162] The initial model training module can be specifically used for:

[0163] Based on a multi-task learning approach, the first classification model is trained using the global color image sample set, the third classification model is trained using the global depth sample set, and the fifth classification model is trained using the global reflectivity sample set, resulting in a first trained model; and...

[0164] Based on a multi-task learning approach, the second classification model is trained using the local color image sample set, the fourth classification model is trained using the local depth sample set, and the sixth classification model is trained using the local reflective sample set, thereby obtaining the second trained model.

[0165] The detection model acquisition module can be specifically used for:

[0166] The first classification model after training is extracted from the first trained model, and the second classification model after training is extracted from the second trained model to obtain the liveness detection model.

[0167] Optionally, the first classification model, the third classification model, and the fifth classification model do not share a network layer, and the first hidden layer in the first classification model, the second hidden layer in the third classification model, and the third hidden layer in the fifth classification model have a first connection relationship. The first connection relationship is used to perform feature fusion processing on some feature vectors at the first hidden layer, the second hidden layer, and the third hidden layer.

[0168] The second classification model, the fourth classification model, and the sixth classification model do not share a network layer, and the fourth hidden layer in the second classification model, the fifth hidden layer in the fourth classification model, and the sixth hidden layer in the sixth classification model have a second connection relationship. The second connection relationship is used to perform feature fusion processing on some feature vectors at the fourth hidden layer, the fifth hidden layer, and the sixth hidden layer.

[0169] Optionally, the network layers at the first classification model, the third classification model, and the fifth classification model are not connected to the network layers at the second classification model, the fourth classification model, and the sixth classification model.

[0170] Optional, Figure 4 The device may further include:

[0171] The color image acquisition module is used to acquire the color image to be detected.

[0172] A global color image to be detected generation module is used to generate a local color image to be detected and a global color image to be detected based on the color image to be detected; the local color image to be detected contains the face information to be detected, and the global color image to be detected contains the face information to be detected and the background information to be detected.

[0173] The first liveness detection result generation module is used to generate a first liveness detection result based on the global color image to be detected using the trained first classification model.

[0174] The second liveness detection result generation module is used to generate a second liveness detection result based on the local color image to be detected using the trained second classification model.

[0175] The third liveness detection result generation module is used to generate a liveness detection result for the color image to be detected based on the first liveness detection result and the second liveness detection result.

[0176] Optionally, the first liveness detection result may include: a first liveness probability value and a first non-liveness probability value; the second liveness detection result may include: a second liveness probability value and a second non-liveness probability value;

[0177] The third liveness detection result generation module can be specifically used for:

[0178] A comprehensive liveness probability value is generated for the color image to be detected based on the first liveness probability value and the second liveness probability value.

[0179] If the overall liveness probability value is greater than the first threshold, a liveness detection result is generated to indicate that the object corresponding to the detected face information is a live object; or,

[0180] Based on the first non-living probability value and the second non-living probability value, a comprehensive non-living probability value is generated for the color image to be detected.

[0181] If the overall non-living probability value is greater than the second threshold, a liveness detection result is generated to indicate that the object corresponding to the face information to be detected is a non-living object.

[0182] Based on the same idea, this specification also provides devices corresponding to the above methods in its embodiments.

[0183] Figure 5 The embodiments provided in this specification correspond to Figure 2 A schematic diagram of a device for generating a liveness detection model. (See diagram below.) Figure 5 As shown, device 500 may include:

[0184] At least one processor 510; and,

[0185] Memory 530 communicatively connected to the at least one processor; wherein,

[0186] The memory 530 stores instructions 520 that can be executed by the at least one processor 510, the instructions being executed by the at least one processor 510 to enable the at least one processor 510 to:

[0187] Obtain an initial model; the initial model includes a first live classifier sub-model and a second live classifier sub-model.

[0188] Based on a multi-task learning approach, the first liveness classification sub-model is trained using a first sample set, and the second liveness classification sub-model is trained using a second sample set to obtain a trained model; the first sample set contains color image samples, and the second sample set contains depth image samples or reflective image samples determined based on the color image samples.

[0189] The first liveness classification sub-model trained after training is extracted from the trained model to obtain the liveness detection model.

[0190] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on its differences from other embodiments. In particular, for... Figure 5 As the device shown is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0191] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program a digital system themselves to "integrate" it onto a PLD, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0192] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, ASICs, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0193] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, a laptop computer, a cellular phone, a camera phone, a smartphone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or any combination of these devices.

[0194] For ease of description, the above devices are described separately by function as various units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware.

[0195] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0196] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0197] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0198] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0199] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0200] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0201] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital character versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0202] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0203] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0204] This application can be described in the general context of computer-executable instructions, such as program modules, that are executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This application can also be practiced in distributed computing environments where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0205] The above description is merely an embodiment of this application and is not intended to limit the scope of this application. Various modifications and variations can be made to this application by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the scope of the claims of this application.

Claims

1. A method for generating a liveness detection model, comprising: Obtain the initial model; The initial model includes a first liveness classification sub-model and a second liveness classification sub-model; Based on a multi-task learning approach, the first liveness classification sub-model is trained using a first sample set, and the second liveness classification sub-model is trained using a second sample set to obtain a trained model. The first sample set contains color image samples, and the second sample set contains depth image samples or reflective image samples determined based on the color image samples. The first liveness classification sub-model and the second liveness classification sub-model do not share network layers, but the hidden layers of the first liveness classification sub-model and the hidden layers of the second liveness classification sub-model are connected. The first liveness classification sub-model trained after training is extracted from the trained model to obtain the liveness detection model.

2. The method as described in claim 1, wherein the second sample set includes a depth sample set and a reflective sample set; Before training the second liveness classification sub-model using the second sample set, the method further includes: If the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is a living body, then the depth estimation model is used to perform depth estimation processing on the color image sample to obtain the depth image sample corresponding to the color image sample. The depth sample set is generated based on the depth image samples corresponding to the color image samples; If the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is not a living body, then the illumination model is used to perform illumination feature analysis on the color image sample to obtain the reflective image sample corresponding to the color image sample. The reflective sample set is generated based on the reflective image samples corresponding to the color image samples.

3. The method of claim 2, further comprising, before training the second liveness classification sub-model using the second sample set: If the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is a non-living object, then the first solid color image is determined as the depth image sample corresponding to the color image sample; If the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is a living body, then the second solid color image is determined as the reflective image sample corresponding to the color image sample.

4. The method of claim 3, wherein the second live classification sub-model comprises: Deep classification models and reflectivity classification models; The multi-task learning approach trains the first liveness classification sub-model using a first sample set and the second liveness classification sub-model using a second sample set to obtain the trained model, specifically including: Based on a multi-task learning approach, the first liveness classification sub-model is trained using the first sample set, the depth classification model is trained using the depth sample set, and the reflective classification model is trained using the reflective sample set, thereby obtaining the trained model.

5. The method of claim 4, wherein the first sample set comprises: A global color image sample set and a local color image sample set; the global color image sample set includes global color image samples carrying facial information and background information, and the local color image sample set includes local color image samples carrying the facial information; The depth sample set includes: a global depth sample set and a local depth sample set; the global depth sample set contains depth image samples corresponding to the global color image samples, and the local depth sample set contains depth image samples corresponding to the local color image samples. The reflective sample set includes a global reflective sample set and a local reflective sample set; the global reflective sample set contains reflective image samples corresponding to the global color image samples, and the local reflective sample set contains reflective image samples corresponding to the local color image samples.

6. The method of claim 5, further comprising, before training the first liveness classification sub-model using the first sample set: Acquire color images captured by the device; Perform face recognition processing on the color image to obtain the predicted face region in the color image; The image within the predicted face region is extracted from the color image to obtain the local color image sample; The global color image sample is obtained by extracting an image from a designated region, including the predicted face region, from the color image; the area of ​​the designated region is larger than the area of ​​the predicted face region.

7. The method as described in claim 5, wherein the first liveness classification sub-model includes a first classification model and a second classification model, the depth classification model includes a third classification model and a fourth classification model, and the reflectivity classification model includes a fifth classification model and a sixth classification model; The multi-task learning approach trains the first liveness classification sub-model using the first sample set, the deep classification model using the deep sample set, and the reflective classification model using the reflective sample set to obtain the trained model. Specifically, this includes: Based on a multi-task learning approach, the first classification model is trained using the global color image sample set, the third classification model is trained using the global depth sample set, and the fifth classification model is trained using the global reflectivity sample set, resulting in a first trained model; and... Based on a multi-task learning approach, the second classification model is trained using the local color image sample set, the fourth classification model is trained using the local depth sample set, and the sixth classification model is trained using the local reflective sample set, to obtain the second trained model. The step of extracting the trained first liveness classification sub-model from the trained model to obtain the liveness detection model specifically includes: The first classification model after training is extracted from the first trained model, and the second classification model after training is extracted from the second trained model to obtain the liveness detection model.

8. The method of claim 7, wherein the first classification model, the third classification model and the fifth classification model do not share a network layer, and the first hidden layer in the first classification model, the second hidden layer in the third classification model and the third hidden layer in the fifth classification model have a first connection relationship, the first connection relationship being used to perform feature fusion processing on some feature vectors at the first hidden layer, the second hidden layer and the third hidden layer; The second classification model, the fourth classification model, and the sixth classification model do not share a network layer, and the fourth hidden layer in the second classification model, the fifth hidden layer in the fourth classification model, and the sixth hidden layer in the sixth classification model have a second connection relationship. The second connection relationship is used to perform feature fusion processing on some feature vectors at the fourth hidden layer, the fifth hidden layer, and the sixth hidden layer.

9. The method as described in claim 8, wherein the network layers at the first classification model, the third classification model, and the fifth classification model are not connected to the network layers at the second classification model, the fourth classification model, and the sixth classification model.

10. The method according to any one of claims 7 to 9, wherein after extracting the trained first classification model from the first trained model and extracting the trained second classification model from the second trained model to obtain the liveness detection model, the method further comprises: Acquire the color image to be detected; Based on the color image to be detected, a local color image to be detected and a global color image to be detected are generated; the local color image to be detected contains the face information to be detected, and the global color image to be detected contains the face information to be detected and the background information to be detected. The trained first classification model is used to generate a first liveness detection result based on the global color image to be detected; The trained second classification model is used to generate a second liveness detection result based on the local color image to be detected. Based on the first liveness detection result and the second liveness detection result, a liveness detection result is generated for the color image to be detected.

11. The method of claim 10, wherein the first liveness detection result includes: First live probability value and first non-live probability value; The second liveness detection result includes: a second liveness probability value and a second non-liveness probability value; The step of generating a liveness detection result for the color image to be detected based on the first liveness detection result and the second liveness detection result specifically includes: Based on the first liveness probability value and the second liveness probability value, a comprehensive liveness probability value is generated for the color image to be detected; If the overall liveness probability value is greater than the first threshold, a liveness detection result is generated to indicate that the object corresponding to the detected face information is a live object; or, Based on the first non-inactive probability value and the second non-inactive probability value, a comprehensive non-inactive probability value is generated for the color image to be detected; If the overall non-living probability value is greater than the second threshold, a liveness detection result is generated to indicate that the object corresponding to the face information to be detected is a non-living object.

12. A device for generating a liveness detection model, comprising: An initial model acquisition module is used to acquire an initial model; the initial model includes a first liveness classification sub-model and a second liveness classification sub-model. The initial model training module is used to train the first liveness classification sub-model using a first sample set and the second liveness classification sub-model using a second sample set to obtain the trained model, based on a multi-task learning approach. The first sample set contains color image samples, and the second sample set contains depth image samples or reflective image samples determined based on the color image samples. The first liveness classification sub-model and the second liveness classification sub-model do not share network layers, and the hidden layers of the first liveness classification sub-model and the hidden layers of the second liveness classification sub-model are connected. The detection model acquisition module is used to extract the trained first liveness classification sub-model from the trained model to obtain a liveness detection model.

13. The apparatus of claim 12, further comprising: The first color image processing module is used to perform depth estimation processing on the color image sample using a depth estimation model if the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is a living body, so as to obtain the depth image sample corresponding to the color image sample. A depth sample set generation module is used to generate the depth sample set based on the depth image samples corresponding to the color image samples; The second color image processing module is used to perform illumination feature analysis on the color image sample using an illumination model if the classification label data carried by the color image sample is used to indicate that the sample object in the color image sample is not a living body, so as to obtain the reflective image sample corresponding to the color image sample. The reflective sample set generation module is used to generate the reflective sample set based on the reflective image samples corresponding to the color image samples.

14. The apparatus of claim 13, further comprising: The first solid color image determination module is used to determine the first solid color image as the depth image sample corresponding to the color image sample if the classification label data carried by the color image sample indicates that the sample object in the color image sample is a non-living object. The second solid color image determination module is used to determine the second solid color image as the reflective image sample corresponding to the color image sample if the classification label data carried by the color image sample indicates that the sample object in the color image sample is a living body.

15. The apparatus of claim 14, wherein the second living organism classification sub-model comprises: Deep classification models and reflectivity classification models; The initial model training module is specifically used for: Based on a multi-task learning approach, the first liveness classification sub-model is trained using the first sample set, the depth classification model is trained using the depth sample set, and the reflective classification model is trained using the reflective sample set, thereby obtaining the trained model.

16. The apparatus of claim 15, wherein the first sample set comprises: A global color image sample set and a local color image sample set; the global color image sample set includes global color image samples carrying facial information and background information, and the local color image sample set includes local color image samples carrying the facial information; The depth sample set includes: a global depth sample set and a local depth sample set; the global depth sample set contains depth image samples corresponding to the global color image samples, and the local depth sample set contains depth image samples corresponding to the local color image samples. The reflective sample set includes a global reflective sample set and a local reflective sample set; the global reflective sample set contains reflective image samples corresponding to the global color image samples, and the local reflective sample set contains reflective image samples corresponding to the local color image samples.

17. The apparatus of claim 16, further comprising: The color image acquisition module is used to acquire color images captured by the device; The predicted face region acquisition module is used to perform face recognition processing on the color image to obtain the predicted face region in the color image; A local color image sample acquisition module is used to extract the image within the predicted face region from the color image to obtain the local color image sample; A global color image sample acquisition module is used to extract an image from a specified region, including the predicted face region, from the color image to obtain the global color image sample; the area of ​​the specified region is larger than the area of ​​the predicted face region.

18. The apparatus of claim 16, wherein the first living organism classification sub-model includes a first classification model and a second classification model, the depth classification model includes a third classification model and a fourth classification model, and the reflectivity classification model includes a fifth classification model and a sixth classification model; The initial model training module is specifically used for: Based on a multi-task learning approach, the first classification model is trained using the global color image sample set, the third classification model is trained using the global depth sample set, and the fifth classification model is trained using the global reflectivity sample set, resulting in a first trained model; and... Based on a multi-task learning approach, the second classification model is trained using the local color image sample set, the fourth classification model is trained using the local depth sample set, and the sixth classification model is trained using the local reflective sample set, to obtain the second trained model. The detection model acquisition module is specifically used for: The first classification model after training is extracted from the first trained model, and the second classification model after training is extracted from the second trained model to obtain the liveness detection model.

19. The apparatus of claim 18, wherein the first classification model, the third classification model, and the fifth classification model do not share a network layer, and the first hidden layer in the first classification model, the second hidden layer in the third classification model, and the third hidden layer in the fifth classification model have a first connection relationship, the first connection relationship being used for feature fusion processing of partial feature vectors at the first hidden layer, the second hidden layer, and the third hidden layer; The second classification model, the fourth classification model, and the sixth classification model do not share a network layer, and the fourth hidden layer in the second classification model, the fifth hidden layer in the fourth classification model, and the sixth hidden layer in the sixth classification model have a second connection relationship. The second connection relationship is used to perform feature fusion processing on some feature vectors at the fourth hidden layer, the fifth hidden layer, and the sixth hidden layer.

20. The apparatus of claim 19, wherein the network layers at the first classification model, the third classification model, and the fifth classification model are not connected to the network layers at the second classification model, the fourth classification model, and the sixth classification model.

21. The apparatus of any one of claims 18 to 20, further comprising: The color image acquisition module is used to acquire the color image to be detected. A global color image to be detected generation module is used to generate a local color image to be detected and a global color image to be detected based on the color image to be detected; the local color image to be detected contains the face information to be detected, and the global color image to be detected contains the face information to be detected and the background information to be detected. The first liveness detection result generation module is used to generate a first liveness detection result based on the global color image to be detected using the trained first classification model. The second liveness detection result generation module is used to generate a second liveness detection result based on the local color image to be detected using the trained second classification model. The third liveness detection result generation module is used to generate a liveness detection result for the color image to be detected based on the first liveness detection result and the second liveness detection result.

22. The apparatus of claim 21, wherein the first liveness detection result includes: The first liveness probability value and the first non-liveness probability value; the second liveness detection result includes: the second liveness probability value and the second non-liveness probability value; The third liveness detection result generation module is specifically used for: Based on the first liveness probability value and the second liveness probability value, a comprehensive liveness probability value is generated for the color image to be detected; If the overall liveness probability value is greater than the first threshold, a liveness detection result is generated to indicate that the object corresponding to the detected face information is a live object; or, Based on the first non-live probability value and the second non-live probability value, a comprehensive non-live probability value is generated for the color image to be detected; if the comprehensive non-live probability value is greater than a second threshold, a liveness detection result is generated to indicate that the object corresponding to the face information to be detected is a non-live object.

23. A device for generating a liveness detection model, comprising: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions executable by the at least one processor, which, when executed by the at least one processor, enable the at least one processor to: Obtain an initial model; the initial model includes a first liveness classification sub-model and a second liveness classification sub-model; Based on a multi-task learning approach, the first liveness classification sub-model is trained using a first sample set, and the second liveness classification sub-model is trained using a second sample set to obtain a trained model. The first sample set contains color image samples, and the second sample set contains depth image samples or reflective image samples determined based on the color image samples. The first liveness classification sub-model and the second liveness classification sub-model do not share network layers, but the hidden layers of the first liveness classification sub-model and the hidden layers of the second liveness classification sub-model are connected. The first liveness classification sub-model trained after training is extracted from the trained model to obtain the liveness detection model.

Images