Face living body detection model training method, face living body detection method and device

Abstract

The present disclosure provides a face living body detection model training method, a face living body detection method and device, and relates to the technical field of artificial intelligence, in particular to the technical field of deep learning, image processing and computer vision. The specific implementation scheme is: determining a first feature output by a first model according to a sample face image; determining a first loss value by using a first loss function according to the first feature; determining a second feature output by a second model according to the first feature and a first text description corresponding to the sample face image; determining a second loss value by using a second loss function according to the second feature; and training the first model according to the first loss value and the second loss value to obtain a face living body detection model. In the embodiment of the present disclosure, the model is supervised and trained by image modalities and text modalities, the supervision of fine-grained semantic information provided by the text modalities is utilized, and the generalization of the face living body detection model is improved.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to the technical field of artificial intelligence, in particular to the technical field of deep learning, image processing and computer vision, and can be applied to the scene of face recognition. BACKGROUND

[0002] Face liveness detection refers to distinguishing whether an image is obtained by shooting a real person, and is the basis of a face recognition system, which can ensure the security of the face recognition system. Using a deep convolutional neural network to train a face liveness detection model is a common method at present. In order to make the model learn more rich and generalizable features, a multi-modal learning method appears. The existing multi-modal face liveness detection method implementation scheme is usually based on multi-modal input data, that is, multiple modal information is input to the model, and each modal feature is added or connected to perform liveness discrimination. SUMMARY

[0003] The present disclosure provides a training method of a face liveness detection model, a face liveness detection method and device.

[0004] According to an aspect of the present disclosure, a training method of a face liveness detection model is provided, comprising: determining a first feature output by a first model according to a sample face image; wherein the sample face image comprises: a live sample image and an attack sample image;

[0005] determining a first loss value by using a first loss function according to the first feature;

[0006] determining a second feature output by a second model according to the first feature and a first text description corresponding to the sample face image;

[0007] determining a second loss value by using a second loss function according to the second feature; and

[0008] training the first model according to the first loss value and the second loss value to obtain the face liveness detection model.

[0009] According to another aspect of the present disclosure, a face liveness detection method is provided, comprising:

[0010] using the face liveness detection model trained by the above-mentioned training method of a face liveness detection model to detect a target face image to obtain a detection result.

[0011] According to another aspect of the present disclosure, a training device of a face liveness detection model is provided, comprising:

[0012] a first determining module configured to determine a first feature output by a first model according to a sample face image; wherein the sample face image comprises: a live sample image and an attack sample image;

[0013] The second determining module is configured to determine a first loss value according to the first feature and using a first loss function.

[0014] The third determining module is configured to determine a second feature output by the second model according to the first feature and a first text description corresponding to the sample face image.

[0015] The fourth determining module is configured to determine a second loss value according to the second feature and using a second loss function.

[0016] The training module is configured to train the first model according to the first loss value and the second loss value to obtain the face living body detection model.

[0017] According to another aspect of the present disclosure, a face living body detection device is provided, comprising:

[0018] The detection module is configured to detect a target face image using the face living body detection model trained by the training device to obtain a detection result.

[0019] According to another aspect of the present disclosure, an electronic device is provided, comprising:

[0020] at least one processor; and

[0021] a memory communicatively connected to the at least one processor; wherein

[0022] The memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any of the embodiments of the present disclosure.

[0023] According to another aspect of the present disclosure, a non-transitory computer readable storage medium storing computer instructions is provided, wherein the computer instructions are used to enable the computer to perform the method according to any of the embodiments of the present disclosure.

[0024] According to another aspect of the present disclosure, a computer program product is provided, comprising a computer program which, when executed by a processor, implements the method according to any of the embodiments of the present disclosure.

[0025] According to the scheme of the embodiments of the present disclosure, the generalization of the face living body detection model is improved by image modal and text modal supervised model training. The text modal is only used in the form of increasing supervision in the model training process, which can realize the auxiliary face living body detection model to learn more discriminative features.

[0026] It should be appreciated that the content described in this section is not intended to identify key or critical features of the embodiments of the present disclosure, nor is it used to limit the scope of the present disclosure. Other features of the present disclosure will become apparent through the following description. BRIEF DESCRIPTION OF DRAWINGS

[0027] The accompanying drawings are used to better understand the present scheme and do not limit the present disclosure. Among them:

[0028] Figure 1 is a flowchart of a face living body detection model training method according to an embodiment of the present disclosure;

[0029] Figure 2 is a flowchart of a sample face image generation method according to an embodiment of the present disclosure;

[0030] Figure 3 is a structural diagram of a face living body detection model according to an embodiment of the present disclosure;

[0031] Figure 4 is a flowchart of a face living body detection method according to an embodiment of the present disclosure;

[0032] Figure 5 is a structural diagram of a face living body detection model training device according to an embodiment of the present disclosure;

[0033] Figure 6 is a structural diagram of a face living body detection device according to an embodiment of the present disclosure;

[0034] Figure 7 is a block diagram of an electronic device for implementing a face living body detection model training method and / or a face living body detection method according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0035] Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to assist in understanding, and should be considered as merely exemplary. Therefore, those of ordinary skill in the art should recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope of the present disclosure. Also, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0036] In related technologies, face living body detection refers to distinguishing whether an image is obtained by shooting a real person. Face living body detection is the basis of a face recognition system, which can ensure the security of the face recognition system. Using a deep convolutional neural network to train a face living body detection method is a common method at present. Since existing living body detection methods only use classification labels as a supervision signal, it is difficult to make the model learn more rich and generalizable features, resulting in limited performance of the model.

[0037] At present, in order to make the model learn more rich and generalizable features, a multi-modal learning method appears. The existing multi-modal face liveness detection method usually implements a scheme based on multi-modal input data, for example, using depth images and infrared images, adding or connecting features of each modality to perform liveness discrimination. The method based on feature connection also uses multiple independent convolutional neural networks to extract features of each modality, then performs a feature connection operation on the multiple features, and finally classifies the connected features. Both methods are multi-modal in the form of input data. These multi-modal data need to consume a large amount of economic cost for collection, which greatly limits the scale of multi-modal data and makes it difficult to be widely applied. In addition, the existing multi-modal only fuses features of each modality, which is easy to cause the fusion model to be biased to one of the modalities, that is, the weight difference between the features of each modality is large, and it is difficult to play the advantage of multi-modal data.

[0038] To at least partially solve one or more of the above problems and other potential problems, embodiments of the present disclosure provide a face liveness detection model training method. By using the technical solutions of the embodiments of the present disclosure, it is possible to promote the face liveness detection model to extract more discriminative features, thereby improving the generalization ability of the model and the performance of face liveness detection.

[0039] Figure 1 is a flowchart of a face liveness detection model training method according to an embodiment of the present disclosure, as shown in Figure 1 The method at least includes the following steps:

[0040] S101: determining first features output by a first model according to a sample face image. The sample face image includes a live sample image and an attack sample image.

[0041] The sample face image of the embodiments of the present disclosure is used for model training, and a plurality of sample face images constitute a sample set. It can be understood that the sample set contains positive samples and negative samples, that is, contains live sample images and attack sample images. The live sample image can be understood as an image containing a real face picture. The attack sample image can be understood as a fake face image, also known as a false body sample image. Although the image contains a face picture, the face picture is not a real face directly collected, but a false face image generated by printing a face picture, playing a face picture, video on a display screen, or making a mask and other cheating means, that is, the image contains clues that can reflect the attack means. It should be noted that the type and cheating method of the attack sample image can be various and changeable according to technical means, which is not limited here.

[0042] The first model can be an image feature extraction network model, and the first feature can be an image modality feature.

[0043] S102: According to the first feature, a first loss value is determined by using a first loss function.

[0044] S103: According to the first feature and a first text description corresponding to the sample face image, a second feature output by a second model is determined.

[0045] The first text description can be understood as generating a natural language text describing the content shown in the sample face image. The first text description can be generated by an existing text description generation model, or can be artificially generated.

[0046] S104: According to the second feature, a second loss value is determined by using a second loss function.

[0047] S105: According to the first loss value and the second loss value, the first model is trained to obtain a face living body detection model.

[0048] It should be noted that the face living body detection model does not contain the second model. The second model is only used in the model training stage and does not need to be used in the model inference stage.

[0049] According to the scheme of the embodiments of the present disclosure, the model is supervised by image modalities and text modalities, and the generalization of the face living body detection model is improved by fully utilizing the supervision of fine-grained semantic information provided by the text modalities. The text modalities are only used in the form of increased supervision in the model training process, which can help the face living body detection model to learn more discriminative features. In the actual deployment and use process, no text input and output are required, so no additional time-consuming and data collection costs are required.

[0050] In a possible implementation, step S101: according to the sample face image, a first feature output by a first model is determined, which can further include the steps of:

[0051] The sample face image is cut into a plurality of image blocks to obtain a sample image block sequence.

[0052] According to the sample image block sequence, a sample image vector sequence is determined by using a linear projection (Linear Projection) layer of the first model.

[0053] According to the sample image vector sequence and the first classification vector, a first feature is determined by using a transformer (Transformer) layer of the first model.

[0054] It should be noted that the network model of the Transformer structure is originally proposed for the machine translation task in the field of natural language processing, and therefore the input of the standard Transformer is one-dimensional sequence data, so when processing image data, the image needs to be converted into sequence data. In an example, a sample face image can be divided into 9 image patches of fixed size to obtain a sample image patch sequence. Then each patch is converted into a one-dimensional vector through a stretching operation, and finally the input patch is converted into a fixed-length vector through a linear projection layer, called patch_embedding, so as to obtain a sample image vector sequence. In addition, for the classification task, the final output is usually a classification label, so the input of the standard Transformer Encoder needs to be adjusted, and a classification vector CLS Token is added at the beginning of the sample image vector sequence. The CLS Token and the 9 sample image vectors are input into the Transformer structure, and 10 encoding vectors are output. Finally, the output corresponding to the CLS Token is taken as the first feature, that is, the image modal feature.

[0055] According to the scheme of the embodiments of the present disclosure, by processing the sample image, the network model of the Transformer structure is used to extract the features of the sample face image, and on the basis of obtaining the image modal feature, the computing resources can be reduced and the model training efficiency can be improved.

[0056] In a possible implementation, the step S102 of determining a first loss value according to the first feature by using a first loss function can further include the following steps:

[0057] According to the first feature, a multilayer perceptron (MLP) layer of the first model is used to determine a predicted classification label.

[0058] It should be noted that the MLP can be a fully connected network containing one hidden layer, which is used for binary classification according to the first feature to obtain the predicted classification label.

[0059] According to the predicted classification label and the actual classification label of the sample face image, a first loss value is determined by using a first loss function.

[0060] The sample face image has a pre-determined actual classification label. According to the predicted classification label and the actual classification label of the sample face image, a first loss value is determined by inputting the first loss function to calculate the distance between the probability distributions represented by the two classification labels.

[0061] According to the scheme of the embodiment of the present disclosure, the first feature is classified by the multi-layer perception to obtain a predicted classification label, and then the first loss value is determined by comparing the predicted classification label with the actual classification label of the sample face image, so as to supervise the model training by using the first loss value.

[0062] In a possible implementation, the step S103 of determining the second feature output by the second model according to the first feature and the first text description corresponding to the sample face image can further include the steps of:

[0063] The third feature is determined by using a causal self-attention (CausalSelf-Attention) layer of the second model according to the first text description corresponding to the sample face image.

[0064] It can be understood that the third feature can be a text modal feature obtained according to the first text description.

[0065] The fusion feature is determined by using a cross attention (Cross Attention) layer of the second model to fuse the first feature and the third feature.

[0066] It can be understood that the image modal feature and the text modal feature are fused by the cross attention mechanism.

[0067] The second feature output by the second model is determined by using a feed forward (Feed Forward) layer of the second model according to the fusion feature.

[0068] It should be noted that the second model can be a text generation network, and the second feature corresponding to the third feature is output by inputting the first text description into the network and combining the first feature generated by the first model according to the third feature obtained from the first text description. During the training process, the second feature generated by the network is supervised by using the second loss function, so that the generated second feature is more close to the third feature. It can be understood that the second model aims to generate the same text description as the first text description.

[0069] According to the scheme of the embodiment of the present disclosure, the text modal feature and the image modal feature are fused by the cross attention mechanism, the second model outputs the second feature according to the fusion feature, and the generalization of the liveness detection model is improved by fully utilizing the supervision of the fine-grained semantic information provided by the text modal.

[0070] In a possible implementation, the step S104 of determining the second loss value by using the second loss function according to the second feature can further include the step of determining the second loss value by using the second loss function according to the third feature and the second feature.

[0071] In the embodiments of the present disclosure, the third feature is taken as a true value, the second feature is taken as a predicted value, and the second loss value is determined by using the second loss function.

[0072] According to the scheme of the embodiments of the present disclosure, the second loss value obtained by the second loss function supervises the generation of the second feature, and the second loss value can also be used to supervise the training of the first model.

[0073] In a possible implementation, the step S105 of training the first model according to the first loss value and the second loss value to obtain the face living body detection model can further include the steps of:

[0074] The third loss value is obtained according to the first loss value and the second loss value.

[0075] The first model is trained according to the third loss value to obtain the face living body detection model.

[0076] In the embodiments of the present disclosure, the first loss value and the second loss value are added to obtain the third loss value. The sample face image in each sample set and the corresponding first text description correspond to the input of the first model and the second model to obtain the first loss value and the second loss value. Adding the two values obtains the third loss value. According to the third loss value, the parameters of the first model are adjusted to realize one round of model training. Using the multiple sample face images contained in the sample set can realize multiple rounds of iterative training of the first model. Thus, the face living body detection model is obtained.

[0077] According to the scheme of the embodiments of the present disclosure, the text modality is only integrated into the model training in the form of an increased supervision mode during the model training process, which is conducive to assisting the image feature extraction network to learn more discriminative features. At the same time, only the trained first model is used as the face living body detection model during actual deployment, without the need for text input and output, so that additional time-consuming and data collection costs are not required.

[0078] In a possible implementation, the first text description is generated by a pre-trained text description generation model according to a training face image, and the sample face image is generated according to the training face image.

[0079] In the embodiments of the present disclosure, the pre-trained text description generation model can be an existing super-large-scale image-text multimodal pre-training model, which can generate a corresponding description for an input image. For example, if the input image is a face attacked by a mobile phone screen, the text description generation model will have a high probability of outputting: "a face image is displayed on the mobile phone screen", and the text description can contain more information about attack clues.

[0080] In the case that the first text description generated by the text description generation model according to the training face image does not conform to the actual image content, the unqualified training face image can be removed, and the remaining training face image is used as the sample face image.

[0081] According to the scheme of the embodiments of the present disclosure, the first text description generated by the pre-trained text description generation model according to the training face image can reduce manual work and unify the standard of the first text description.

[0082] In a possible implementation, the generation process of the first text description includes:

[0083] The training face image is input into the text description generation model to obtain the first text description corresponding to the training face image.

[0084] In the case that the first text description meets the generation rule, the training face image corresponding to the first text description is used as the sample face image.

[0085] Specifically, the text description obtained by inputting any training face image into the text description generation model needs to be judged by a keyword. If a normal live sample image is input, it is judged whether the generated text description contains attack words. If it contains, the text description generation is performed again. If the attack words are contained in the repeated three times, the training face image is discarded. Otherwise, the qualified data is entered. If an attack sample image is input, it is also judged whether the generated text description contains attack words. If the attack words are contained, the qualified data is directly entered. Otherwise, the text description generation is repeatedly performed. If the attack words are not contained after the repeated three times, the training face image is discarded.

[0086] According to the scheme of the embodiments of the present disclosure, the training face image is screened according to whether the first text description meets the generation rule, and the qualified training face image is used as the sample face image, so as to obtain the sample set.

[0087] In a possible implementation, the generation rule includes:

[0088] In the case that the training face image is a live sample image, the first text description generated by the text description generation model does not contain the preset attack words.

[0089] In the case that the training face image is an attack sample image, the first text description generated by the text description generation model contains the preset attack words.

[0090] In the embodiments of the present disclosure, each training face image is pre-determined to be a live sample image or an attack sample image. The attack vocabulary can be understood as a clue that reflects the cheating and attack behavior appearing in the image content shown in the face image. The vocabulary describing the clue is called attack vocabulary. It can also be understood as a vocabulary describing the face shown in the face image as a false face. The attack vocabulary is a pre-set vocabulary set. Due to the uncertainty of cheating and attack methods, the attack vocabulary set is an open set, which can be increased according to newly added attack methods. For example, the vocabulary set of the attack vocabulary includes but is not limited to: mask, paper, screen, moire, head model, photo, deformity, etc.

[0091] In an example, as shown in Figure 2 The sample face image generation process includes:

[0092] Step 1: input the training image data (training face image) into the text description generation model to obtain the text description. The training image data includes two categories: live sample images and attack sample images.

[0093] Step 2: determine whether the text description contains attack vocabulary. In the case that the text description generated for the live sample image does not contain attack vocabulary, or the text description generated for the attack sample image contains attack vocabulary, go to step 3.

[0094] In the case that the text description generated for the live sample image contains attack vocabulary, or the text description generated for the attack sample image does not contain attack vocabulary, go to step 4.

[0095] Step 3: obtain qualified data, add the training face image as a sample face image to the sample set.

[0096] Step 4: determine the size relationship between the number of cycles of the text description generation of the training face image and the pre-set number threshold (for example, the number is 3). In the case that the number of cycles is less than 3, input the training face image into the text description model again to generate a third text description, set the number of cycles of the training face image to 1, and return the third text description to step 2. In the case that the number of cycles is greater than 3, discard the training face image and do not use it as a sample face image.

[0097] According to the scheme of the embodiments of the present disclosure, the training face image is screened according to whether the attack vocabulary is accurately contained in the text description, and a more accurate sample set is obtained.

[0098] In a possible implementation, as Figure 3As shown, the first model is a neural network model based on a transformer structure, and the second model is a Bidirectional Encoder Representation from Transformers (Bert) model.

[0099] The first model includes a linear projection layer, a transformer layer, and a multi-layer perception layer. The second model includes a causal self-attention layer, a cross-attention layer, and a forward feedback layer.

[0100] The first loss function can be a Cross Entropy (CE) loss function. The second loss function can be a Language Modeling (LM) loss function.

[0101] According to the scheme of the embodiments of the present disclosure, the model with a Transformer structure can more friendly combine the image modal and the text modal.

[0102] Figure 4 is a flowchart of a face liveness detection method according to an embodiment of the present disclosure, as shown, the method at least includes the following steps: Figure 4

[0103] S401: using the face liveness detection model trained by the training method of any embodiment to detect the target face image to obtain a detection result.

[0104] According to the scheme of the embodiments of the present disclosure, by detecting the target face image by the face liveness detection model trained based on multi-modal supervision, the accuracy of the detection result is improved.

[0105] In one possible implementation, step S401: using the face liveness detection model trained by the training method of any embodiment to detect the target face image to obtain a detection result, can further include the steps of:

[0106] cutting the target face image into multiple image blocks to obtain a target image block sequence.

[0107] According to the target image block sequence, using the face liveness detection model trained by the training method of any embodiment to determine an image feature.

[0108] According to the image feature, determine the classification label of the target face image to obtain a detection result.

[0109] According to the scheme of the embodiments of the present disclosure, by cutting the target face image into blocks, the face liveness detection model with a Transformer structure is facilitated to be inputted, so as to obtain a detection result. ​

[0110] Figure 5 is a structural schematic diagram of a training device of a face liveness detection model according to an embodiment of the present disclosure, as shown, the training device 500 at least includes: Figure 5

[0111] A first determination module 501 is configured to determine a first feature output by a first model according to a sample face image. The sample face image includes a live sample image and an attack sample image.

[0112] A second determination module 502 is configured to determine a first loss value by using a first loss function according to the first feature.

[0113] A third determination module 503 is configured to determine a second feature output by a second model according to the first feature and a first text description corresponding to the sample face image.

[0114] A fourth determination module 504 is configured to determine a second loss value by using a second loss function according to the second feature.

[0115] A training module 505 is configured to train the first model according to the first loss value and the second loss value to obtain a face liveness detection model.

[0116] In a possible implementation, the first determination module 501 includes:

[0117] A first segmentation sub-module is configured to segment the sample face image into a plurality of image blocks to obtain a sample image block sequence.

[0118] The first determination module 501 is further configured to determine a sample image vector sequence by using a linear projection layer of the first model according to the sample image block sequence.

[0119] The first feature is determined by using a converter layer of the first model according to the sample image vector sequence and the first classification vector.

[0120] In a possible implementation, the second determination module 502 is configured to:

[0121] Determine a predicted classification label by using a multi-layer perception layer of the first model according to the first feature.

[0122] The first loss value is determined by using the first loss function according to the predicted classification label of the sample face image and an actual classification label.

[0123] In a possible implementation, the third determination module 503 is configured to:

[0124] Determine a third feature by using a causal self-attention layer of the second model according to the first text description corresponding to the sample face image. ​

[0125] According to the first feature and the third feature, the feature fusion is performed by using the cross-attention layer of the second model to determine the fused feature.

[0126] According to the fused feature, the second feature output by the second model is determined by using the forward feedback layer of the second model.

[0127] In a possible implementation, the training module 505 is configured to:

[0128] According to the first loss value and the second loss value, a third loss value is obtained.

[0129] According to the third loss value, the first model is trained to obtain the face living body detection model.

[0130] In a possible implementation, the first text description is generated according to the training face image by using a pre-trained text description generation model, and the sample face image is generated according to the training face image.

[0131] In a possible implementation, the generation process of the first text description includes:

[0132] The training face image is input into the text description generation model to obtain the first text description corresponding to the training face image.

[0133] In a case where the first text description meets the generation rule, the training face image corresponding to the first text description is taken as the sample face image.

[0134] In a possible implementation, the generation rule includes:

[0135] In a case where the training face image is a living body sample image, the first text description generated by the text description generation model does not contain the preset attack vocabulary.

[0136] In a case where the training face image is an attack sample image, the first text description generated by the text description generation model contains the preset attack vocabulary.

[0137] In a possible implementation, the first model is a neural network model based on a transformer structure, and the second model is a bidirectional encoder representation model based on a transformer. Alternatively, the first loss function is a cross-entropy loss function, and the second loss function is a language modeling loss function.

[0138] The specific functions and examples of the modules and sub-modules of the face living body detection model training apparatus 500 are described above in the corresponding steps of the face living body detection model training method embodiments, which will not be described herein again.

[0139] Figure 6is a structural schematic diagram of a face living body detection device according to an embodiment of the disclosure, as shown in the figure, the device 600 at least includes: Figure 6

[0140] The detection module 601 is configured to perform detection on the target face image by using the face living body detection model trained by the training device 500 in any of the above embodiments, to obtain a detection result.

[0141] In a possible implementation, the detection module 601 includes:

[0142] The second segmentation sub-module is configured to segment the target face image into a plurality of image blocks, to obtain a target image block sequence.

[0143] The detection module 601 is configured to determine the image feature according to the target image block sequence, by using the face living body detection model obtained by the training device 500 in any of the above embodiments.

[0144] According to the image feature, a classification label of the target face image is determined, to obtain the detection result.

[0145] The specific functions and examples of each module and sub-module of the face living body detection device 600 in the embodiments of the disclosure are described in the above face living body detection method embodiments, and will not be described here.

[0146] In the technical solutions of the disclosure, the acquisition, storage and application of user personal information comply with relevant laws and regulations and do not violate public order and good customs.

[0147] According to the embodiments of the disclosure, the disclosure further provides an electronic device, a readable storage medium and a computer program product.

[0148] Figure 7 A schematic block diagram of an example electronic device 700 that can be used to implement embodiments of the disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptops, desktops, tablets, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular telephones, smartphones, wearable devices, and other similar computing devices. The components shown here, their connections and relationships, and their functions, are meant to be examples only, and are not intended to limit the implementations of the disclosure described and / or claimed in this document.

[0149] As shown in the figure, Figure 7 ​As shown, the device 700 includes a computing unit 701 that can perform various appropriate actions and processes in accordance with a computer program stored in a read-only memory (ROM) 702 or a computer program loaded into a random access memory (RAM) 703 from a storage unit 708. In the RAM 703, various programs and data required for the operation of the device 700 can also be stored. The computing unit 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0150] A plurality of components in the device 700 are connected to the I / O interface 705, including: an input unit 706, such as a keyboard, a mouse, and the like; an output unit 707, such as various types of displays, speakers, and the like; a storage unit 708, such as a magnetic disk, an optical disk, and the like; and a communication unit 709, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 709 allows the device 700 to exchange information / data with other devices through a computer network, such as the Internet, and / or various telecommunication networks.

[0151] The computing unit 701 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 701 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various specialized artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, and the like. The computing unit 701 performs various methods and processes described above, such as the training method of a face liveness detection model or the face liveness detection method. For example, in some embodiments, the training method of a face liveness detection model or the face liveness detection method can be implemented as a computer software program tangibly embodied in a machine-readable medium, such as the storage unit 708. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 700 via the ROM 702 and / or the communication unit 709. When the computer program is loaded into the RAM 703 and executed by the computing unit 701, one or more steps of the training method of a face liveness detection model or the face liveness detection method described above can be performed. Alternatively, in other embodiments, the computing unit 701 can be configured to perform the training method of a face liveness detection model or the face liveness detection method by any other appropriate means, such as by means of firmware.

[0152] Various implementations of the systems and techniques described above can be realized in digital electronic circuitry, integrated circuitry, a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on a chip (SOC), a programmable logic device (PLD), a computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementation in one or more computer programs that are executable and / or interpretable on a programmable system including at least one programmable processor, which can be special or general purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a storage system, at least one input device, and at least one output device.

[0153] Program code for carrying out methods of the present disclosure can be written in any combination of one or more programming languages. The program code can be provided to a processor or controller of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the program code, when executed by the processor or controller, produces the functions / operations specified in the flowcharts and / or the block diagrams. The program code can be executed entirely on a machine, partially on a machine, partially on a machine as a stand-alone software package, partially on a machine and partially on a remote machine or entirely on a remote machine or server.

[0154] In the context of the present disclosure, a machine-readable medium can be a tangible medium that contains or stores a program for use by or in connection with an instruction execution system, apparatus, or device. The machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include but is not limited to an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination of the foregoing. More specific examples of the machine-readable storage medium will include one or more lines of electrical connections, portable computer disks, hard disk drives, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or Flash memory), optical fibers, portable compact disc read-only memories (CD-ROMs), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0155] To provide for interaction with a user, the systems and techniques described here can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the computer. Other kinds of devices can be used to provide for interaction with a user as well; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form, including acoustic, speech, or tactile input.

[0156] The systems and techniques described here can be implemented in a computing system that includes a back end component (e.g., as a data server), or that includes a middleware component (e.g., an application server), or that includes a front end component (e.g., a user computer having a graphical user interface or a Web browser through which a user can interact with an implementation of the systems and techniques described here), or any combination of such back end, middleware, or front end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0157] The computer system can include clients and servers. A client and server are generally remote from each other and typically interact through a communication network. The relationship of client and server arises by virtue of computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0158] It should be understood that various forms of flow shown above can be used, with steps reordered, added, or removed. For example, the steps recited in the present disclosure can be performed in parallel, in series, or in a different order, without limitation herein, so long as the desired results of the technology disclosed in the present disclosure are achieved.

[0159] The specific embodiments described above are not intended to be limiting, and persons skilled in the art will appreciate that various modifications, combinations, sub-combinations and alternatives can be made to the specific embodiments without departing from the principles of the disclosure. Accordingly, modifications, equivalent alternatives and improvements should be included within the scope of the disclosure.

Claims

1. A method for training a face liveness detection model, comprising: determining a first feature output by a first model according to a sample face image, wherein the sample face image comprises a live sample image and an attack sample image; determining a first loss value using a first loss function according to the first feature; determining a third feature using a causal self-attention layer of a second model according to a first text description corresponding to the sample face image, wherein the first text description corresponding to the live sample image does not contain a preset attack vocabulary, and the first text description corresponding to the attack sample image contains the preset attack vocabulary; determining a fused feature using a cross-attention layer of the second model according to the first feature and the third feature; determining a second feature output by the second model using a forward feedback layer of the second model according to the fused feature; determining a second loss value using a second loss function according to the second feature; obtaining a third loss value according to the first loss value and the second loss value; and training the first model according to the third loss value to obtain a face liveness detection model.

2. The method of claim 1, wherein, The determining of the first feature output by the first model according to the sample face image comprises: segmenting the sample face image into a plurality of image blocks to obtain a sample image block sequence; determining a sample image vector sequence using a linear projection layer of the first model according to the sample image block sequence; determining the first feature using a transformer layer of the first model according to the sample image vector sequence and a first classification vector.

3. The method of claim 1, wherein, The determining of the first loss value using the first loss function according to the first feature comprises: determining a predicted classification label using a multi-layer perceptron layer of the first model according to the first feature; determining the first loss value using the first loss function according to the predicted classification label and an actual classification label of the sample face image.

4. The method of claim 1, wherein, The first text description is generated by a pre-trained text description generation model according to a training face image, and the sample face image is generated according to the training face image.

5. The method of claim 4, wherein the generation process of the first text description comprises: inputting the training face image into the text description generation model to obtain a first text description corresponding to the training face image; in a case where the first text description satisfies a generation rule, taking the training face image corresponding to the first text description as the sample face image.

6. The method of claim 5, wherein, The generation rule comprises: in a case where the training face image is a live sample image, the first text description generated by the text description generation model does not contain a preset attack vocabulary; in a case where the training face image is an attack sample image, the first text description generated by the text description generation model contains the preset attack vocabulary.

7. The method according to any one of claims 1 to 6, wherein, The first model is a neural network model based on a transformer structure, and the second model is a transformer-based bidirectional encoder representation model; or the first loss function is a cross-entropy loss function, and the second loss function is a language modeling loss function.

8. A face liveness detection method, comprising: detecting a target face image by using a face liveness detection model trained by any one of the methods of claims 1 to 7 to obtain a detection result.

9. The method of claim 8, wherein, detecting a target face image by using a face liveness detection model trained by any one of the methods of claims 1 to 7 to obtain a detection result, comprising: segmenting the target face image into a plurality of image blocks to obtain a target image block sequence; determining an image feature by using a face liveness detection model trained by any one of the methods of claims 1 to 7 according to the target image block sequence; determining a classification label of the target face image according to the image feature to obtain a detection result.

10. A device for training a face liveness detection model, comprising: a first determining module configured to determine a first feature output by a first model according to a sample face image, wherein the sample face image comprises a live sample image and an attack sample image; a second determining module configured to determine a first loss value by using a first loss function according to the first feature; a third determining module configured to determine a third feature by using a causal self-attention layer of a second model according to a first text description corresponding to the sample face image, wherein the first text description corresponding to the live sample image does not contain a preset attack vocabulary, and the first text description corresponding to the attack sample image contains the preset attack vocabulary; determining a fusion feature by using a cross-attention layer of the second model according to the first feature and the third feature; determining a second feature output by the second model by using a forward feedback layer of the second model according to the fusion feature; a fourth determining module configured to determine a second loss value by using a second loss function according to the second feature; and a training module configured to obtain a third loss value according to the first loss value and the second loss value; training the first model according to the third loss value to obtain a face liveness detection model.

11. The apparatus of claim 10, wherein, The first determining module comprises: a segmentation sub-module configured to segment the sample face image into a plurality of image blocks to obtain a sample image block sequence; the first determining module is configured to determine a sample image vector sequence by using a linear projection layer of the first model according to the sample image block sequence; determine the first feature by using a converter layer of the first model according to the sample image vector sequence and a first classification vector.

12. The apparatus of claim 10, wherein, The second determining module is configured to: determine a predicted classification label by using a multi-layer perceptron layer of the first model according to the first feature; determine the first loss value by using the first loss function according to the predicted classification label and an actual classification label of the sample face image.

13. The apparatus of claim 10, wherein, The first text description is generated by a pre-trained text description generation model according to a training face image, and the sample face image is generated according to the training face image.

14. The device of claim 13, wherein the generation process of the first text description comprises: inputting the training face image into the text description generation model to obtain a first text description corresponding to the training face image; in a case where the first text description meets a generation rule, taking the training face image corresponding to the first text description as the sample face image.

15. The apparatus of claim 14, wherein, The generation rule comprises: in a case where the training face image is a live sample image, the first text description generated by the text description generation model does not contain a preset attack vocabulary; in a case where the training face image is an attack sample image, the first text description generated by the text description generation model contains a preset attack vocabulary.

16. The apparatus of any one of claims 10 to 15, wherein, The first model is a neural network model based on a transformer structure, and the second model is a bidirectional encoder representation model based on a transformer; or, the first loss function is a cross-entropy loss function, and the second loss function is a language modeling loss function.

17. A face live body detection apparatus, comprising: a detection module configured to detect a target face image by using a face live body detection model trained by the apparatus of any one of claims 10 to 16 to obtain a detection result.

18. The apparatus of claim 17, wherein, The detection module comprises: a second segmentation module configured to segment the target face image into a plurality of image blocks to obtain a target image block sequence; The detection module is configured to determine an image feature by using the face live body detection model trained by the apparatus of any one of claims 10 to 16 according to the target image block sequence; and determine a classification label of the target face image according to the image feature to obtain the detection result.

19. An electronic device, comprising: at least one processor; and a memory connected to the at least one processor in communication; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 9. The computer instructions are used to enable the computer to perform the method of any one of claims 1 to 9.

20. A non-transitory computer readable storage medium having stored thereon computer instructions, wherein, 21. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 9.

21. A computer program product comprising a computer program which, when executed by a processor, implements the method of any one of claims 1 to 9.

Images