Face living body detection method and model training method, device, equipment and medium

Abstract

The present disclosure provides a face living body detection method and device, a model training method and device, a medium, and relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, image processing, deep learning, etc., and can be applied to scenes such as living body detection. The specific implementation scheme is as follows: obtaining a to-be-recognized face image; inputting the to-be-recognized face image into a pre-trained face living body detection model to obtain a living body detection result; wherein the face living body detection model comprises a plurality of convolutional neural network modules and a plurality of visual self-attention modules, and the convolutional neural network modules and the visual self-attention modules are alternately connected.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence technology, particularly to computer vision, image processing, and deep learning, and can be applied to scenarios such as liveness detection. Specifically, this disclosure relates to a face liveness detection method, a model training method, an apparatus, a device, and a medium. Background Technology

[0002] Face liveness detection distinguishes whether an image is taken of a real person. It is a basic component of a face recognition system and can be applied to many applications in the field of face recognition, such as attendance, access control, security, and financial payment.

[0003] Ensuring the accuracy of face liveness detection helps guarantee the security of face recognition systems. Summary of the Invention

[0004] This disclosure provides a face liveness detection method, a model training method, a device, an equipment, and a medium.

[0005] According to a first aspect of this disclosure, a method for face liveness detection is provided, the method comprising:

[0006] Acquire the image of the face to be identified;

[0007] The face image to be identified is input into a pre-trained face liveness detection model to obtain liveness detection results;

[0008] The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected.

[0009] According to a second aspect of this disclosure, a method for training a face liveness detection model is provided, the method comprising:

[0010] Acquire multiple training face images and the corresponding liveness detection labels for the training face images;

[0011] The face liveness detection model is trained based on the training face images and the corresponding liveness detection labels.

[0012] The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected.

[0013] According to a third aspect of this disclosure, an apparatus for face liveness detection is provided, the apparatus comprising:

[0014] The test data module is used to acquire images of the faces to be identified.

[0015] The inference module is used to input the face image to be identified into a pre-trained face liveness detection model to obtain liveness detection results;

[0016] The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected.

[0017] According to a fourth aspect of this disclosure, a training apparatus for a face liveness detection model is provided, the apparatus comprising:

[0018] The training data module is used to acquire multiple training face images and the corresponding liveness detection labels for the training face images;

[0019] The training module is used to train the face liveness detection model based on the training face image and the corresponding liveness detection label of the training face image;

[0020] The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected.

[0021] According to a fifth aspect of this disclosure, an electronic device is provided, the electronic device comprising:

[0022] At least one processor; and

[0023] A memory communicatively connected to at least one of the aforementioned processors; wherein,

[0024] The memory stores instructions that can be executed by at least one processor, and the instructions are executed by at least one processor to enable at least one processor to perform the face liveness detection method and / or the face liveness detection model training method.

[0025] According to a sixth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are used to cause a computer to execute the above-described face liveness detection method and / or face liveness detection model training method.

[0026] According to the seventh aspect of this disclosure, a computer program product is provided, including a computer program that, when executed by a processor, implements the above-described face liveness detection method and / or the training method for a face liveness detection model.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this disclosure, nor is it intended to limit the scope of this disclosure. Other features of this disclosure will become readily apparent from the following description. Attached Figure Description

[0028] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein:

[0029] Figure 1 This is a schematic flowchart of a face liveness detection method provided in an embodiment of this disclosure;

[0030] Figure 2 This is a schematic diagram of the structure of an implementation of the face liveness detection model provided in this disclosure, in which the convolutional neural network module and the visual self-attention module are alternately connected;

[0031] Figure 3 This is a flowchart illustrating some steps of another face liveness detection method provided in this embodiment of the present disclosure;

[0032] Figure 4 This is a schematic diagram of the network training process of another face liveness detection method provided in this disclosure, which uses the method of freezing the parameters of the module for model training.

[0033] Figure 5 This is a schematic flowchart of a training method for a face liveness detection network provided in an embodiment of this disclosure;

[0034] Figure 6 This is a flowchart illustrating some steps of a training method for a face liveness detection network provided in an embodiment of this disclosure.

[0035] Figure 7 This disclosure provides a schematic diagram of the structure of a face liveness detection device according to an embodiment;

[0036] Figure 8 This is a schematic diagram of the structure of a training device for a face liveness detection network provided in an embodiment of this disclosure;

[0037] Figure 9 This is a block diagram of an electronic device used to implement the face liveness detection method and the face liveness detection network training method of the embodiments of this disclosure. Detailed Implementation

[0038] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0039] In some related technologies, traditional manual facial feature extraction and classification methods are used to achieve face liveness detection. Specifically, facial features are extracted based on manual feature extractors, and then feature classification is performed based on traditional SVM (Support Vector Machine) classifiers to obtain face liveness detection results.

[0040] However, traditional manual facial feature extraction and classification methods are not robust enough in real-world scenarios where facial poses are too large or lighting differences are significant, resulting in unsatisfactory recognition results.

[0041] In some related technologies, deep learning models, such as convolutional neural network models and LSTM (Long Short-Term Memory) networks, are used to extract and classify facial features and obtain facial liveness detection results. Compared with traditional methods, deep learning methods can extract more stable facial features and have a significant performance improvement.

[0042] However, deep learning models based on single images as input suffer from high sensitivity to light and poor generalization to planar attacks such as photos and videos, as well as 3D (three-dimensional) attacks such as headgear and head models, which affects the performance of practical applications.

[0043] The face liveness detection method, face liveness detection model training method, face liveness detection device, face liveness detection model training device, electronic device, and computer-readable storage medium provided in this disclosure are intended to solve at least one of the above-mentioned technical problems of the prior art.

[0044] The face liveness detection method and face liveness detection model training method provided in this disclosure can be executed by electronic devices such as terminal devices or servers. Terminal devices can be in-vehicle devices, user equipment (UE), mobile devices, user terminals, terminals, cellular phones, cordless phones, personal digital assistants (PDAs), handheld devices, computing devices, in-vehicle devices, wearable devices, etc. The method can be implemented by a processor calling computer-readable program instructions stored in memory. Alternatively, the method can be executed by a server.

[0045] Figure 1 A schematic flowchart of the face liveness detection method provided in this disclosure embodiment is shown. Figure 1 As shown, the face liveness detection method of this disclosure embodiment may include steps S110 and S120.

[0046] In step S110, the image of the face to be identified is acquired;

[0047] In step S120, the face image to be identified is input into a pre-trained face liveness detection model to obtain the liveness detection result;

[0048] The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected.

[0049] For example, in step S110, the face image to be identified can be an image containing only a face.

[0050] In some possible implementations, the face image to be identified can be an image acquired directly by taking a picture.

[0051] In some possible implementations, the face image to be identified can be an image obtained by preprocessing a person's image.

[0052] In some possible implementations, preprocessing of the human image may include face detection, key point detection, face alignment, affine transformation, and normalization. Preprocessing can reduce the influence of factors such as lighting on the face image to be recognized, thereby improving the accuracy of the obtained liveness detection results.

[0053] In some possible implementations, the face image to be identified can be an RGB (red, green, blue) image; in some possible implementations, the face image to be identified can be a depth image; in some possible implementations, the face image to be identified can be an infrared image.

[0054] In some possible implementations, in step S120, the liveness detection result is whether the face image to be identified is a live face image.

[0055] In some possible implementations, the pre-trained face liveness detection model can be obtained by training multiple training face images and corresponding liveness detection labels.

[0056] In some possible implementations, the training face images can include real face images, planar attack images such as screen attacks and paper attacks, and stereoscopic attack images such as headgear and head molds. The liveness detection label corresponding to the real face image can be a label indicating that the image contains a live face, while the liveness detection label corresponding to the planar attack image and the stereoscopic attack image can be a label indicating that the image does not contain a live face.

[0057] In some possible implementations, the training face images can all be RGB images, depth images, or infrared images.

[0058] When the training face image is an RGB image, the accuracy of the liveness detection result obtained by using the face liveness detection method provided in this disclosure is higher when the face image to be identified is an RGB image.

[0059] When the training face image is a depth image, the accuracy of the liveness detection result obtained by using the face liveness detection method provided in this disclosure is higher when the face image to be identified is a depth image.

[0060] When the training face image is an infrared image, the accuracy of the liveness detection result obtained by using the face liveness detection method provided in this disclosure is higher when the face image to be identified is an infrared image.

[0061] In some possible implementations, the face liveness detection model includes multiple alternating convolutional neural network modules and multiple ViT (VisionTransformer, visual self-attention) modules.

[0062] The convolutional neural network (CNN) module consists of one or more convolutional layers. It can be used to extract image features. A CNN module with fewer convolutional layers can extract low-level and local features, while a CNN module with more convolutional layers can extract high-level semantic and global features. The input to the CNN module can be an image or a feature map of an image, and the output is an image feature map.

[0063] The ViT module consists of multiple connected ViT structures, each of which includes a Transformer Encoder (self-attention encoder). The input to the ViT module can be an image or a small block of image feature map segmentation, and the output can be a small block of image feature map segmentation.

[0064] In some possible implementations, the ViT module may also include Linear Projection of Flattened Patches.

[0065] Among them, LinearProjectionofFlattenedPatches is an image feature layer whose main function is to realize the segmentation of the image and the generation of token (vector) sequences. The generated sequences are then input into TransformerEncoder, which can segment the feature maps output by the convolutional neural network module to generate token sequences.

[0066] For example, for a 224x224 image, LinearProjectionofFlattenedPatches can divide it into 16x16 patches, resulting in 196 patches. Each patch is flattened into a vector of length 16x16x3 = 768, ultimately resulting in a 196x768 two-dimensional matrix, which is the sequence format that TransformerEncoder needs to input.

[0067] In addition, there is a ClassEmbedding (or ClassToken, a classification vector), which is a learnable vector. Its dimension is 1x768. Concatenating this ClassToken with the matrix generated above yields a matrix with a dimension of 197x768.

[0068] In addition, there is a learnable vector PositionEmbedding with a dimension of 197x768. This position encoding is combined with the matrix with a dimension of 197x768 generated above to obtain the final solution input matrix of TransformerEncoder, which has a dimension of 197x768.

[0069] The TransformerEncoder comprises multiple EncodeBlocks, each consisting of a Multi-head Attention layer and Add & Norm layers (Add stands for residual structure, Norm stands for Layer Normalization). It extracts high-level semantic features and global features from the image and determines the parameters of the ClassToken by learning from these features. The high-level semantic features and global features acquired by the TransformerEncoder are stored in a matrix, which includes image features corresponding to multiple image patches. By concatenating the image features corresponding to each image patch, a feature map that can be input into a convolutional neural network module can be generated.

[0070] In some possible implementations, a ViT module comprising multiple connected ViT structures could be a ViT module consisting of a LinearProjectionofFlattenedPatches connected in sequence and multiple TransformerEncoders. For example, if a ViT module comprises three ViT structures, then the ViT module would consist of a LinearProjectionofFlattenedPatches connected in sequence and three TransformerEncoders.

[0071] In some possible implementations, the convolutional neural network modules and visual self-attention modules can be connected alternately, such that the input of visual self-attention module 21 includes the output of convolutional neural network module 11, the input of convolutional neural network 12 includes the output of visual self-attention module 21, the input of visual self-attention module 22 includes the output of convolutional neural network module 12, and so on, until all convolutional neural network modules and visual self-attention modules are connected.

[0072] Figure 2 This diagram illustrates a structural approach where convolutional neural network modules and visual self-attention modules are alternately connected. Figure 2 As shown, the face liveness detection model may include a first convolutional neural network module CM1, a first visual self-attention module VM1, a second convolutional neural network module CM2, a second visual self-attention module VM2, a third convolutional neural network module CM3, and a third visual self-attention module VM3 connected in sequence.

[0073] The convolutional neural network module and the visual self-attention module are distinct feature extraction modules, capable of extracting low-level features, local features, high-level semantic features, and global features from the image of the face to be identified. Low-level and local features can be used to identify planar attacks such as screen attacks and paper attacks, while high-level semantic features and global features can be used to identify stereoscopic attacks such as those involving masks or mannequins. Therefore, the face liveness detection model composed of the convolutional neural network module and the visual self-attention module can acquire multiple features required for face liveness detection, thereby improving the model's generalization and robustness against various attacks and enhancing its detection accuracy.

[0074] Meanwhile, by alternately connecting the convolutional neural network module and the visual self-attention module, the features extracted by the convolutional neural network module and the visual self-attention module can be fully integrated, further improving the richness of the acquired features, enhancing the generalization and robustness of the face liveness detection model against various attacks, and improving the detection accuracy of the face liveness detection model.

[0075] In some possible implementations, such as Figure 2 As shown, the first convolutional neural network module CM1 is also connected to the second convolutional neural network module CM2, and the second convolutional neural network module CM2 is also connected to the third convolutional neural network module CM3.

[0076] In other words, the input of the second convolutional neural network module CM2 also includes the output of the first convolutional neural network module CM1; the input of the third convolutional neural network module CM3 also includes the output of the second convolutional neural network module CM2.

[0077] In some possible implementations, the first convolutional neural network module CM1 can be composed of multiple convolutional layers, and the second convolutional neural network module CM2 and the third convolutional neural network module CM3 can be composed of multiple ResNetBlocks.

[0078] In some possible implementations, such as Figure 2 As shown, the face liveness detection model also includes a classification module composed of fully connected layers, which is used to obtain liveness detection results based on features extracted by alternating convolutional neural network modules and visual self-attention modules.

[0079] In some possible implementations, the face liveness detection model can be trained by freezing the parameters of the modules, so that the convolutional neural network module and the visual self-attention module can better learn the different features of the image.

[0080] In the face liveness detection method provided in this disclosure, a face liveness detection model composed of alternatingly connected convolutional neural network modules and visual self-attention modules detects the face image to be identified and obtains liveness detection results. Since the convolutional neural network module and the visual self-attention module are different feature extraction modules, they can extract different image features, obtaining multiple features required for face liveness detection, thus improving the generalization and robustness of the face liveness detection model against various attacks. Simultaneously, the alternating connection of the convolutional neural network module and the visual self-attention module allows for the full fusion of the features extracted by both modules, further enhancing the richness of the acquired features and improving the generalization and robustness of the face liveness detection model against various attacks. Therefore, the liveness detection results obtained using the face liveness detection method provided in this disclosure are more accurate.

[0081] The face liveness detection method provided in the embodiments of this disclosure will be described in detail below.

[0082] As mentioned above, in some possible implementations, the face image to be identified can be an image obtained by preprocessing a person image.

[0083] Figure 3 A flowchart illustrating the steps involved in obtaining a face image to be identified through preprocessing is shown. Figure 3 As shown, obtaining the face image to be identified through preprocessing may include steps S310, S320, S330, and S340.

[0084] In step S310, an image of the person to be identified is obtained, face detection is performed on the image of the person to be identified, and the face region of the image of the person to be identified is obtained.

[0085] In step S320, facial landmark detection is performed on the face region to obtain the coordinates of the facial landmarks;

[0086] In step S330, the face region is aligned according to the coordinates of the face key points, and a face image of a preset size is obtained through affine transformation.

[0087] In step S340, the face image is normalized to obtain the face image to be identified.

[0088] In some possible implementations, in step S310, the image of the person to be identified can be an image obtained by taking a picture of the person, and its size is not limited.

[0089] In some possible implementations, face detection can be performed on the image of the person to be identified using traditional machine learning methods to obtain the face position in the image; alternatively, a pre-trained face detection model can be used to obtain the face position in the image of the person to be identified and determine the face region based on the face position.

[0090] The face detection model can be any face liveness detection model, and the embodiments disclosed herein do not limit the specific composition of the face detection model.

[0091] In some possible implementations, in step S320, facial landmark detection of the acquired face region can be performed using a pre-trained facial landmark detection model.

[0092] The facial landmark detection model can be any deep learning model that can perform facial landmark detection. This disclosure does not limit the specific composition of the facial landmark detection model.

[0093] In some possible implementations, there are a total of 72 facial landmarks.

[0094] In some possible implementations, the preset size can be 224x224. In step S330, the target face of the person to be identified is aligned according to the key point coordinates of the face. At the same time, the face image containing only the face region is cropped by affine transformation, and the face image is adjusted to the preset size.

[0095] In some possible implementations, normalizing a face image can be done by subtracting 128 from the pixel value of each pixel in the face image and then dividing by 256, so that the pixel value of each pixel is between [-0.5, 0.5].

[0096] Preprocessing can remove non-face regions from the image of the person to be identified, reducing the impact of non-face regions on face liveness detection. At the same time, normalization can reduce the influence of factors such as lighting on the face image to be identified, improving the accuracy of the obtained liveness detection results.

[0097] As mentioned above, in some possible implementations, such as Figure 2 As shown, the first convolutional neural network module CM1 is also connected to the second convolutional neural network module CM2, and the second convolutional neural network module CM2 is also connected to the third convolutional neural network module CM3.

[0098] In other words, the input of the second convolutional neural network module CM2 also includes the output of the first convolutional neural network module CM1; the input of the third convolutional neural network module CM3 also includes the output of the second convolutional neural network module CM2.

[0099] By connecting convolutional neural network modules, the low-level feature information extracted by the low-level convolutional neural network module (such as the first convolutional neural network module CM1) can be propagated to the high-level convolutional neural network module (such as the third convolutional neural network module CM3), thereby improving the richness of features obtained by the high-level convolutional neural network module.

[0100] In some possible implementations, the first convolutional neural network module CM1 can be composed of multiple convolutional layers, and the second convolutional neural network module CM2 and the third convolutional neural network module CM3 can be composed of multiple ResNetBlocks.

[0101] Compared to ordinary convolutional layers, ResNetBlock is easier to optimize and can alleviate the gradient vanishing problem caused by increasing depth in deep neural networks. Therefore, ResNetBlock is more suitable as a high-level convolutional neural network than ordinary convolutional layers.

[0102] Experiments showed that the best performance of the face liveness detection model was achieved by using two convolutional layers to form the first convolutional neural network module CM1, four ResNetBlocks to form the second convolutional neural network module CM2, and three convolutional neural network modules CM3.

[0103] In some possible implementations, such as Figure 2 As shown, the face liveness detection model also includes a classification module composed of fully connected layers, which is used to obtain liveness detection results based on features extracted by alternating convolutional neural network modules and visual self-attention modules.

[0104] In some specific implementations, the process of inputting the image of the face to be identified into a pre-trained face liveness detection model to obtain liveness detection results may include:

[0105] The face image to be identified is input into the first convolutional neural network module CM1, which consists of two convolutional layers. Downsampling is performed in the second convolutional layer of the first convolutional neural network module CM1 to extract the low-level feature information of the face image to be identified.

[0106] Then, the feature map extracted by the first convolutional neural network module CM1 is input into the first visual self-attention module VM1, which consists of 3 ViT structures. The features output by the first convolutional neural network module CM1 and the first visual self-attention module VM1 are added together and input into the second convolutional neural network module CM2, which consists of 4 ResNet blocks. Similarly, downsampling is performed on the last convolutional layer of the second convolutional neural network module CM2.

[0107] Then, the feature maps extracted by the second convolutional neural network module CM2 are input into the second visual self-attention module VM2, which consists of three ViT structures. The features obtained by the second convolutional neural network module CM2 and the second visual self-attention module VM2 are added together and input into the third convolutional neural network module CM3, which consists of four ResNetBlocks. Similarly, downsampling is performed in the last convolutional layer.

[0108] Finally, the feature map extracted by the third convolutional neural network module CM3 is input into the third visual self-attention module VM3, which consists of three ViT structures. Based on the features output by the third visual self-attention module VM3 (specifically, ClassToken), the final liveness detection result is obtained using the fully connected layer FC1.

[0109] As mentioned above, in some possible implementations, the face liveness detection model can be trained by freezing the parameters of the modules, so that the convolutional neural network module and the visual self-attention module can better learn the different features of the image.

[0110] Figure 4 This is a schematic diagram illustrating a network training process that demonstrates one implementation of model training by freezing the parameters of modules.

[0111] like Figure 4 As shown in the diagram, during training, the parameters of all visual self-attention modules are first frozen (i.e., Figure 4 The module marked with a diagonal line in the first network structure diagram uses training face images and corresponding liveness detection labels to train the pre-trained model corresponding to the face liveness detection model, thus obtaining the first training model.

[0112] The pre-trained model corresponding to the human liveness detection model has the same main structure as the human liveness detection model, which also includes multiple convolutional neural network modules and a visual self-attention module that are alternately connected. The difference is that the last convolutional neural network module is connected to a fully connected layer FC2 to obtain the liveness detection result. The network loss is determined based on the liveness detection result and the liveness detection label corresponding to the training face image. The parameters of the convolutional neural network module of the pre-trained model corresponding to the human liveness detection model are modified based on the network loss.

[0113] Then freeze the convolutional neural network module of the first trained model (i.e. Figure 4 The parameters of the module (marked with a diagonal line in the second network structure diagram) are used to train the first training model using training face images and the corresponding liveness detection labels to obtain the second training model.

[0114] The first training model has the same structure as the human liveness detection model. The liveness detection result is obtained based on the output of the fully connected layer FC1 of the first training model. The network loss is determined based on the liveness detection result and the liveness detection label corresponding to the training face image. The parameters of the visual self-attention module of the first training model are modified based on the network loss.

[0115] Finally, the training face images and their corresponding liveness detection labels are used to train the second training model, resulting in a trained face liveness detection model.

[0116] The second training model has the same structure as the human liveness detection model. The liveness detection result is obtained based on the output of the fully connected layer FC1 of the second training model. The network loss is determined based on the liveness detection result and the liveness detection label corresponding to the training face image. The parameters of the convolutional neural network module and the visual self-attention module of the second training model are modified based on the network loss.

[0117] By freezing the parameters of the convolutional neural network module and the visual self-attention module separately, the convolutional neural network module and the visual self-attention module can learn the features of the face image to be recognized independently, avoiding mutual interference in the image feature learning process of the convolutional neural network module and the visual self-attention module, and helping the convolutional neural network module and the visual self-attention module learn better features.

[0118] In some possible implementations, the training data for the pre-trained model and the first trained model corresponding to the face liveness detection model are different.

[0119] The first training model is obtained by training the pre-trained model corresponding to the face liveness detection model using the planar attack image and the corresponding liveness detection label of the planar attack image.

[0120] The first training model is trained using stereo attack images and corresponding liveness detection labels to obtain the second training model.

[0121] Since identifying planar attacks such as screen attacks and paper attacks requires extracting low-level and local features of the image, by freezing the parameters of all visual self-attention modules, training the pre-trained model corresponding to the face liveness detection model using planar attack images and the corresponding liveness detection labels can enable the convolutional neural network module to learn to extract low-level and local features.

[0122] Identifying stereo attacks such as hood attacks and head-mounted attacks requires extracting high-level semantic features and global features from images. Therefore, by freezing the parameters of all convolutional neural network modules, training the first training model with stereo attack images and corresponding liveness detection labels can enable the visual self-attention module to learn to extract high-level semantic features and global features.

[0123] Finally, using all the data to train the second training model allows for the full fusion of features extracted by the convolutional neural network module and features extracted by the visual self-attention module.

[0124] This training method enables the face liveness detection model to simultaneously acquire low-level features and high-level semantic features, as well as local features and global features. By fully fusing the acquired features, the richness of the extracted features is improved, thereby enhancing the generalization and accuracy of the face liveness detection model against various attacks.

[0125] Figure 5 This illustration shows a flowchart of a training method for a face liveness detection model provided in an embodiment of this disclosure, as shown below. Figure 5 As shown in the figure, the training method of the face liveness detection model provided in this embodiment may include steps S510 and S520.

[0126] In step S510, multiple training face images and corresponding liveness detection labels for the training face images are obtained;

[0127] In step S520, the face liveness detection model is trained based on the training face image and the corresponding liveness detection label.

[0128] The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected.

[0129] For example, in step S510, the training face images may include real face images, planar attack images such as screen attacks and paper attacks, and stereoscopic attack images such as headgear and head molds. The liveness detection label corresponding to the real face image can be a label indicating that the image contains a live face, while the liveness detection labels corresponding to the planar attack image and the stereoscopic attack image can be labels indicating that the image does not contain a live face.

[0130] In some possible implementations, the training face images can be images acquired directly through photography.

[0131] In some possible implementations, training face images can also be obtained by preprocessing images.

[0132] In some possible implementations, image preprocessing may include face detection, key point detection, face alignment, affine transformation, and normalization. Preprocessing can reduce the influence of lighting and other factors on training face images, thereby improving the performance of the obtained face liveness detection model by improving the quality of the training data.

[0133] In some possible implementations, the training face image can be an RGB image; in some possible implementations, the training face image can be a depth image; in some possible implementations, the training face image can be an infrared image.

[0134] When the training face image is an RGB image, the accuracy of the liveness detection result obtained by using the face liveness detection method provided in this disclosure is higher when the face image to be identified is an RGB image.

[0135] When the training face image is a depth image, the accuracy of the liveness detection result obtained by using the face liveness detection method provided in this disclosure is higher when the face image to be identified is a depth image.

[0136] When the training face image is an infrared image, the accuracy of the liveness detection result obtained by using the face liveness detection method provided in this disclosure is higher when the face image to be identified is an infrared image.

[0137] In some possible implementations, in step S520, the face liveness detection model includes multiple alternately connected convolutional neural network modules and multiple ViT (VisionTransformer, visual self-attention) modules.

[0138] The convolutional neural network (CNN) module consists of one or more convolutional layers. It can be used to extract image features. A CNN module with fewer convolutional layers can extract low-level and local features, while a CNN module with more convolutional layers can extract high-level semantic and global features. The input to the CNN module can be an image or a feature map of an image, and the output is an image feature map.

[0139] The ViT module consists of multiple connected ViT structures, each of which includes a Transformer Encoder (self-attention encoder). The input to the ViT module can be an image or a small block of image feature map segmentation, and the output can be a small block of image feature map segmentation.

[0140] In some possible implementations, the ViT module may also include Linear Projection of Flattened Patches.

[0141] Among them, LinearProjectionofFlattenedPatches is an image feature layer whose main function is to realize the segmentation of the image and the generation of token (vector) sequences. The generated sequences are then input into TransformerEncoder, which can segment the feature maps output by the convolutional neural network module to generate token sequences.

[0142] For example, for a 224x224 image, LinearProjectionofFlattenedPatches can divide it into 16x16 patches, resulting in 196 patches. Each patch is flattened into a vector of length 16x16x3 = 768, ultimately resulting in a 196x768 two-dimensional matrix, which is the sequence format that TransformerEncoder needs to input.

[0143] In addition, there is a ClassEmbedding (or ClassToken, a classification vector), which is a learnable vector. Its dimension is 1x768. Concatenating this ClassToken with the matrix generated above yields a matrix with a dimension of 197x768.

[0144] In addition, there is a learnable vector PositionEmbedding with a dimension of 197x768. This position encoding is combined with the matrix with a dimension of 197x768 generated above to obtain the final solution input matrix of TransformerEncoder, which has a dimension of 197x768.

[0145] The TransformerEncoder comprises multiple EncodeBlocks, each consisting of a Multi-head Attention layer and Add & Norm layers (Add stands for residual structure, Norm stands for Layer Normalization). It extracts high-level semantic features and global features from the image and determines the parameters of the ClassToken by learning from these features. The high-level semantic features and global features acquired by the TransformerEncoder are stored in a matrix, which includes image features corresponding to multiple image patches. By concatenating the image features corresponding to each image patch, a feature map that can be input into a convolutional neural network module can be generated.

[0146] In some possible implementations, a ViT module comprising multiple connected ViT structures could be a ViT module consisting of a LinearProjectionofFlattenedPatches connected in sequence and multiple TransformerEncoders. For example, if a ViT module comprises three ViT structures, then the ViT module would consist of a LinearProjectionofFlattenedPatches connected in sequence and three TransformerEncoders.

[0147] In some possible implementations, the convolutional neural network modules and visual self-attention modules can be connected alternately, such that the input of visual self-attention module 21 includes the output of convolutional neural network module 11, the input of convolutional neural network 12 includes the output of visual self-attention module 21, the input of visual self-attention module 22 includes the output of convolutional neural network module 12, and so on, until all convolutional neural network modules and visual self-attention modules are connected.

[0148] Figure 2 This diagram illustrates a structural approach where convolutional neural network modules and visual self-attention modules are alternately connected. Figure 2 As shown, the face liveness detection model may include a first convolutional neural network module CM1, a first visual self-attention module VM1, a second convolutional neural network module CM2, a second visual self-attention module VM2, a third convolutional neural network module CM3, and a third visual self-attention module VM3 connected in sequence.

[0149] The convolutional neural network module and the visual self-attention module are distinct feature extraction modules, used to extract low-level features, local features, high-level semantic features, and global features from training face images, respectively. Low-level and local features can be used to identify planar attacks such as screen attacks and paper attacks, while high-level semantic features and global features can be used to identify stereoscopic attack images such as those using headgear or head molds. Therefore, the face liveness detection model composed of the convolutional neural network module and the visual self-attention module can acquire multiple features required for face liveness detection, thereby improving the model's generalization and robustness against various attacks and enhancing its detection accuracy.

[0150] Meanwhile, by alternately connecting the convolutional neural network module and the visual self-attention module, the features extracted by the convolutional neural network module and the visual self-attention module can be fully integrated, further improving the richness of the acquired features, enhancing the generalization and robustness of the face liveness detection model against various attacks, and improving the detection accuracy of the face liveness detection model.

[0151] In some possible implementations, such as Figure 2 As shown, the first convolutional neural network module CM1 is also connected to the second convolutional neural network module CM2, and the second convolutional neural network module CM2 is also connected to the third convolutional neural network module CM3.

[0152] In other words, the input of the second convolutional neural network module CM2 also includes the output of the first convolutional neural network module CM1; the input of the third convolutional neural network module CM3 also includes the output of the second convolutional neural network module CM2.

[0153] In some possible implementations, the first convolutional neural network module CM1 can be composed of multiple convolutional layers, and the second convolutional neural network module CM2 and the third convolutional neural network module CM3 can be composed of multiple ResNetBlocks.

[0154] In some possible implementations, such as Figure 2As shown, the face liveness detection model also includes a classification module composed of fully connected layers, which is used to obtain liveness detection results based on features extracted by alternating convolutional neural network modules and visual self-attention modules.

[0155] In some possible implementations, the face liveness detection model can be trained by freezing the parameters of the modules, so that the convolutional neural network module and the visual self-attention module can better learn the different features of the image.

[0156] In the face liveness detection model training method provided in this disclosure, the face liveness detection model consisting of alternatingly connected convolutional neural network modules and visual self-attention modules can extract different image features since the convolutional neural network module and the visual self-attention module are different feature extraction modules. This allows for the acquisition of multiple features required for face liveness detection, improving the generalization and robustness of the face liveness detection model against various attacks. Furthermore, the alternating connection of the convolutional neural network module and the visual self-attention module allows for the full fusion of the features extracted by both modules, further enhancing the richness of the acquired features and improving the generalization and robustness of the face liveness detection model against various attacks. Therefore, the face liveness detection model obtained using the training method provided in this disclosure has better performance.

[0157] The face liveness detection method provided in the embodiments of this disclosure will be described in detail below.

[0158] As mentioned above, in some possible implementations, the training face image can be an image obtained by preprocessing a person image.

[0159] Preprocessing to obtain training face images can include:

[0160] Acquire training images, perform face detection on the training images, and obtain the face regions of the training images;

[0161] Facial landmark detection is performed on the face region to obtain the coordinates of the facial landmarks;

[0162] The face region is aligned based on the coordinates of the facial key points, and a face image of a preset size is obtained through affine transformation.

[0163] Normalize the face images to obtain training face images.

[0164] In some possible implementations, the acquired training images can be images taken of people, and their size is not limited.

[0165] In some possible implementations, face detection can be performed on the training images using traditional machine learning methods to obtain the face locations in the training images; alternatively, a pre-trained face detection model can be used to obtain the face locations in the training images and determine the face region based on the face locations.

[0166] The face detection model can be any face liveness detection model, and the embodiments disclosed herein do not limit the specific composition of the face detection model.

[0167] In some possible implementations, facial landmark detection of the acquired face region can be performed using a pre-trained facial landmark detection model.

[0168] The facial landmark detection model can be any deep learning model that can perform facial landmark detection. This disclosure does not limit the specific composition of the facial landmark detection model.

[0169] In some possible implementations, there are a total of 72 facial landmarks.

[0170] In some possible implementations, the preset size can be 224x224. The target face in the training image is aligned with the key point coordinates of the face. At the same time, the face image containing only the face region is cropped through affine transformation and adjusted to the preset size.

[0171] In some possible implementations, normalizing a face image can be done by subtracting 128 from the pixel value of each pixel in the face image and then dividing by 256, so that the pixel value of each pixel is between [-0.5, 0.5].

[0172] Preprocessing can remove non-face regions from the training images, reducing their impact on liveness detection. Meanwhile, normalization can reduce the influence of lighting conditions on the training images, improving the accuracy of the obtained liveness detection results.

[0173] In some possible implementations, data augmentation methods can also be used to augment the training face images. This disclosure does not limit the methods used for data augmentation.

[0174] As mentioned above, in some possible implementations, such as Figure 2 As shown, the first convolutional neural network module CM1 is also connected to the second convolutional neural network module CM2, and the second convolutional neural network module CM2 is also connected to the third convolutional neural network module CM3.

[0175] In other words, the input of the second convolutional neural network module CM2 also includes the output of the first convolutional neural network module CM1; the input of the third convolutional neural network module CM3 also includes the output of the second convolutional neural network module CM2.

[0176] By connecting convolutional neural network modules, the low-level feature information extracted by the low-level convolutional neural network module (such as the first convolutional neural network module CM1) can be propagated to the high-level convolutional neural network module (such as the third convolutional neural network module CM3), thereby improving the richness of features obtained by the high-level convolutional neural network module.

[0177] In some possible implementations, the first convolutional neural network module CM1 can be composed of multiple convolutional layers, and the second convolutional neural network module CM2 and the third convolutional neural network module CM3 can be composed of multiple ResNetBlocks.

[0178] Compared to ordinary convolutional layers, ResNetBlock is easier to optimize and can alleviate the gradient vanishing problem caused by increasing depth in deep neural networks. Therefore, ResNetBlock is more suitable as a high-level convolutional neural network than ordinary convolutional layers.

[0179] Experiments showed that the best performance of the face liveness detection model was achieved by using two convolutional layers to form the first convolutional neural network module CM1, four ResNetBlocks to form the second convolutional neural network module CM2, and three convolutional neural network modules CM3.

[0180] In some possible implementations, such as Figure 2 As shown, the face liveness detection model also includes a classification module composed of fully connected layers, which is used to obtain liveness detection results based on features extracted by alternating convolutional neural network modules and visual self-attention modules.

[0181] In some specific implementations, the process of training a face liveness detection model based on training face images and corresponding liveness detection labels can include:

[0182] The training face image is input into the first convolutional neural network module CM1, which consists of two convolutional layers. Downsampling is performed in the second convolutional layer of the first convolutional neural network module CM1 to extract the low-level feature information of the training face image.

[0183] Then, the feature map extracted by the first convolutional neural network module CM1 is input into the first visual self-attention module VM1, which consists of 3 ViT structures. The features output by the first convolutional neural network module CM1 and the first visual self-attention module VM1 are added together and input into the second convolutional neural network module CM2, which consists of 4 ResNet blocks. Similarly, downsampling is performed on the last convolutional layer of the second convolutional neural network module CM2.

[0184] Then, the feature maps extracted by the second convolutional neural network module CM2 are input into the second visual self-attention module VM2, which consists of three ViT structures. The features obtained by the second convolutional neural network module CM2 and the second visual self-attention module VM2 are added together and input into the third convolutional neural network module CM3, which consists of four ResNetBlocks. Similarly, downsampling is performed in the last convolutional layer.

[0185] Finally, the feature map extracted by the third convolutional neural network module CM3 is input into the third visual self-attention module VM3, which consists of three ViT structures. Based on the features output by the third visual self-attention module VM3 (specifically, ClassToken), the final liveness detection result is obtained using the fully connected layer FC1. The network loss is determined based on the final liveness detection result and the liveness detection label corresponding to the training face image, and the parameters for face liveness detection are modified accordingly.

[0186] As mentioned above, in some possible implementations, the face liveness detection model can be trained by freezing the parameters of the modules, so that the convolutional neural network module and the visual self-attention module can better learn the different features of the image.

[0187] Figure 6 This is a flowchart illustrating one implementation of model training by freezing module parameters, as shown below. Figure 6 As shown, model training by freezing the parameters of the module may include steps S610, S620, and S630.

[0188] In step S610, with the parameters of the visual self-attention module of the face liveness detection model frozen, the training face image is input into the pre-trained model corresponding to the face liveness detection model. The pre-trained model corresponding to the face liveness detection model is trained according to the output of the fully connected layer connected by the convolutional neural network module of the face liveness detection model and the liveness detection label corresponding to the training face image, so as to obtain the first training model.

[0189] In step S620, with the parameters of the convolutional neural network module of the first training model frozen, the training face image is input into the first training model, and the first training model is trained according to the output of the fully connected layer connected by the visual self-attention module of the first training model and the liveness detection label corresponding to the training face image, so as to obtain the second training model.

[0190] In step S630, the training face image is input into the second training model. The second training model is trained based on the output of the fully connected layer connected by the visual self-attention module of the second training model and the liveness detection label corresponding to the training face image, so as to obtain the trained face liveness detection model.

[0191] During training, the parameters of all visual self-attention modules are first frozen. The pre-trained model corresponding to the face liveness detection model is trained using the training face image and the corresponding liveness detection label to obtain the first training model.

[0192] The pre-trained model corresponding to the human liveness detection model has the same main structure as the human liveness detection model, which also includes multiple convolutional neural network modules and a visual self-attention module that are alternately connected. The difference is that the last convolutional neural network module is connected to a fully connected layer FC2 to obtain the liveness detection result. The network loss is determined based on the liveness detection result and the liveness detection label corresponding to the training face image. The parameters of the convolutional neural network module of the pre-trained model corresponding to the human liveness detection model are modified based on the network loss.

[0193] Then freeze the parameters of the convolutional neural network module of the first training model, and train the first training model using the training face image and the corresponding liveness detection label to obtain the second training model.

[0194] The first training model has the same structure as the human liveness detection model. The liveness detection result is obtained based on the output of the fully connected layer FC1 of the first training model. The network loss is determined based on the liveness detection result and the liveness detection label corresponding to the training face image. The parameters of the visual self-attention module of the first training model are modified based on the network loss.

[0195] Finally, the training face images and their corresponding liveness detection labels are used to train the second training model, resulting in a trained face liveness detection model.

[0196] The second training model has the same structure as the human liveness detection model. The liveness detection result is obtained based on the output of the fully connected layer FC1 of the second training model. The network loss is determined based on the liveness detection result and the liveness detection label corresponding to the training face image. The parameters of the convolutional neural network module and the visual self-attention module of the second training model are modified based on the network loss.

[0197] By freezing the parameters of the convolutional neural network module and the visual self-attention module separately, the convolutional neural network module and the visual self-attention module can learn the features of the face image to be recognized independently, avoiding mutual interference in the image feature learning process of the convolutional neural network module and the visual self-attention module, and helping the convolutional neural network module and the visual self-attention module learn better features.

[0198] In some possible implementations, the training data for the pre-trained model and the first trained model corresponding to the face liveness detection model are different.

[0199] The first training model is obtained by training the pre-trained model corresponding to the face liveness detection model using the planar attack image and the corresponding liveness detection label of the planar attack image.

[0200] The first training model is trained using stereo attack images and corresponding liveness detection labels to obtain the second training model.

[0201] Since identifying planar attacks such as screen attacks and paper attacks requires extracting low-level and local features of the image, by freezing the parameters of all visual self-attention modules, training the pre-trained model corresponding to the face liveness detection model using planar attack images and the corresponding liveness detection labels can enable the convolutional neural network module to learn to extract low-level and local features.

[0202] Identifying stereo attacks such as hood attacks and head-mounted attacks requires extracting high-level semantic features and global features from images. Therefore, by freezing the parameters of all convolutional neural network modules, training the first training model with stereo attack images and corresponding liveness detection labels can enable the visual self-attention module to learn to extract high-level semantic features and global features.

[0203] Finally, using all the data to train the second training model allows for the full fusion of features extracted by the convolutional neural network module and features extracted by the visual self-attention module.

[0204] This training method enables the face liveness detection model to simultaneously acquire low-level features and high-level semantic features, as well as local features and global features. By fully fusing the acquired features, the richness of the extracted features is improved, thereby enhancing the generalization and accuracy of the face liveness detection model against various attacks.

[0205] Based on and Figure 1 The method shown follows the same principle. Figure 7 A schematic diagram of the structure of a face liveness detection device provided in an embodiment of this disclosure is shown, as follows: Figure 7 As shown, the face liveness detection device 70 may include:

[0206] Test data module 710 is used to acquire the face image to be identified;

[0207] The inference module 720 is used to input the face image to be identified into a pre-trained face liveness detection model to obtain the liveness detection result;

[0208] The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected.

[0209] In the face liveness detection device provided in this embodiment, a face liveness detection model composed of alternating convolutional neural network modules and visual self-attention modules detects the face image to be identified and obtains liveness detection results. Since the convolutional neural network module and the visual self-attention module are different feature extraction modules, they can extract different image features, obtaining multiple features required for face liveness detection, thus improving the generalization and robustness of the face liveness detection model against various attacks. Simultaneously, the alternating connection of the convolutional neural network module and the visual self-attention module allows for the full fusion of the features extracted by both modules, further enhancing the richness of the acquired features and improving the generalization and robustness of the face liveness detection model against various attacks. Therefore, the liveness detection results obtained using the face liveness detection device provided in this embodiment are more accurate.

[0210] It is understood that the above-mentioned modules of the face liveness detection device in the embodiments of this disclosure have the ability to implement... Figure 1 The embodiments shown illustrate the functions of corresponding steps in the face liveness detection method. These functions can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the aforementioned functions. These modules can be software and / or hardware, and each module can be implemented individually or multiple modules can be integrated. For a detailed description of the functions of each module in the aforementioned face liveness detection device, please refer to [link to relevant documentation]. Figure 1The corresponding description of the face liveness detection method in the illustrated embodiments will not be repeated here.

[0211] Based on and Figure 5 The method shown follows the same principle. Figure 8 This diagram illustrates the structure of a training device for a face liveness detection model according to an embodiment of the present disclosure. Figure 8 As shown, the training device 80 for the face liveness detection model may include:

[0212] The training data module 810 is used to acquire multiple training face images and the corresponding liveness detection labels for the training face images;

[0213] The training module 820 is used to train the face liveness detection model based on the training face image and the corresponding liveness detection label of the training face image;

[0214] The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected.

[0215] In the training apparatus for the face liveness detection model provided in this embodiment, the face liveness detection model, composed of alternating convolutional neural network modules and visual self-attention modules, can extract different image features since the convolutional neural network module and the visual self-attention module are different feature extraction modules. This allows for the acquisition of multiple features required for face liveness detection, improving the generalization and robustness of the face liveness detection model against various attacks. Furthermore, the alternating connection of the convolutional neural network module and the visual self-attention module allows for the full fusion of the features extracted by both modules, further enhancing the richness of the acquired features and improving the generalization and robustness of the face liveness detection model against various attacks. Therefore, the face liveness detection model acquired using the training apparatus provided in this embodiment exhibits better performance.

[0216] It is understood that the above-mentioned modules of the training device for the face liveness detection model in the embodiments of this disclosure have the ability to implement... Figure 5 The embodiments shown illustrate the functionality of the corresponding steps in the training method for the face liveness detection model. This functionality can be implemented in hardware or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above-described functions. These modules can be software and / or hardware, and each module can be implemented individually or integrated from multiple modules. For a detailed description of the functions of each module in the training device for the face liveness detection model, please refer to [link to relevant documentation]. Figure 5 The corresponding description of the training method of the face liveness detection model in the illustrated embodiment will not be repeated here.

[0217] The acquisition, storage, and application of user personal information involved in the technical solution disclosed herein comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0218] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0219] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the face liveness detection method and the face liveness detection model training method provided in the embodiments of this disclosure.

[0220] Compared with existing technologies, this electronic device uses a face liveness detection model composed of alternating convolutional neural network modules and visual self-attention modules to detect faces in images and obtain liveness detection results. Since the convolutional neural network module and the visual self-attention module are different feature extraction modules, they can extract different image features, acquiring multiple features required for face liveness detection, thus improving the generalization and robustness of the face liveness detection model against various attacks. Simultaneously, the alternating connection of the convolutional neural network module and the visual self-attention module allows for the full fusion of the features extracted by both modules, further enhancing the richness of the acquired features and improving the generalization and robustness of the face liveness detection model against various attacks. Therefore, the liveness detection results obtained using the electronic device provided in this disclosure are more accurate.

[0221] The readable storage medium is a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to execute the face liveness detection method and the face liveness detection model training method provided in the embodiments of this disclosure.

[0222] Compared with existing technologies, this readable storage medium uses a face liveness detection model composed of alternating convolutional neural network modules and visual self-attention modules to detect faces and obtain liveness detection results. Since the convolutional neural network module and the visual self-attention module are different feature extraction modules, they can extract different image features, obtaining multiple features required for face liveness detection, thus improving the generalization and robustness of the face liveness detection model against various attacks. Simultaneously, the alternating connection of the convolutional neural network module and the visual self-attention module allows for the full fusion of the features extracted by both modules, further enhancing the richness of the acquired features and improving the generalization and robustness of the face liveness detection model against various attacks. Therefore, the liveness detection results obtained using the readable storage medium provided in this disclosure are more accurate.

[0223] The computer program product includes a computer program that, when executed by a processor, implements the face liveness detection method and the face liveness detection model training method provided in the embodiments of this disclosure.

[0224] Compared with existing technologies, this computer program product uses a face liveness detection model composed of alternating convolutional neural network modules and visual self-attention modules to detect faces and obtain liveness detection results. Since the convolutional neural network module and the visual self-attention module are different feature extraction modules, they can extract different image features, obtaining multiple features required for face liveness detection, thus improving the generalization and robustness of the face liveness detection model against various attacks. Simultaneously, the alternating connection of the convolutional neural network module and the visual self-attention module allows for the full fusion of the features extracted by both modules, further enhancing the richness of the acquired features and improving the generalization and robustness of the face liveness detection model against various attacks. Therefore, the liveness detection results obtained using the computer program product provided in this disclosure are more accurate.

[0225] Figure 9 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0226] like Figure 9As shown, device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in read-only memory (ROM) 902 or a computer program loaded into random access memory (RAM) 903 from storage unit 908. RAM 903 may also store various programs and data required for the operation of device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. Input / output (I / O) interface 905 is also connected to bus 904.

[0227] Multiple components in device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of monitors, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0228] The computing unit 901 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as face liveness detection methods and / or face liveness detection model training methods. For example, in some embodiments, the face liveness detection methods and / or face liveness detection model training methods can be implemented as computer software programs tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the face liveness detection methods and / or face liveness detection model training methods described above can be performed. Alternatively, in other embodiments, the computing unit 901 may be configured by any other suitable means (e.g., by means of firmware) to perform a face liveness detection method and / or a face liveness detection model training method.

[0229] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0230] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0231] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0232] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0233] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as a data server), or computing systems that include middleware components (e.g., an application server), or computing systems that include frontend components (e.g., a user computer with a graphical user interface or web browser through which a user can interact with embodiments of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., a communication network). Examples of communication networks include local area networks (LANs), wide area networks (WANs), and the Internet.

[0234] Computer systems can include clients and servers. Clients and servers are generally located far apart and typically interact via communication networks. Client-server relationships are created by computer programs running on the respective computers and having a client-server relationship with each other. Servers can be cloud servers, servers in distributed systems, or servers incorporating blockchain technology.

[0235] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0236] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. A method for face liveness detection, comprising: Acquire the image of the face to be identified; The face image to be identified is input into a pre-trained face liveness detection model to obtain liveness detection results; The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected. The model comprises a first convolutional neural network module, a first visual self-attention module, a second convolutional neural network module, a second visual self-attention module, a third convolutional neural network module, and a third visual self-attention module connected sequentially. The first convolutional neural network module includes multiple convolutional layers; the second convolutional neural network module includes multiple residual network blocks; the third convolutional neural network module includes multiple residual network blocks; and the third visual self-attention module is connected to a fully connected layer. The face liveness detection model is obtained by training a second training model using multiple training face images and corresponding liveness detection labels for the training face images; the multiple training face images include planar attack images and stereo attack images; The second training model is obtained by training the first training model with the stereo attack image and the corresponding liveness detection label of the stereo attack image while freezing the parameters of the convolutional neural network module. The first training model is obtained by training the pre-trained model corresponding to the face liveness detection model using the planar attack image and the liveness detection label corresponding to the planar attack image while freezing the parameters of the visual self-attention module. The step of inputting the face image to be identified into a pre-trained face liveness detection model to obtain liveness detection results includes: The face image to be identified is input into the first convolutional neural network module, and the liveness detection result is determined based on the output of the fully connected layer.

2. The method according to claim 1, wherein, The first convolutional neural network module is connected to the second convolutional neural network module; the second convolutional neural network module is connected to the third convolutional neural network module.

3. The method according to claim 1, wherein, The acquisition of the face image to be identified includes: Obtain an image of the person to be identified, perform face detection on the image of the person to be identified, and obtain the face region of the image of the person to be identified; Facial landmark detection is performed on the face region to obtain the coordinates of the facial landmarks; The face region is aligned according to the coordinates of the facial key points, and a face image of a preset size is obtained through affine transformation. The face image is normalized to obtain the face image to be identified.

4. A training method for a face liveness detection model, comprising: Acquire multiple training face images and the corresponding liveness detection labels for the training face images; The multiple training face images include planar attack images and stereo attack images; The face liveness detection model is trained based on the training face images and the corresponding liveness detection labels. The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected. The model comprises a first convolutional neural network module, a first visual self-attention module, a second convolutional neural network module, a second visual self-attention module, a third convolutional neural network module, and a third visual self-attention module connected sequentially. The first convolutional neural network module includes multiple convolutional layers; the second convolutional neural network module includes multiple residual network blocks; the third convolutional neural network module includes multiple residual network blocks; and the third visual self-attention module is connected to a fully connected layer. The step of training the face liveness detection model based on the training face image and the corresponding liveness detection label includes: With the parameters of the visual self-attention module of the pre-trained model corresponding to the face liveness detection model frozen, the planar attack image is input into the pre-trained model corresponding to the face liveness detection model. The pre-trained model corresponding to the face liveness detection model is trained according to the output of the fully connected layer connected by the convolutional neural network module of the face liveness detection model and the liveness detection label corresponding to the planar attack image to obtain the first training model. With the parameters of the convolutional neural network module of the first training model frozen, the stereo attack image is input into the first training model. The first training model is trained based on the output of the fully connected layer connected by the visual self-attention module of the first training model and the liveness detection label corresponding to the stereo attack image, so as to obtain the second training model. The training face image is input into the second training model. The second training model is trained based on the output of the fully connected layer connected by the visual self-attention module of the second training model and the liveness detection label corresponding to the training face image, so as to obtain the trained face liveness detection model. The step of training the face liveness detection model based on the training face image and the corresponding liveness detection label of the training face image further includes: The training face image is input into the first convolutional neural network module, and the face liveness detection model is trained based on the output of the fully connected layer and the liveness detection label corresponding to the training face image.

5. The method according to claim 4, wherein, The first convolutional neural network module is connected to the second convolutional neural network module; the second convolutional neural network module is connected to the third convolutional neural network module.

6. A device for face liveness detection, comprising: The test data module is used to acquire images of the faces to be identified. The inference module is used to input the face image to be identified into a pre-trained face liveness detection model to obtain liveness detection results; The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected. The model comprises a first convolutional neural network module, a first visual self-attention module, a second convolutional neural network module, a second visual self-attention module, a third convolutional neural network module, and a third visual self-attention module connected sequentially. The first convolutional neural network module includes multiple convolutional layers; the second convolutional neural network module includes multiple residual network blocks; the third convolutional neural network module includes multiple residual network blocks; and the third visual self-attention module is connected to a fully connected layer. The face liveness detection model is obtained by training a second training model using multiple training face images and corresponding liveness detection labels for the training face images; the multiple training face images include planar attack images and stereo attack images; The second training model is obtained by training the first training model with the stereo attack image and the corresponding liveness detection label of the stereo attack image while freezing the parameters of the convolutional neural network module. The first training model is obtained by training the pre-trained model corresponding to the face liveness detection model using the planar attack image and the liveness detection label corresponding to the planar attack image while freezing the parameters of the visual self-attention module. The inference module is also used to input the face image to be identified into the first convolutional neural network module and determine the liveness detection result based on the output of the fully connected layer.

7. A training device for a face liveness detection model, comprising: The training data module is used to acquire multiple training face images and the corresponding liveness detection labels for the training face images; The multiple training face images include planar attack images and stereo attack images; The training module is used to train the face liveness detection model based on the training face image and the corresponding liveness detection label of the training face image; The face liveness detection model includes multiple convolutional neural network modules and multiple visual self-attention modules, which are alternately connected. The model comprises a first convolutional neural network module, a first visual self-attention module, a second convolutional neural network module, a second visual self-attention module, a third convolutional neural network module, and a third visual self-attention module connected sequentially. The first convolutional neural network module includes multiple convolutional layers; the second convolutional neural network module includes multiple residual network blocks; the third convolutional neural network module includes multiple residual network blocks; and the third visual self-attention module is connected to a fully connected layer. The training module is specifically used for: with the parameters of the visual self-attention module of the pre-trained model corresponding to the face liveness detection model frozen, inputting the planar attack image into the pre-trained model corresponding to the face liveness detection model, and training the pre-trained model corresponding to the face liveness detection model based on the output of the fully connected layer connected to the convolutional neural network module of the face liveness detection model and the liveness detection label corresponding to the planar attack image to obtain a first training model; with the parameters of the convolutional neural network module of the first training model frozen, inputting the stereo attack image into the first training model, and training the first training model based on the output of the fully connected layer connected to the visual self-attention module of the first training model and the liveness detection label corresponding to the stereo attack image to obtain a second training model; inputting the training face image into the second training model, and training the second training model based on the output of the fully connected layer connected to the visual self-attention module of the second training model and the liveness detection label corresponding to the training face image to obtain a trained face liveness detection model; The training face image is input into the first convolutional neural network module, and the face liveness detection model is trained based on the output of the fully connected layer and the liveness detection label corresponding to the training face image.

8. An electronic device, comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-3 or the method of claim 4 or 5.

9. A non-transitory computer-readable storage medium storing computer instructions, wherein, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-3 or the method according to claim 4 or 5.

10. A computer program product comprising a computer program that, when executed by a processor, implements the method according to any one of claims 1-3 or the method described in claim 4 or 5.

Images