Face liveness detection method, electronic device, and storage medium

Abstract

The present application provides a face liveness detection method, an electronic device, and a storage medium. The method includes: obtaining a visible light image and a near-infrared image of the same face object; determining a visible light pseudo-depth image corresponding to the visible light image and a near-infrared pseudo-depth image corresponding to the near-infrared image; generating a difference thermal image based on the difference between the visible light pseudo-depth image and the near-infrared pseudo-depth image; and determining whether the face object is a live body according to the difference thermal image. Through the technical solution of the present application, it is possible to accurately identify whether the currently recognized face is a live face without the user's cooperation in performing corresponding actions.

Description

Technical Field

[0001] This application relates to the field of face liveness detection technology, and in particular to a face liveness detection method, an electronic device, and a storage medium. Background Art

[0002] With the development of electronic technology, face recognition has become a potential biometric authentication method. While greatly improving the convenience of people's lives, the security issues of face recognition technology have gradually attracted people's attention. In order to prevent criminals from cracking face recognition through methods such as photos (i.e., aligning a face photo with the camera of a face recognition system for face recognition), face liveness detection technology has also become a research hotspot. The purpose of face liveness detection is to distinguish whether an image is obtained by photographing a real person, and it is a basic component of a face recognition system.

[0003] In related face liveness detection technologies, the detection device will issue random action instructions, such as blinking, shaking the head, or even reading out a string of random numbers, etc., and then detect whether the user's response meets the expectation through the camera. Obviously, this requires the user to cooperate to perform corresponding actions to complete, with a long detection time and poor user experience.

[0004] In view of this, this application is specifically proposed. Summary of the Invention

[0005] This application aims to provide a face liveness detection method, an electronic device, and a storage medium, which can accurately identify whether the currently recognized face is a live face without the user's cooperation to perform corresponding actions.

[0006] In a first aspect, an embodiment of this application provides a face liveness detection method, including:

[0007] Obtain a visible light image and a near-infrared image of the same face object;

[0008] Determine a visible light pseudo-depth image corresponding to the visible light image, and a near-infrared pseudo-depth image corresponding to the near-infrared image;

[0009] Generate a difference heat map according to the difference between the visible light pseudo-depth image and the near-infrared pseudo-depth image;

[0010] Determine whether the face object is a live body according to the difference heat map.

[0011] According to the technical solution provided by the embodiment of this application, optionally, the generating a difference heat map according to the difference between the visible light pseudo-depth image and the near-infrared pseudo-depth image includes:

[0012] Determine the visible light feature map of the visible light pseudo-depth image and the near-infrared feature map of the near-infrared pseudo-depth image based on the trained contrast network;

[0013] Determine the difference between the visible light feature map and the near-infrared feature map;

[0014] Generate a difference heat map according to the difference between the visible light feature map and the near-infrared feature map.

[0015] According to the technical solution provided by the embodiment of the present application, optionally, the trained contrast network includes a plurality of cascaded feature extraction layers in sequence. The determining the visible light feature map of the visible light pseudo-depth image and the near-infrared feature map of the near-infrared pseudo-depth image based on the trained contrast network includes:

[0016] Input the visible light pseudo-depth image into the trained contrast network, obtain the first feature sub-maps respectively output by each of the feature extraction layers, and fuse the first feature sub-maps respectively output by each of the feature extraction layers through a Feature Pyramid Network (FPN) to obtain the visible light feature map of the visible light pseudo-depth image;

[0017] Input the near-infrared pseudo-depth image into the trained contrast network, obtain the second feature sub-maps respectively output by each of the feature extraction layers; fuse the second feature sub-maps respectively output by each of the feature extraction layers through a Feature Pyramid Network (FPN) to obtain the near-infrared feature map of the near-infrared pseudo-depth image.

[0018] According to the technical solution provided by the embodiment of the present application, optionally, the trained contrast network is trained in the following manner:

[0019] Input the visible light pseudo-depth image and the near-infrared pseudo-depth image in the normal sample group into the contrast network to be trained in sequence, and output the visible light feature vector corresponding to the visible light pseudo-depth image and the near-infrared feature vector corresponding to the near-infrared pseudo-depth image through the last-level feature extraction layer of the contrast network;

[0020] Input the visible light feature vector and the near-infrared feature vector into two different branch networks of an asymmetric contrast twin network, and respectively obtain a first prediction value corresponding to the visible light pseudo-depth image and a second prediction value corresponding to the near-infrared pseudo-depth image through the two different branch networks;

[0021] Calculate the loss value between the first prediction value and the second prediction value, and adjust the parameters of each feature extraction layer in the contrast network to be trained with the loss value being zero as the training target.

[0022] Among them, the feature extraction layer includes a convolutional network module and a spatial transformation network module; the normal sample group refers to a combination of visible light pseudo-depth images and near-infrared pseudo-depth images including living human faces.

[0023] According to the technical solution provided by the embodiment of the present application, optionally, determining the difference between the visible light feature map and the near-infrared feature map includes:

[0024] Extracting a first feature vector corresponding to each coordinate position in the visible light feature map, and extracting a second feature vector corresponding to each coordinate position in the near-infrared feature map;

[0025] Combining the first feature vectors corresponding to all coordinate positions in the visible light feature map and the second feature vectors corresponding to all coordinate positions in the near-infrared feature map to form a feature vector matrix, and calculating a covariance matrix according to the feature vector matrix;

[0026] Calculating the Mahalanobis distance between the visible light feature map and the near-infrared feature map at each coordinate position according to the inverse matrix of the covariance matrix, the first feature vector and the second feature vector corresponding to each coordinate position;

[0027] Determining the Mahalanobis distance at each coordinate position as the difference between the visible light feature map and the near-infrared feature map.

[0028] According to the technical solution provided by the embodiment of the present application, optionally, generating a difference heat map according to the difference between the visible light pseudo-depth image and the near-infrared pseudo-depth image includes:

[0029] Mapping the Mahalanobis distance to the color value of the difference heat map at the coordinate position.

[0030] According to the technical solution provided by the embodiment of the present application, optionally, determining whether the face object is a living body according to the difference heat map includes:

[0031] Converting the difference heat map into a grayscale image;

[0032] Calculating the average value of the grayscale values of each pixel in the grayscale image;

[0033] If the average value is less than the threshold, determining that the face object is a living body.

[0034] According to the technical solution provided by the embodiment of the present application, optionally, determining the visible light pseudo-depth image corresponding to the visible light image and the near-infrared pseudo-depth image corresponding to the near-infrared image includes:

[0035] Inputting the visible light image into a trained first feature extraction network to obtain the visible light pseudo-depth image;

[0036] Input the near-infrared image into the trained second feature extraction network to obtain the near-infrared pseudo-depth image;

[0037] Wherein, the training sample label of the first feature extraction network is the pseudo-depth image corresponding to the visible light image, and the training sample label of the second feature extraction network is the pseudo-depth image corresponding to the near-infrared image.

[0038] In a second aspect, an embodiment of the present application further provides an electronic device, which includes:

[0039] A processor and a memory;

[0040] The processor is configured to execute the steps of the face liveness detection method according to any one of the embodiments by calling the program or instruction stored in the memory.

[0041] In a third aspect, an embodiment of the present application further provides a computer-readable storage medium, which stores a program or instruction, and the program or instruction causes a computer to execute the steps of the face liveness detection method according to any one of the embodiments.

[0042] In summary, the present application proposes a face liveness detection method, which acquires visible light images and near-infrared images of the same face object; determines the visible light pseudo-depth image corresponding to the visible light image and the near-infrared pseudo-depth image corresponding to the near-infrared image; generates a difference heat map according to the difference between the visible light pseudo-depth image and the near-infrared pseudo-depth image; determines whether the face object is a live body according to the difference heat map, and can accurately identify whether the currently recognized face is a live face without the user's cooperation to perform corresponding actions, improving the intelligence level and accuracy of the detection and enhancing the user experience. Description of the Drawings

[0043] Figure 1 is a flowchart of a face liveness detection method provided by an embodiment of the present application;

[0044] Figure 2 is a schematic diagram of a depthwise separable convolution structure provided by an embodiment of the present application;

[0045] Figure 3 is a schematic diagram of the structure of a trained contrast network provided by an embodiment of the present application;

[0046] Figure 4 is a schematic diagram when training the contrast network provided by an embodiment of the present application;

[0047] Figure 5 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0048] The present application will be further described in detail below with reference to the accompanying drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention, rather than limiting the invention. In addition, it should be noted that, for the sake of convenience of description, only the parts related to the invention are shown in the drawings.

[0049] It should be noted that, without conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the drawings and embodiments.

[0050] The Driver Monitor System (DMS) is an important part of the intelligent cockpit and is of great significance to commercial vehicles and passenger vehicles. The driver monitoring system can be roughly divided into functions such as camera occlusion, face occlusion, dangerous behavior, fatigue monitoring, distraction monitoring, and face identification FACE ID from a functional perspective. Face liveness detection is a key link in face recognition in the FACE ID function, which can effectively resist attacks by fraud means such as photos, videos, and high-fidelity masks.

[0051] At the present stage, the mainstream liveness detection technologies include interactive cooperative liveness detection technology, silent liveness detection technology, and 3D liveness detection technology. Among them, the interactive cooperative liveness detection technology requires users to perform specific actions (such as blinking, opening the mouth, shaking the head, etc.) according to the system prompts, which requires a high degree of user participation and cooperation and does not have a good user experience. The 3D liveness detection technology uses devices such as three-dimensional depth cameras or structured light cameras to obtain three-dimensional information of the face, and the device cost is relatively high. The silent liveness detection technology does not require users to cooperate to perform specific actions. It monitors by capturing the natural behaviors or physiological characteristics of users, has a good user experience and a low hardware cost. The current mainstream method is to use a single network model, collect real human faces of the liveness and fraudulent human faces of non-liveness as the training data set. However, the acquisition of fraudulent human face images of non-liveness often requires a large amount of manpower and material resources, and the trained network model has poor generalization in various scenarios. When facing a new application scenario, it is necessary to collect new images in the new application scenario again to increment the training data set and iterate the previously trained network model to make it adapt to the new scenario.

[0052] Aiming at the defects existing in the above-mentioned prior art, the present application proposes a liveness detection method based on a multi-modal unsupervised contrast network. This method can be applied to the intelligent cockpit scenario or other face detection scenarios. This method belongs to the category of silent liveness detection and does not require users to cooperate to complete specific actions. Therefore, it has a good user experience and does not require relying on special hardware devices. Therefore, the hardware cost is relatively low, and it has high detection accuracy and low implementation difficulty.

[0053] Figure 1 It is a flowchart of a face liveness detection method provided by an embodiment of the present application. Refer to Figure 1 , the face liveness detection method specifically includes the following steps:

[0054] S110. Obtain a visible light image and a near-infrared image of the same face object.

[0055] Among them, the visible light image is also called an RGB image, which is obtained by imaging with red, green, and blue color components. The near-infrared image is obtained by reflecting near-infrared light. Therefore, there are differences in the imaging principles between the visible light image and the near-infrared image. The differences in the imaging principles result in different imaging results for non-living faces, and there are obvious differences between the imaging results.

[0056] In particular, due to its unique imaging principle, the near-infrared camera cannot image the face displayed on the electronic device and has natural defenses. For a printed face, there are texture differences between its visible light image and near-infrared image. Based on this, the solution of the present application uses multi-modal input data (specifically referring to visible light images and near-infrared images) to implement face liveness detection, improving the detection accuracy and reducing the detection difficulty.

[0057] S120. Determine the visible light pseudo-depth image corresponding to the visible light image and the near-infrared pseudo-depth image corresponding to the near-infrared image.

[0058] Among them, the pseudo-depth image is an image that simulates real depth information generated through a specific algorithm or technology. By using information such as color, texture, and the geometric shape of an object, the relative depth of the object is estimated through calculation or reasoning, thereby creating a pseudo-depth image with a sense of depth.

[0059] Exemplarily, input the visible light image into a trained first feature extraction network to obtain the visible light pseudo-depth image; input the near-infrared image into a trained second feature extraction network to obtain the near-infrared pseudo-depth image; among them, the training sample label of the first feature extraction network is the pseudo-depth image corresponding to the visible light image, and the training sample label of the second feature extraction network is the pseudo-depth image corresponding to the near-infrared image.

[0060] Using the pseudo-depth map as the label for model training enables the model to extract the texture difference features between the visible light image and the near-infrared image, thereby training a multi-modal feature extraction model. There are differences between the visible light pseudo-depth image and the near-infrared pseudo-depth image inferred by the model.

[0061] The differences between live human faces and non-live ones (such as printed photos, electronic screens, 3D face models) are reflected in the texture of local details. By using pseudo-depth images rather than conventional classification labels (live or non-live) as the true training labels, the local texture features of the images are greatly utilized to distinguish the detailed differences between live and non-live faces, thus improving the final detection accuracy.

[0062] In the application scenario of intelligent cockpits, during the model training stage, to adapt to the scenario with limited computing power on the in-vehicle device, the backbone feature network adopts a network structure of depthwise separable convolution, such as the MobileNet series, as Figure 2 shown, which is a depthwise separable convolution structure. By performing depthwise and pointwise separable convolution operations, the number of model parameters is reduced, thus adapting to the scenario with limited computing power on the in-vehicle device.

[0063] Among them, the loss Loss between the model output result and the label can be determined by the following calculation formula:

[0064] Loss = LMSE + LCDL

[0065] Among them, both LMSE and LCDL are mean squared error losses. LMSE is the loss of the category (the category refers to live or non-live), and LCDL is the depth loss of the pseudo-depth map.

[0066] Optionally, the visible light pseudo-depth images and near-infrared pseudo-depth images in the training dataset can be automatically generated by the existing PRNET network, rather than obtained through expensive 3D scanning devices, reducing the difficulty and cost of obtaining training samples. The PRNET network can directly predict the dense 3D face geometry structure (including feature information such as depth, normal, and vertex coordinates) from a single 2D face image. By using the feature extraction network, the visible light image and the near-infrared image are mapped to pseudo-depth images in a unified representation form. Here, the representation form means that the visible light image is an image with three color channels of red, green, and blue, while the near-infrared image is a single-channel grayscale image. Therefore, their representation forms are inconsistent, which are different types of inputs for the model. Therefore, different types of input data need to be mapped to the same type, aiming to reduce the perturbation to the subsequent model accuracy caused by the morphological differences of the input data.

[0067] The multi-modal visible light image and near-infrared image are mapped to pseudo-depth maps, and the pseudo-depth maps are used as the input of the subsequent contrast network, rather than directly using the multi-modal visible light image and near-infrared image as the input of the contrast network, achieving the purpose of unifying the feature representation forms of multi-modal data and reducing the accuracy perturbation caused by data morphological differences to the contrast network.

[0068] S130. Generate a difference heat map based on the differences between the visible light pseudo-depth image and the near-infrared pseudo-depth image.

[0069] Exemplarily, a visible light feature map of the visible light pseudo-depth image and a near-infrared feature map of the near-infrared pseudo-depth image are determined based on the trained contrast network; the difference between the visible light feature map and the near-infrared feature map is determined; and a difference heat map is generated according to the difference between the visible light feature map and the near-infrared feature map. That is, features are further extracted from the visible light pseudo-depth image and the near-infrared pseudo-depth image respectively, and the extracted features are compared. If the difference is large, the detection result is considered as non-living body. If the difference is small, the detection result is considered as living body.

[0070] As Figure 3 shown, the trained contrast network 300 includes a plurality of cascaded feature extraction layers 310 in sequence. The determining of the visible light feature map of the visible light pseudo-depth image and the near-infrared feature map of the near-infrared pseudo-depth image based on the trained contrast network includes:

[0071] The visible light pseudo-depth image is input into the trained contrast network to obtain first feature sub-maps respectively output by the feature extraction layers. The first feature sub-maps respectively output by the feature extraction layers are fused through a Feature Pyramid Network (FPN) to obtain the visible light feature map of the visible light pseudo-depth image; the near-infrared pseudo-depth image is input into the trained contrast network to obtain second feature sub-maps respectively output by the feature extraction layers; and the second feature sub-maps respectively output by the feature extraction layers are fused through a Feature Pyramid Network (FPN) to obtain the near-infrared feature map of the near-infrared pseudo-depth image.

[0072] Among them, the feature extraction layer includes a convolutional network module (i.e., "C" in Figure 3 ), and a Spatial Transformer Network (STN) (i.e., "S" in Figure 3 ). It combines the powerful feature extraction ability of convolution and the flexible spatial invariance of the spatial transformer network module, improving the generalization of the feature extraction layer. The STN is a spatial transformer network, which is a plug-and-play module in deep learning networks, enabling the network to have spatial invariance and having strong robustness to transformations such as translation, rotation, and scaling.

[0073] The Feature Pyramid Network (FPN) is a commonly used structure in computer vision tasks such as object detection. The process of fusing the feature sub-maps respectively output by the feature extraction layers is as follows:

[0074] First, through the bottom-up path, collect feature subgraphs, which is the forward propagation process of a traditional convolutional neural network. In this process, as the number of network layers deepens, the resolution of the feature subgraphs gradually decreases, but the semantic information of the features gradually increases. The feature subgraphs output by convolutional layers at different levels are used as the input of the FPN. Second, continue to collect feature subgraphs through the top-down path. Starting from the highest layer (the feature subgraph output by the highest layer has the richest semantic information but the lowest resolution), perform an upsampling operation on it to increase its resolution to the same resolution as the feature subgraph of the next layer. Third, fuse the upsampled feature subgraph in the second step with the corresponding-level feature subgraph in the first step at the same resolution. Specifically, the feature subgraphs can be added element by element. The purpose of doing this is to make the fused feature subgraph include both rich semantic information and a high resolution. Fourth, perform a convolution operation on the fused feature subgraph, aiming to adjust the number of channels of the feature subgraph.

[0075] Compare the differences between the visible-light pseudo-depth map and the near-infrared pseudo-depth map through the idea of image comparison to obtain the difference heat map of the visible-light pseudo-depth map and the near-infrared pseudo-depth map, and determine the live state of the face based on this difference heat map.

[0076] Among them, the comparison network is an unsupervised network. The unsupervised idea mainly lies in using a general image comparison network for zero-shot transfer to any other application scenario (such as the cockpit scenario) for face liveness detection. An unsupervised network means that there is no need to iterate the network with a face image dataset in a specific application scenario (such as the cockpit scenario). The comparison network can be trained only with normal samples, and calculates the difference in a comparison manner in cooperation with the Mahalanobis distance, rather than simply using the main feature extraction network to map the difference. Therefore, the main feature extraction network of the face pseudo-depth map in a specific application scenario (such as the cockpit scenario) does not need to be specifically trained in the cockpit scenario. The mainstream feature extraction network can be used for zero-shot transfer to the cockpit scenario, and then fuse the sub-feature maps at multiple scales to calculate the Mahalanobis distance of the feature maps corresponding to the visible-light pseudo-depth map and the near-infrared pseudo-depth map respectively, and determine the live state of the face by calculating the difference between the two types of pseudo-depth maps. The comparison network can be trained with normal samples in any scenario, can adapt to any other scenario including the cockpit, has excellent scene transfer generalization, does not need to deliberately obtain face data and non-live data in the cockpit environment, avoids the human and material input for obtaining the training dataset of the comparison network, and reduces the difficulty of the implementation solution of this application.

[0077] The concept of "normal" in normal samples comes from the two states of normal and abnormal samples in anomaly detection. In the embodiment of this application, normal samples refer to the pseudo-depth images of live human faces. Therefore, the construction of the training dataset only requires normal samples and does not require abnormal samples, avoiding the time cost and economic cost required to obtain or collect abnormal samples.

[0078] Reference Figure 4 As shown, the contrast network is trained as follows:

[0079] The visible-light pseudo-depth image and the near-infrared pseudo-depth image in the normal sample group are sequentially input into the contrast network 410 to be trained. The visible-light feature vector corresponding to the visible-light pseudo-depth image and the near-infrared feature vector corresponding to the near-infrared pseudo-depth image are output through the last-level feature extraction layer of the contrast network;

[0080] The visible-light feature vector and the near-infrared feature vector are input into two different branch networks of the asymmetric contrast siamese network 420. The first prediction value Pa corresponding to the visible-light pseudo-depth image and the second prediction value Za corresponding to the near-infrared pseudo-depth image are obtained through the two different branch networks respectively; calculate the loss value between the first prediction value Pa and the second prediction value Za, and use the loss value being zero as the training objective to adjust the parameters of each feature extraction layer in the contrast network to be trained; the normal sample group refers to a combination of the visible-light pseudo-depth image and the near-infrared pseudo-depth image including a live human face. The asymmetric contrast siamese network can enhance data adaptability, extract complementary features, support multimodal data fusion, etc. Specifically, the asymmetric contrast siamese network can adopt different data augmentation strategies, different network structures or parameter settings for the input siamese samples, so as to be able to extract features from different angles for the input data and learn different levels and types of features. Among them, P and Z represent different projection modules, that is, different operations are performed on the input siamese samples (in this embodiment, the visible-light pseudo-depth image and the near-infrared pseudo-depth image), reflecting the "asymmetric" characteristic. E represents the encoding module shared by the siamese samples, usually composed of a convolutional neural network, a recurrent neural network, etc. Loss represents the loss calculation module. After the encoding module E transforms the feature vector output by the last-level feature extraction layer into a high-dimensional space, and then through the mapping of the mapping module P or Z, the first prediction value Pa and the second prediction value Za are obtained, and then the loss value between the first prediction value Pa and the second prediction value Za is calculated, and the loss value being zero is used as the training objective to adjust the parameters of each feature extraction layer in the contrast network to be trained. Among them, a gradient truncation Stop Gradient operation can be introduced to prevent the gradient from backpropagating on a specific path, thereby controlling the parameter update path, preventing the parameters of some modules from being updated during a specific training process, helping the model to learn more stably, and avoiding the problems of gradient explosion or gradient disappearance.

[0081] In some embodiments, determining the difference between the visible-light feature map and the near-infrared feature map includes:

[0082] Extract the first feature vector corresponding to each coordinate position in the visible light feature map, and extract the second feature vector corresponding to each coordinate position in the near-infrared feature map; form a feature vector matrix with the first feature vectors corresponding to all coordinate positions in the visible light feature map and the second feature vectors corresponding to all coordinate positions in the near-infrared feature map, and calculate the covariance matrix according to the feature vector matrix; calculate the Mahalanobis distance between the visible light feature map and the near-infrared feature map at the one coordinate position according to the inverse matrix of the covariance matrix, the first feature vector and the second feature vector corresponding to the one coordinate position; determine the Mahalanobis distance as the difference between the visible light feature map and the near-infrared feature map at the one coordinate position.

[0083] The generating a difference heat map according to the difference between the visible light pseudo-depth image and the near-infrared pseudo-depth image includes:

[0084] Map the Mahalanobis distance to the color value of the difference heat map at the one coordinate position.

[0085] S140. Determine whether the face object is a live body according to the difference heat map.

[0086] Specifically, convert the difference heat map into a grayscale image; calculate the average value of the grayscale values of the pixels in the grayscale image; if the average value is less than the threshold, determine that the face object is a live body.

[0087] The face liveness detection method provided by the embodiments of the present application uses a pseudo-depth map as a label to train a multi-modal feature extraction network. Compared with the traditional face liveness detection method that uses a relatively simple binary classification model to map the entire image to a single classification probability and has difficulty in distinguishing the local texture information of the face, the solution of the present application can better utilize the local detail texture features of the image by using the pseudo-depth map as a label, more accurately represent the local texture information of the face, and better distinguish the image features of live bodies and non-live bodies.

[0088] By using the PRNET network to generate pseudo-depth maps for training labels from visible light images and near-infrared images, it is avoided to generate pseudo-depth maps by scanning a large number of real human faces through a 3D scanning device, greatly saving the economic cost of making training set labels.

[0089] Introduce the idea of unsupervised image contrast into the field of liveness detection, map multi-modal visible light images and near-infrared images into pseudo-depth maps, and use the pseudo-depth maps as the input of the contrast network instead of directly using the multi-modal visible light images and near-infrared images as the input of the contrast network, achieving the purpose of unifying the feature representation forms of multi-modal data and reducing the accuracy perturbation caused by the data form difference to the contrast network.

[0090] The idea of image comparison is introduced, enabling the comparison network to be trained with only normal samples without the need to collect abnormal samples. At the network design level, it avoids a series of equipment and labor inputs such as the hardware device construction of the abnormal sample dataset, image acquisition, selection, annotation, correction, and balanced labeling. After the multi-modal input data presentation forms are aligned, the comparison network can be applied to any scenario. Face liveness detection in the cockpit scenario is just one of the scenarios, truly achieving the effect of zero-sample adaptation to any scenario.

[0091] The Mahalanobis distance is used instead of the Euclidean distance calculated by conventional eigenvalues as the distance calculation method between the feature map of the visible light pseudo-depth map and the feature map of the near-infrared pseudo-depth map. This takes into account the relationship between the single-point difference and the overall distribution in the feature map, excludes the correlation interference between points, and makes the difference calculation perform better.

[0092] Figure 5 It is a schematic structural diagram of an electronic device provided by an embodiment of the present application. As Figure 5 shown, the electronic device 500 includes one or more processors 501 and a memory 502.

[0093] The processor 501 can be a central processing unit (CPU) or other forms of processing units with data processing capabilities and / or instruction execution capabilities, and can control other components in the electronic device 500 to perform desired functions.

[0094] The memory 502 can include one or more computer program products, and the computer program products can include various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory. The volatile memory can include, for example, random access memory (RAM) and / or cache memory. The non-volatile memory can include, for example, read-only memory (ROM), hard disk, flash memory, etc. One or more computer program instructions can be stored on the computer-readable storage medium, and the processor 501 can run the program instructions to implement the face liveness detection method of any embodiment of the present application described above and / or other desired functions. Various contents such as initial external parameters and thresholds can also be stored in the computer-readable storage medium.

[0095] In one example, the electronic device 500 may further include: an input device 503 and an output device 504, and these components are interconnected through a bus system and / or other forms of connection mechanisms (not shown). The input device 503 can include, for example, a keyboard, a mouse, etc. The output device 504 can output various information to the outside, including warning prompt information, braking force, etc. The output device 504 can include, for example, a display, a speaker, a printer, and a communication network and its connected remote output devices, etc.

[0096] Of course, for simplicity, Figure 5 only some of the components related to this application in the electronic device 500 are shown, and components such as buses, input / output interfaces, etc. are omitted. In addition, according to specific application scenarios, the electronic device 500 may further include any other appropriate components.

[0097] In addition to the above methods and devices, an embodiment of the present application may also be a computer program product, which includes computer program instructions that, when run by a processor, cause the processor to execute the steps of the face liveness detection method provided in any embodiment of the present application.

[0098] The computer program product can be written in any combination of one or more programming languages for the program code to perform the operations of the embodiments of the present application. The programming languages include object-oriented programming languages such as Java, C++, etc., and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code can be executed entirely on the user computing device, partially on the user device, executed as an independent software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server.

[0099] Furthermore, an embodiment of the present application may also be a computer-readable storage medium, on which computer program instructions are stored, and when the computer program instructions are run by a processor, the processor is caused to execute the steps of the face liveness detection method provided in any embodiment of the present application.

[0100] The computer-readable storage medium may adopt any combination of one or more readable media. The readable media may be a readable signal medium or a readable storage medium. The readable storage medium may, for example, include but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples (non-exhaustive list) of the readable storage medium include: an electrical connection having one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0101] It should be noted that the terms used in this application are only for describing specific embodiments and do not limit the scope of this application. As shown in the specification and claims of this application, unless the context clearly indicates otherwise, words such as "a", "an", "one", and / or "the" are not specifically singular and may also include plural. The term "comprising", "including" or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or also includes elements inherent to such process, method or device. Without further limitation, an element defined by the statement "including one..." does not exclude the existence of additional identical elements in the process, method or device including the said element.

[0102] It should also be noted that the orientation or positional relationship indicated by terms such as "center", "upper", "lower", "left", "right", "vertical", "horizontal", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing this application and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation to this application. Unless otherwise clearly specified and defined, terms such as "installed", "connected", "joined" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection; it can be a mechanical connection, an electrical connection; it can be directly connected, or indirectly connected through an intermediate medium, and can be the communication inside two elements. For those of ordinary skill in the art, the specific meanings of the above terms in this application can be understood according to specific circumstances.

[0103] Specific examples are used in this article to elaborate on the principles and implementation manners of this application. The description of the above embodiments is only for helping to understand the method and its core idea of this application. The above are only the preferred implementation manners of this application. It should be noted that due to the limited nature of written expression and objectively infinite specific structures, for those of ordinary skill in the art in this technical field, without departing from the principle of this invention, several improvements, retouches or changes can also be made, or the above technical features can be combined in an appropriate manner; these improvements, retouches, changes or combinations, or directly applying the concept and technical solution of the invention to other occasions without improvement, should all be regarded as the protection scope of this application.

Claims

1. A face liveness detection method, characterized in that, Including: Obtaining a visible light image and a near-infrared image of the same face object; Determining a visible light pseudo-depth image corresponding to the visible light image and a near-infrared pseudo-depth image corresponding to the near-infrared image; Generating a difference heat map according to the difference between the visible light pseudo-depth image and the near-infrared pseudo-depth image; Determining whether the face object is a live body according to the difference heat map; The generating a difference heat map according to the difference between the visible light pseudo-depth image and the near-infrared pseudo-depth image includes: Determining a visible light feature map of the visible light pseudo-depth image and a near-infrared feature map of the near-infrared pseudo-depth image based on a trained contrast network; Determining the difference between the visible light feature map and the near-infrared feature map; Generating a difference heat map according to the difference between the visible light feature map and the near-infrared feature map; the determining the difference between the visible light feature map and the near-infrared feature map includes: Extracting a first feature vector corresponding to each coordinate position in the visible light feature map and extracting a second feature vector corresponding to each coordinate position in the near-infrared feature map; Combining the first feature vectors corresponding to all coordinate positions in the visible light feature map and the second feature vectors corresponding to all coordinate positions in the near-infrared feature map to form a feature vector matrix, and calculating a covariance matrix according to the feature vector matrix; Calculating the Mahalanobis distance between the visible light feature map and the near-infrared feature map at each coordinate position according to the inverse matrix of the covariance matrix, the first feature vector and the second feature vector corresponding to each coordinate position; Determining the Mahalanobis distance at each coordinate position as the difference between the visible light feature map and the near-infrared feature map; The generating a difference heat map according to the difference between the visible light pseudo-depth image and the near-infrared pseudo-depth image includes: Mapping the Mahalanobis distance to the color value at the corresponding coordinate position of the difference heat map.

2. The method according to claim 1, wherein The trained contrast network includes a plurality of feature extraction layers cascaded in sequence. The determining a visible light feature map of the visible light pseudo-depth image and a near-infrared feature map of the near-infrared pseudo-depth image based on the trained contrast network includes: Inputting the visible light pseudo-depth image into the trained contrast network to obtain first feature sub-maps respectively output by the feature extraction layers, and fusing the first feature sub-maps respectively output by the feature extraction layers through a Feature Pyramid Network (FPN) to obtain the visible light feature map of the visible light pseudo-depth image; Inputting the near-infrared pseudo-depth image into the trained contrast network to obtain second feature sub-maps respectively output by the feature extraction layers; fusing the second feature sub-maps respectively output by the feature extraction layers through a Feature Pyramid Network (FPN) to obtain the near-infrared feature map of the near-infrared pseudo-depth image.

3. The method according to claim 2, wherein The contrast network is trained in the following manner: The visible light pseudo-depth image and the near-infrared pseudo-depth image in the normal sample group are sequentially input into the contrast network to be trained. Through the last-level feature extraction layer of the contrast network, the visible light feature vector corresponding to the visible light pseudo-depth image and the near-infrared feature vector corresponding to the near-infrared pseudo-depth image are output; The visible light feature vector and the near-infrared feature vector are input into two different branch networks of the asymmetric contrast siamese network, and the first prediction value corresponding to the visible light pseudo-depth image and the second prediction value corresponding to the near-infrared pseudo-depth image are respectively obtained through the two different branch networks; Calculate the loss value between the first prediction value and the second prediction value. With the loss value being zero as the training target, adjust the parameters of each feature extraction layer in the contrast network to be trained; Among them, the feature extraction layer includes a convolutional network module and a spatial transformation network module; the normal sample group refers to a combination of a visible light pseudo-depth image and a near-infrared pseudo-depth image including a live face.

4. The method according to claim 1, characterized in that The determining whether the face object is a live body according to the difference thermal image includes: Convert the difference thermal image into a grayscale image; Calculate the average value of the gray values of each pixel in the grayscale image; If the average value is less than the threshold, determine that the face object is a live body.

5. The method according to claim 1, characterized in that The determining the visible light pseudo-depth image corresponding to the visible light image and the near-infrared pseudo-depth image corresponding to the near-infrared image includes: Input the visible light image into the trained first feature extraction network to obtain the visible light pseudo-depth image; Input the near-infrared image into the trained second feature extraction network to obtain the near-infrared pseudo-depth image; Among them, the training sample label of the first feature extraction network is the pseudo-depth image corresponding to the visible light image, and the training sample label of the second feature extraction network is the pseudo-depth image corresponding to the near-infrared image.

6. An electronic device, characterized in that, The electronic device includes: A processor and a memory; The processor is configured to execute the steps of the face liveness detection method according to any one of claims 1 to 5 by calling the program or instruction stored in the memory.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program or instruction, and the program or instruction causes the computer to execute the steps of the face liveness detection method according to any one of claims 1 to 5.

Images