Face living body recognition method and device, electronic equipment and storage medium

Abstract

Embodiments of the present application disclose a face living body recognition method and device based on binocular images, electronic equipment and storage medium, the method comprises: acquiring binocular images; the binocular images are generated by using binocular camera to shoot the face from different angles, and the binocular images comprise left eye images and right eye images; disparity estimation is carried out on the face by using the binocular images, and a disparity map of the face is obtained; the disparity map is used to describe the position difference of the face from different angles; the features of the disparity map and binocular images are extracted as a gating component, and a confidence map is obtained by feature fusion through the gating component; the confidence map is used to represent the confidence of the face belonging to a real face or a plane attack; and living body recognition is carried out on the face according to the confidence map, and a living body recognition result is obtained. The present application solves the problems of large calculation amount and unstable recognition result of related art face living body recognition, and is difficult to be applied to real scenes.

Description

Technical Field

[0001] This invention relates to the field of computer vision, and in particular to a method, apparatus, electronic device, and storage medium for human face liveness recognition based on binocular images. Background Technology

[0002] In robust face systems, static face recognition is an important prerequisite for ensuring the security of face recognition systems. However, static face recognition remains a challenging and under-explored problem because, from a data perspective, learning to perform face recognition from RGB images depends primarily on the image context. Single-view images without additional annotations contain only limited semantic information.

[0003] With the widespread application of computer vision and deep learning, facial recognition has made significant progress in recent years, primarily used for authentication in many real-world scenarios such as mobile payments and access control. Therefore, in addition to accuracy, the security of facial recognition systems must also be considered. Facial recognition (FAS) technology has attracted widespread attention for addressing this critical need, as it aims to distinguish between genuine human faces and planar attacks.

[0004] However, existing FAS methods require a video or a series of consecutive images as input, which involves cumbersome design. In addition, single-view images contain only limited semantic information, and often cannot achieve stable results due to the limitations of the prior and the complexity of real-world scenes. Video-based multi-frame algorithms have high requirements for the number of parameters and computation, making them difficult to apply in real-life scenarios, especially mobile phones, fixed embedded devices, and mobile embedded devices, resulting in a poor user experience.

[0005] Therefore, there is an urgent need for a face liveness detection method that is computationally efficient, produces stable results, and is suitable for real-world scenarios. Summary of the Invention

[0006] The embodiments of the present invention provide a method, device, electronic device and storage medium for face liveness recognition based on binocular images, so as to solve the problems of large computational load and unstable recognition results in face liveness recognition in related technologies, which makes it difficult to apply to real-world scenarios.

[0007] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0008] According to one aspect of the present invention, a face liveness detection method based on binocular images is provided. The method includes: acquiring binocular images; the binocular images are generated by capturing a face from different perspectives using a binocular camera, the binocular images including a left-eye image and a right-eye image; performing disparity estimation on the face using the binocular images to obtain a disparity map of the face; the disparity map is used to describe the positional differences of the face at different perspectives; extracting features from the disparity map and the binocular images as gating components, and performing feature fusion through the gating components to obtain a confidence map; the confidence map is used to represent the confidence level of whether the face belongs to a real face or a planar attack; and performing liveness detection on the face based on the confidence map to obtain a liveness detection result.

[0009] In an exemplary embodiment, performing disparity estimation on the face using binocular images to obtain a disparity map of the face includes extracting features from the left and right binocular images to obtain a left feature map and a right feature map; obtaining a feature-matched left and right feature maps based on the self-attention and cross-attention of the left and right feature maps; obtaining attention weights based on the similarity of pixels between the feature-matched left and right feature maps; and calculating the disparity map of the face using the attention weights.

[0010] In an exemplary embodiment, obtaining the feature-matched left and right eye images based on the self-attention and cross-attention of the left and right feature images includes: calculating the self-attention of the left and right feature images using a first number of attention heads; calculating the cross-attention between the left and right feature images using a second number of attention heads; and performing feature matching on the left and right eye images based on the self-attention and cross-attention to obtain the feature-matched left and right eye images.

[0011] In an exemplary embodiment, features of the disparity map and the binocular image are extracted as gating components. Obtaining a confidence map based on the gating components includes extracting features from the disparity map using a first gating branch and extracting features from the left or right binocular image using a second gating branch. During feature extraction, the features extracted by the first gating branch are introduced into the second gating branch and multiplied with the features extracted by the second gating branch to obtain a first image feature and a second image feature. The first image feature and the second image feature are fused to obtain a first confidence map and a second confidence map, respectively. The first confidence map represents the confidence that the face belongs to a planar attack, and the second confidence map represents the confidence that the face belongs to a real face.

[0012] In an exemplary embodiment, performing liveness detection on the face based on the confidence map to obtain the liveness detection result includes performing global average pooling on the first confidence map and the second confidence map to obtain the average pool of the confidence map; performing linear calculation on the average pool of the confidence map to obtain a first probability that the face belongs to a planar attack and a second probability that the face belongs to a real face; and comparing the first probability and the second probability with a set threshold to obtain the liveness detection result.

[0013] In an exemplary embodiment, face liveness recognition based on binocular images is implemented by calling a face liveness recognition model, which is a trained machine learning model capable of performing face liveness recognition on the binocular images.

[0014] In an exemplary embodiment, the face liveness recognition model includes a disparity estimation module and a confidence map generator. The training process of the face liveness recognition model includes acquiring a training set, wherein the training images in the training set refer to binocular images including a left-eye image and a right-eye image; inputting the training images in the training set into the face liveness recognition model for training to obtain the loss value of the disparity estimation module and the loss value of the confidence map generator; obtaining a total loss value based on the loss value of the disparity estimation module and the loss value of the confidence map generator; if the total loss value satisfies the convergence condition, the training is completed, and the face liveness recognition model is obtained; otherwise, the model parameters of the machine learning model are updated, and the training images in the training set are acquired and input into the machine learning model for further training until the total loss value satisfies the convergence condition.

[0015] In an exemplary embodiment, training images from the training set are input into the face liveness recognition model for training. The loss values ​​of the disparity estimation module and the confidence map generator are obtained by: performing 3D face alignment on the training images in the training set to obtain a 3D face reconstruction depth map of the training images; performing image segmentation on the human figure and background in the training images to obtain a human body region segmentation map of the training images; obtaining a target depth image based on the 3D face reconstruction depth map and the human body region segmentation map of the training images; and obtaining the loss values ​​of the disparity estimation module and the confidence map generator based on the disparity map, confidence map, and target depth image output by the face liveness recognition model.

[0016] According to one aspect of the present invention, a face liveness recognition device based on binocular images is provided. The device includes a binocular image acquisition module for acquiring binocular images, which are generated by capturing a face from different perspectives using a binocular camera. The binocular images include a left-eye image and a right-eye image. A disparity estimation module is used to estimate the disparity of the face using the binocular images to obtain a disparity map of the face. A confidence acquisition module is used to extract features from the disparity map and the binocular images as gating components, and to obtain a confidence map through feature fusion performed by the gating components. A result acquisition module is used to perform liveness recognition on the face based on the confidence map to obtain a liveness recognition result.

[0017] According to one aspect of the present invention, an electronic device includes a processor and a memory, wherein the memory stores computer-readable instructions, which, when executed by the processor, implement the face liveness recognition method based on binocular images as described above.

[0018] According to one aspect of the present invention, a storage medium having a computer program stored thereon, which, when executed by a processor, implements the face liveness recognition method based on binocular images as described above.

[0019] In the above technical solution, the present invention solves the problem that the related technologies for face liveness recognition have a large computational load and unstable recognition results, making them difficult to apply to real-world scenarios.

[0020] Specifically, this invention first uses a binocular camera to capture images of a face from different perspectives, obtaining binocular images including left and right eye images. Disparity estimation is then performed on the face using these binocular images to obtain a disparity map describing the positional differences of the face at different perspectives. Features from the disparity map and the binocular images are extracted as gating components. Based on the gating components, a confidence level is obtained to indicate whether the face is a real face or a planar attack. Liveness detection is then performed on the face based on the confidence map, ultimately yielding the liveness detection result. In this process, this invention uses binocular face images containing different lighting and distances to provide conditions for stereo matching and depth estimation. Depth estimation yields a pixel-level disparity map, and the generation of a confidence map avoids the loss of facial texture information, thus improving recognition accuracy. This significantly reduces computational load and improves the stability of face liveness detection, making it suitable for real-world scenarios.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit the invention. Attached Figure Description

[0022] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with the invention and, together with the description, serve to explain the principles of the invention.

[0023] Figure 1 This is a schematic diagram of a face liveness detection scenario in the background technology;

[0024] Figure 2 This is a schematic diagram based on the implementation environment involved in this application;

[0025] Figure 3 This is a flowchart illustrating a face liveness detection method based on binocular images according to an exemplary embodiment;

[0026] Figure 4 yes Figure 3 A schematic diagram illustrating the training of the face liveness recognition model in the corresponding embodiment;

[0027] Figure 5 yes Figure 3 A schematic diagram of the disparity estimation module in the corresponding embodiment;

[0028] Figure 6 yes Figure 3 A schematic diagram of the structure of the dynamically matched attention block in the corresponding embodiment;

[0029] Figure 7 yes Figure 3 A schematic diagram of the confidence graph generator in the corresponding embodiment;

[0030] Figure 8 This is a schematic diagram illustrating the effect of an exemplary embodiment in an application scenario;

[0031] Figure 9 This is a schematic diagram of an application scenario according to an exemplary embodiment;

[0032] Figure 10 This is a block diagram illustrating a face liveness recognition device based on binocular images according to an exemplary embodiment;

[0033] Figure 11 This is a hardware structure diagram of an electronic device according to an exemplary embodiment;

[0034] Figure 12 This is a block diagram illustrating an electronic device according to an exemplary embodiment.

[0035] The accompanying drawings have illustrated specific embodiments of the present invention, which will be described in more detail below. These drawings and descriptions are not intended to limit the scope of the inventive concept in any way, but rather to illustrate the concept of the invention to those skilled in the art by referring to specific embodiments. Detailed Implementation

[0036] The embodiments of this application are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain this application, and should not be construed as limiting this application.

[0037] Those skilled in the art will understand that, unless specifically stated otherwise, the singular forms “a,” “an,” “the,” and “the” used herein may also include the plural forms. It should be further understood that the term “comprising” as used in this application means the presence of the stated features, integers, steps, operations, elements, and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components, and / or groups thereof. It should be understood that when we say an element is “connected” or “coupled” to another element, it can be directly connected or coupled to the other element, or there may be intermediate elements. Furthermore, “connected” or “coupled” as used herein can include wireless connections or wireless coupling. The term “and / or” as used herein includes all or any units and all combinations of one or more associated listed items.

[0038] like Figure 1 As shown, with the widespread application of computer vision and deep learning, facial recognition has made significant progress, mainly used for authentication in many real-world scenarios, such as mobile payment and access control. Therefore, in addition to accuracy, the security of facial recognition systems also needs to be considered. For this critical requirement, facial liveness detection technology has attracted widespread attention. It aims to distinguish between genuine faces and 2D attacks, but remains a challenging problem with insufficient exploration.

[0039] Therefore, existing face liveness detection algorithms can be divided into two main categories: monocular image algorithms and video-based multi-frame algorithms. Monocular image algorithms require tedious manual design and often lack stable performance due to limitations in prior knowledge and the complexity of real-world scenarios. Deep learning-based monocular image methods are highly dependent on data. Furthermore, directly learning the mapping from RGB images to liveness binary classification from monocular images without effective prior information constraints results in poor generalization performance on other datasets, especially in unfamiliar scenarios. Video-based multi-frame algorithms require tedious video processing. The inherent characteristics of video, such as motion blur, large file size, and uncertain frame rates, also make video methods highly dependent on data. Additionally, learning from multiple frames within a video requires very complex model design and demands a large number of parameters and computational resources, making them difficult to apply in real-world scenarios, especially on mobile phones, fixed embedded devices, and mobile embedded devices, resulting in a poor user experience.

[0040] As can be seen from the above, the relevant technologies still have the drawbacks of large computational load and unstable recognition results for face liveness recognition, making them difficult to apply to real-world scenarios.

[0041] To address this, the binocular image-based face liveness detection method provided in this application first uses a binocular camera to capture images of a face from different perspectives, obtaining binocular images including left and right eye images. Disparity estimation is then performed on the face using the binocular images to obtain a disparity map describing the positional differences of the face at different perspectives. Features from the disparity map and the binocular images are extracted as gating components. Based on the gating components, a confidence level is obtained to indicate whether the face belongs to a real face or is a planar attack. Liveness detection is then performed on the face based on the confidence map, ultimately yielding the liveness detection result. The binocular image-based face liveness detection method provided by this invention significantly reduces computational load and improves the stability of face liveness detection. It is suitable for real-world scenarios. This binocular image-based face liveness detection method is applicable to binocular image-based face liveness detection devices, which can be deployed on electronic devices. These electronic devices can be computer devices configured with a von Neumann architecture, such as desktop computers, laptops, servers, etc. They can also be devices with binocular image acquisition capabilities.

[0042] Figure 2 This is a schematic diagram of an implementation environment for a face liveness detection method based on binocular images. The implementation environment includes a data acquisition terminal 110 and a server terminal 130.

[0043] Specifically, the acquisition terminal 110 acquires binocular facial images. The acquisition terminal 110 can be a smartphone, tablet, laptop, desktop computer, or other electronic device with image acquisition function (such as a smart camera), and there is no limitation on it here.

[0044] The acquisition terminal 110 and the server terminal 130 can establish a communication connection through wired or wireless means to realize data transmission between them. For example, the transmitted data can be binocular face images, etc.

[0045] The server-side 130 can also be considered as the cloud, cloud platform, platform-side, server-side, etc. This server-side 130 can be a single server, a server cluster consisting of multiple servers, or a cloud computing center consisting of multiple servers, in order to better provide backend services to the massive data collection terminal 110. For example, the backend services include face liveness recognition services based on binocular images.

[0046] With the interaction between the acquisition terminal 110 and the server terminal 130, in one application scenario, taking the server terminal 130 providing a face liveness recognition service based on binocular images as an example, after the acquisition terminal 110 acquires a binocular face image, it sends the binocular face image to the server terminal 130. The server terminal 130 can then receive the binocular face image sent by the acquisition terminal 110 and provide a face liveness recognition service based on the binocular image. Specifically, after acquiring the binocular face image, the server terminal 130 can perform disparity estimation on the face using a face liveness recognition algorithm based on binocular images, obtaining a disparity map describing the positional differences of the face at different viewpoints. Features from the disparity map and the binocular image are extracted as gating components. Based on the gating components, a confidence level is obtained to indicate whether the face belongs to a real face or a planar attack. Liveness recognition is then performed on the face based on the confidence map, ultimately obtaining the liveness recognition result.

[0047] Of course, in another application scenario, the acquisition terminal 110 can also simultaneously acquire binocular face images and perform face liveness recognition based on binocular images, which is not limited here.

[0048] Please see Figure 3 This application provides a face liveness recognition method based on binocular images. This method is applicable to electronic devices, such as desktop computers, laptops, servers, etc., or electronic devices with image acquisition functions.

[0049] In the following method embodiments, for ease of description, the execution subject of each step of the method is an electronic device, but this does not constitute a specific limitation.

[0050] like Figure 3 As shown, the method may include the following steps:

[0051] Step 110: Obtain binocular images. Binocular images are generated by capturing images of a face from different angles using a binocular camera.

[0052] The binocular face image can be obtained by capturing and acquiring images through a data acquisition device. This acquisition device can be an electronic device with binocular image acquisition capabilities, such as a binocular camera or a smartphone equipped with a binocular camera. It can be understood that capturing can be a single shot or multiple shots, i.e., multiple photos. Therefore, the binocular face image can be any one of these multiple photos. In other words, in this embodiment, the binocular image-based face liveness detection method is implemented for binocular face images.

[0053] Regarding image acquisition, the images can originate from binocular images captured in real-time by the acquisition device, or from binocular images captured by the acquisition device within a historical time period and pre-stored in the electronic device. Therefore, for the electronic device, after capturing and acquiring the binocular images, it can process them in real-time or pre-store them for later processing. For example, it can process the binocular images when the CPU of the electronic device is low, or process them according to the instructions of the staff. Thus, the face liveness recognition based on binocular images in this embodiment can be applied to images acquired in real-time or images acquired within a historical time period; no specific limitation is made here.

[0054] The inventors realized that using monocular images would result in a lack of depth information, over-reliance on image texture, and poor model generalization. Using video would result in problems such as motion blur, large file size, and uncertain frame rate, and would also be highly dependent on data. Therefore, they used binocular images, which can provide conditions for stereo matching and depth estimation, to avoid the instability of results caused by over-reliance on image data, thereby significantly improving the generalization of face liveness recognition and making it suitable for real-world scenarios.

[0055] In one possible implementation, a binocular near-infrared camera is used to capture binocular facial images, which can reflect light from all electronic screens, increasing the physical ability to recognize faces and reducing the difficulty of facial liveness detection.

[0056] Step 130: Use binocular images to estimate the disparity of the face to obtain a disparity map of the face; the disparity map is used to describe the positional differences of the face at different viewpoints.

[0057] Among them, disparity map refers to the difference between the image points of the same physical point in different images. Disparity estimation refers to obtaining the three-dimensional geometric information of the object by calculating the positional deviation between corresponding points in the images, based on the principle of disparity and by using imaging equipment to acquire two images of the object from different positions.

[0058] In other words, the embodiments of the present invention utilize binocular images to perform disparity estimation on faces, obtaining pixel-level disparity maps. This allows for the learning of three-dimensional geometric information of the face and the acquisition of depth information from the face image, enabling more accurate face liveness recognition. This improves the accuracy and stability of the recognition results and is applicable to real-world scenarios.

[0059] In one possible implementation, the disparity estimation in this embodiment of the invention is achieved through a self-attention mechanism and a cross-attention mechanism, enabling the disparity estimation to obtain a more accurate disparity map with depth information, thus providing useful information for subsequent liveness detection. Specifically, features are extracted from the left and right binocular images to obtain a left feature map and a right feature map. Based on the self-attention and cross-attention of the left and right feature maps, feature-matched left and right images are obtained. Attention weights are obtained based on the similarity of pixels between the feature-matched left and right images. Finally, the disparity map of the face is calculated using these attention weights.

[0060] The inventors realized that focusing different attention heads on extracting different features can enhance feature representation capabilities. Self-attention partial attention heads focus on the information flow within the same image, while cross-attention partial attention heads focus on feature matching between the left and right images. Specifically, self-attention is calculated between the left and right feature maps using a first number of attention heads, and cross-attention is calculated between the left and right feature maps using a second number of attention heads. Feature matching is then performed on the left and right visual images based on self-attention and cross-attention, resulting in feature-matched left and right visual images. This achieves more accurate and stable feature matching, thereby generating disparity maps with richer and more accurate depth information.

[0061] Step 150: Extract the features of the disparity map and the binocular image as a gating component, and obtain the confidence map through feature fusion performed by the gating component.

[0062] The inventors realized that using only disparity maps to perform face liveness detection would result in the loss of all facial texture information. Therefore, in one possible implementation, face liveness detection is performed by generating a confidence map. Specifically, a first gating branch is used to extract features from the disparity map, and a second gating branch is used to extract features from either the left or right eye image of the binoculars. During feature extraction, the features extracted by the first gating branch are multiplied by the features extracted by the second gating branch to obtain first image features and second image features. These first and second image features are then fused to obtain the confidence map. This confidence map represents the confidence level of whether a face belongs to a planar attack or is a genuine face.

[0063] In one possible implementation, the confidence graph includes a first confidence graph and a second confidence graph, wherein the first confidence graph represents the confidence that a face belongs to a planar attack, and the second confidence graph represents the confidence that a face belongs to a real face.

[0064] Through the above process, the embodiments of the present invention generate a discriminative confidence map by gating and feature fusion, thereby obtaining accurate and stable liveness detection results.

[0065] Step 170: Perform liveness detection on the face based on the confidence map to obtain the liveness detection result.

[0066] For example, based on the confidence level of the face represented by the confidence map, whether it belongs to a planar attack or a real face, if the confidence level of the confidence map representing a face as a planar attack is less than 0.5 and / or the confidence level of the confidence map representing a face as a real face is greater than 0.5, it indicates that the face belongs to a real face; otherwise, it indicates that the face belongs to a planar attack.

[0067] Through the above process, this invention first uses a binocular camera to capture images of a face from different perspectives, obtaining binocular images including left and right eye images. Disparity estimation is then performed on the face using these binocular images to obtain a disparity map describing the positional differences of the face at different perspectives. Features from the disparity map and the binocular images are extracted as gating components. Based on the gating components, a confidence level is obtained to indicate whether the face is a real face or a planar attack. Liveness detection is then performed on the face based on the confidence map, ultimately yielding the liveness detection result. In this process, this invention uses binocular face images containing different lighting and distances to provide conditions for stereo matching and depth estimation. Depth estimation yields a pixel-level disparity map, and the generation of a confidence map avoids the loss of facial texture information, thus improving recognition accuracy. This significantly reduces computational load and improves the stability of face liveness detection, making it suitable for real-world scenarios.

[0068] In an exemplary embodiment, the disparity estimation in this invention is obtained through a face liveness detection model. Figure 4 A schematic diagram of the structure of a face liveness recognition model in one embodiment is shown. The face liveness recognition model includes a disparity estimation module (Disparity Estimation Transformer) for disparity estimation of binocular face images, a confidence map generator (Confidence Map Generator) for generating confidence maps, and a dual-teacher distillation module (Dual-teacher Distillation Module) for weakly supervising the training of the face liveness recognition model. The disparity estimation module includes a dynamic matching attention module (DMAblock).

[0069] The following is combined with Figure 4 The structure of the face liveness detection model and the training process of the face liveness detection model are explained in detail below:

[0070] like Figure 4 As shown, in an exemplary embodiment, training a face liveness detection model may include the following steps:

[0071] Step S1: Obtain the training set.

[0072] The training images in the training set refer to binocular images, including both the left and right eye images. See also: Figure 4 The binocular images, including the left image (Left Img) and the right image (Right Img), are acquired and constructed in real-world scenes using binocular near-infrared (NIR) cameras. The use of NIR cameras can capture reflections from all electronic screens, increasing the physical face recognition capability. The training set contains real faces under different lighting and distances, as well as planar attacks on different materials, which can be used to comprehensively and realistically train the face liveness recognition model.

[0073] Step S2: Input the training images from the training set into the face liveness recognition model for training, and obtain the loss value of the disparity estimation module and the loss value of the confidence map generator.

[0074] In one possible implementation, weak supervision of the face liveness detection model training is achieved through a Dual-teacher Distillation Module. This includes performing 3D face alignment on the training images in the training set to obtain a 3D face reconstruction depth map of the training images; performing image segmentation on the human figure and background in the training images to obtain a human body region segmentation map of the training images; obtaining a target depth image based on the 3D face reconstruction depth map and the human body region segmentation map of the training images; and obtaining the loss values ​​of the disparity estimation module and the confidence map generator based on the disparity map, confidence map, and target depth image output by the face liveness detection model.

[0075] Step S3: Obtain the total loss value based on the loss value of the disparity estimation module and the loss value of the confidence map generator.

[0076] Step S4: If the total loss value meets the convergence condition, the training is completed and a face liveness recognition model is obtained; otherwise, the model parameters of the machine learning model are updated, and the training images in the training set are obtained to continue training the machine learning model until the total loss value meets the convergence condition.

[0077] Continue reading Figure 4The left image (Left Img) and right image (Right Img) are input into the disparity estimation module (Disparity Estimation Transformer) to obtain a disparity map (Disparity). The Disparity map (Disparity) is then reconstructed (Rec) to obtain a reconstruction map. Based on the reconstruction map and the left image (Left Img), a reconstruction loss (Reconstruction Loss) is calculated. Based on the Disparity map (Disparity), a disparity smoothing loss (Disparity Smooth Loss) and a weighted relative disparity loss (Weighted Relative Disparity Loss) are also calculated. The Disparity map (Disparity) and the left image (Left Img) are then input into the confidence map generator (Confidence Map Generator). Based on the trained Confidence Map Generator, a first confidence map (Confidence Map for Plane Attack) representing the confidence that a face belongs to a plane attack and a second confidence map (Confidence Map for Real Face) representing the confidence that a face belongs to a real face are obtained. Based on the second confidence map (Confidence Map for Real Face), the confidence loss (Focal) is calculated. Confidence Map Loss is used for training, and the probability of belonging to a planar attack is 0.01 and the probability of belonging to a real human face is 0.99, thus obtaining the result of face liveness detection.

[0078] The inventors realized that the training images in the training set were captured in real-world scenes using a binocular near-infrared (NIR) camera, making it difficult to obtain the true depth maps of these binocular images. Furthermore, determining the true disparity was also challenging. While true depth information was difficult to obtain, relative depth information was readily available. Manually labeling such a large amount of point-to-point data was extremely time-consuming and unnecessary. Therefore, this invention, based on the concept of knowledge distillation, constructs a Dual-teacher Distillation Module for weakly supervising the training of a face liveness recognition model. (Continue reading...) Figure 4The Dual-teacher Distillation Module includes two teachers: a Face Label Teacher and a Body Label Teacher. In one possible implementation, the 3D face alignment method 3DDFA-V2 is used as the Face Label Teacher, and the human image segmentation method PP-HumanSeg is used as the Body Label Teacher. The two teacher labels are fused by directly adding and averaging the values ​​of the face and body parts. The student model, a face liveness recognition model, is trained by generating a Weighted Relative Disparity Loss and a Focal Confidence Map Loss.

[0079] Specifically, before inputting the disparity map and the left image (Left Img) into the confidence map generator, the trained confidence map generator has already been obtained based on the triple loss value of disparity smoothing loss, weighted relative disparity loss, and confidence map loss (Focal Confidence Map Loss).

[0080] Under the above embodiments, the present invention obtains pixel-level disparity maps through the disparity estimation module Disparity EstimationTransformer, uses a trained confidence map generator in cooperation with a dual-teacher extraction module for weakly supervised training to obtain the final discrimination result, and utilizes various losses to achieve interpretable weak supervision, thereby obtaining a trained face liveness recognition model, improving the accuracy and stability of face liveness recognition, and providing a face liveness recognition method suitable for real-world scenarios.

[0081] In an exemplary embodiment, the disparity estimation in this invention is obtained through the disparity estimation module Disparity Estimation Transformer. Figure 5 A schematic diagram of the disparity estimation module in one embodiment is shown, which includes a dynamic matching attention module (DMAblock).

[0082] The following is combined with Figure 5 The structure of the disparity estimation module and the disparity estimation process are explained in detail below:

[0083] like Figure 5 As shown, the binocular images are input into the disparity estimation module. First, a Feature Extractor is performed on the left and right binocular images. The extracted features are then input into a Dynamic Matching Attention (DMA) block for multiple upsampling and downsampling operations. Feature matching is then performed on the left and right binocular images. To better transfer attention information between multiple DMA blocks, a Residual Connection of Attention Weight is used. Furthermore, to prevent incorrect matching during feature matching, a Cross-Attention Mask is used in the last DMA block. Attention weights are obtained based on the pixel similarity between the left and right binocular images after feature matching, as output by the last DMA block. A Disp Estimator is then used based on these attention weights to obtain the disparity map of the binocular images.

[0084] Figure 6 A schematic diagram of the dynamic matching attention module (DMA block) in one embodiment is shown. The following section combines... Figure 6 The structure of the Dynamic Matching Attention Module (DMA block) provides a detailed explanation of the feature matching process:

[0085] like Figure 6 As shown, the attention module is used to calculate the attention between sequence elements. It uses dot products to integrate information from the query vector and key vector to obtain the attention weights of the value vector. In this embodiment, to allow different heads to focus on extracting different features, the inventors use half of the head for calculating Self-attention and the other half for calculating Cross-attention in the Dynamic Matching Attention Module (DMA block), enhancing the feature representation capability of the attention module. Specifically, half of the head for Self-attention focuses on the information flow within the same image, while the other half of the head for Cross-attention focuses on feature matching between the left and right images. Furthermore, we use depthwise convolution and downsampling to extract feature information between context and channels. Considering that the size of the matching window should dynamically change with the sampling scale to adapt to the pixel correspondence between the left and right images, the inventors proposed a dynamic matching attention block, such as... Figure 6As shown, patch concatenation (upsampling) and decomposition (downsampling) are added to different layers. Patch concatenation connects the features of four adjacent patches, halving their resolution, and then uses a linear layer to double their feature dimension. Patch decomposition is the opposite. Within the dynamic matching attention block, after the input features are assigned to Q, K, and V vectors, depthwise convolution is used in different heads to downsample these Q, K, and V vectors to different resolutions, allowing the features between the left and right images of the binocular image to flow and merge fully, and outputting the left and right images after feature matching.

[0086] Through the above process, this invention introduces a dynamic matching attention mechanism, a multi-head self-attention mechanism, and a cross-attention mechanism into the face liveness recognition process based on binocular images, ensuring the accuracy and stability of feature matching and significantly improving the accuracy and stability of face liveness recognition based on binocular images.

[0087] Furthermore, the upsampling and downsampling methods used in the embodiments of the present invention ensure that the total amount of information is not lost, further improving the accuracy and stability of face liveness recognition based on binocular images.

[0088] Figure 7 A schematic diagram of a confidence map generator in one embodiment is shown below. Figure 7 The structure of the confidence map generator and the process of generating the confidence map are explained in detail below:

[0089] like Figure 7 As shown, the confidence map has two channels with the same resolution as the input binocular image. The values ​​of the two channels in the confidence map represent the probability that the position in the image belongs to a real face or a planar attack. The confidence map is generated using a gated branch network with Swin-MLP blocks. The encoder of this network has two branches: Texture Branch and Gate Branch, corresponding to the disparity map and the left image, respectively. Each encoder branch consists of two convolutional blocks and two Swin-MLP blocks with downsampling. The outputs of different branches are multiplied accordingly to achieve the gating effect. Then, the features of the two branches are connected to the two Swin-MLP blocks and the two convolutional blocks with upsampling for feature fusion to obtain a first confidence map that discriminates the confidence that the face belongs to a planar attack and a second confidence map that indicates the confidence that the face belongs to a real face. Global average pooling and linear calculation are then performed on the two confidence maps to obtain the probability that the face belongs to a planar attack and the probability that the face belongs to a real face, respectively, thus obtaining the result of face liveness detection.

[0090] Under the above embodiments, the confidence map generation module of the present invention generates a discriminative confidence map, avoiding the shortcomings of the prior art that rely too much on image datasets, and improving the generalization in real-world scenarios. Based on the trained face liveness recognition model, this model can be used to perform face liveness recognition in various real-world scenarios.

[0091] Figure 8 The illustration shows the effect of the embodiments of this application in an application scenario. By comparing the corresponding positions of pixels in the binocular images to perform depth estimation, a disparity map and a feature-matched image are generated to obtain the face liveness recognition result. This significantly improves the accuracy and stability of face liveness recognition based on binocular images and is applicable to various real-world scenarios.

[0092] Figure 9 The illustrations show several application scenarios of embodiments of this application. This invention is mainly applied in security fields requiring facial liveness recognition, such as human-computer interaction, access control systems, transaction systems, and attendance systems. Examples of its application scenarios are shown in Figures a to c. Figure a primarily illustrates the application of this invention in a smart door lock system. A binocular industrial infrared camera embedded in the door lock captures binocular infrared images of people entering the system. A face detection algorithm is used to identify the person, and then the face liveness recognition method provided by this invention is applied for face liveness recognition. Because this invention is naturally immune to electronic screen attacks, it focuses on identifying planar attacks, preventing fraudulent individuals from using planar attacks to enter core areas in financial security systems. Figure b primarily illustrates the application of this invention in an automated vending machine transaction system. A binocular camera captures faces, and the face liveness recognition method provided by this invention is used for face liveness recognition to prevent fraudulent planar attacks from harming citizens' property. Figure c primarily illustrates the application of this invention in enterprise or school attendance systems. A camera captures video of people, and the face liveness recognition method provided by this invention is used for face liveness recognition to prevent employees or students from using fraudulent planar attacks to undermine the fairness of attendance rules.

[0093] This invention is primarily applied in fields requiring facial recognition, such as human-computer interaction, access control systems, transaction systems, and attendance systems. It is mainly used for authentication in many real-world scenarios, such as mobile payment and access control. The primary consideration in facial liveness detection technology is the security of the facial recognition system. It aims to distinguish whether an input image in a real system represents a genuine face or is a 2D attack. By inputting a single frame of a near-infrared facial image captured by a binocular camera, and then inputting it into the facial liveness detection model provided by this invention, it can output a disparity map and a confidence map, thus obtaining the facial liveness detection result. Specific application scenarios include indoor access control systems, main gate access control systems, company attendance systems, mobile payment systems, embedded payment systems in vending machines and other devices, security systems in banking and finance, public transportation verification systems, and so on.

[0094] The following are embodiments of the apparatus described in this application, which can be used to execute the face liveness recognition method based on binocular images involved in this application. For details not disclosed in the apparatus embodiments of this application, please refer to the method embodiments of the face liveness recognition method based on binocular images involved in this application.

[0095] Please see Figure 10 This application provides a face liveness recognition device 900 based on binocular images. The device 900 includes, but is not limited to: a binocular image acquisition module 910, a disparity estimation module 930, a confidence acquisition module 950, and a result acquisition module 970.

[0096] The binocular image acquisition module 910 is used to acquire binocular images. The binocular images are generated by taking pictures of the face from different angles using a binocular camera. The binocular images include a left eye image and a right eye image.

[0097] The disparity estimation module 930 is used to estimate the disparity of a face using binocular images to obtain a disparity map of the face.

[0098] The confidence acquisition module 950 is used to extract features from the disparity map and the binocular image as a gating component, and obtain the confidence map based on the gating component.

[0099] The result acquisition module 970 is used to perform liveness detection on a face based on the confidence map and obtain the liveness detection result.

[0100] It should be noted that the above embodiments of the face liveness recognition device based on binocular images are only illustrated by the division of the above functional modules when performing face liveness recognition based on binocular images. In actual applications, the above functions can be assigned to different functional modules as needed. That is, the internal structure of the face liveness recognition device based on binocular images will be divided into different functional modules to complete all or part of the functions described above.

[0101] Furthermore, the embodiments of the face liveness recognition device based on binocular images and the face liveness recognition method based on binocular images provided above belong to the same concept. The specific way each module performs its operation has been described in detail in the method embodiments, and will not be repeated here.

[0102] Figure 11 A schematic diagram of the structure of a server is shown according to an exemplary embodiment. This server is suitable for... Figure 2 The server-side 130 in the implementation environment is shown.

[0103] It should be noted that this server is merely an example adapted to this application and should not be construed as providing any limitation on the scope of use of this application. Nor should this server be interpreted as requiring or depending on any specific feature. Figure 11 One or more components of the exemplary server 2000.

[0104] The hardware architecture of Server 2000 can vary significantly due to differences in configuration or performance, such as... Figure 11 As shown, the server 2000 includes: a power supply 210, an interface 230, at least one memory 250, and at least one central processing unit (CPU) 270.

[0105] Specifically, power supply 210 is used to provide operating voltage for the various hardware devices on server 2000.

[0106] Interface 230 includes at least one wired or wireless network interface 231 for interacting with external devices. For example, in Figure 2 The implementation environment is shown, consisting of server 130 and acquisition terminal 110.

[0107] Of course, in other examples adapted in this application, interface 230 may further include at least one serial-to-parallel conversion interface 233, at least one input / output interface 235, and at least one USB interface 237, etc. Figure 11 As shown, this does not constitute a specific limitation.

[0108] The memory 250 serves as a carrier for resource storage and can be a read-only memory, random access memory, disk, or optical disk, etc. The resources stored on it include the operating system 251, application programs 253, and data 255, etc., and the storage method can be temporary storage or permanent storage.

[0109] The operating system 251 is used to manage and control the various hardware devices and application programs 253 on the server 2000, so as to enable the central processing unit 270 to perform calculations and processing on the massive data 255 in the memory 250. It can be Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, etc.

[0110] Application 253 is a computer program that performs at least one specific task based on operating system 251, and may include at least one module ( Figure 11 (Not shown), each module can contain a computer program for the server 2000. For example, a face liveness recognition device based on binocular images can be considered as application 253 deployed on the server 2000.

[0111] Data 255 can be photos, pictures, etc. stored on a disk, or it can be input image data, etc., stored in memory 250.

[0112] The central processing unit 270 may include one or more processors and is configured to communicate with the memory 250 via at least one communication bus to read computer programs stored in the memory 250, thereby performing operations and processing on massive amounts of data 255 stored in the memory 250. For example, a face liveness recognition method based on binocular images can be implemented by the central processing unit 270 reading a series of computer programs stored in the memory 250.

[0113] Furthermore, this application can also be implemented through hardware circuits or a combination of hardware circuits and software. Therefore, the implementation of this application is not limited to any specific hardware circuit, software, or combination thereof.

[0114] Please see Figure 12 This application provides an electronic device 4000, which may include: desktop computer, laptop computer, server, smart camera, drone, etc.

[0115] exist Figure 12 The electronic device 4000 includes at least one processor 4001, at least one communication bus 4002, and at least one memory 4003.

[0116] The processor 4001 and memory 4003 are connected, for example, via a communication bus 4002. Optionally, the electronic device 4000 may also include a transceiver 4004, which can be used for data interaction between the electronic device and other electronic devices, such as sending and / or receiving data. It should be noted that in practical applications, the transceiver 4004 is not limited to one, and the structure of the electronic device 4000 does not constitute a limitation on the embodiments of this application.

[0117] Processor 4001 may be a CPU (Central Processing Unit), a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. It can implement or execute the various exemplary logic blocks, modules, and circuits described in conjunction with the disclosure of this application. Processor 4001 may also be a combination that implements computational functions, such as including one or more microprocessor combinations, a combination of a DSP and a microprocessor, etc.

[0118] The communication bus 4002 may include a path for transmitting information between the aforementioned components. The communication bus 4002 may be a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus, etc. The communication bus 4002 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 12 The bus is represented by a single thick line, but this does not mean that there is only one bus or one type of bus.

[0119] The memory 4003 may be ROM (Read Only Memory) or other types of static storage devices capable of storing static information and instructions, RAM (Random Access Memory) or other types of dynamic storage devices capable of storing information and instructions, or EEPROM (Electrically Erasable Programmable Read Only Memory), CD-ROM (Compact Disc Read Only Memory) or other optical disc storage, optical disc storage (including compressed optical discs, laser discs, optical discs, digital universal optical discs, Blu-ray discs, etc.), magnetic disk storage media or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of instructions or data structures and accessible by a computer, but not limited thereto.

[0120] The memory 4003 stores a computer program, and the processor 4001 reads the computer program stored in the memory 4003 through the communication bus 4002.

[0121] When the computer program is executed by the processor 4001, it implements the face liveness recognition method based on binocular images in the above embodiments.

[0122] Furthermore, this application provides a storage medium storing a computer program that, when executed by a processor, implements the face liveness recognition method based on binocular images in the above embodiments.

[0123] This application provides a computer program product, which includes a computer program stored in a storage medium. A processor of a computer device reads the computer program from the storage medium and executes the computer program, causing the computer device to perform the binocular image-based face liveness recognition method described in the above embodiments.

[0124] Compared with related technologies, the beneficial effects of the present invention are:

[0125] 1. This invention proposes a novel face liveness detection method based on binocular images. By using binocular face images with varying illumination and distances to provide conditions for stereo matching and depth estimation, this invention obtains a pixel-level disparity map using depth estimation, and generates a confidence map to avoid losing facial texture information, thereby achieving accurate recognition. This significantly reduces computational load and improves the stability of face liveness detection, making it suitable for real-world scenarios.

[0126] 2. This invention constructs and releases a large-scale binocular near-infrared (NIR) image dataset, which is the first and, to date, the only binocular near-infrared face image dataset that can be used for single-frame research. The large-scale binocular near-infrared (NIR) image dataset contains more than 300,000 real human face and planar attack images under different environmental conditions.

[0127] 3. This invention achieves interpretable face liveness recognition with limited annotations. We propose an interpretable face liveness recognition model (IFAST) that only requires weak supervision to solve the lack of differential labels in near-infrared images of both eyes in real-world scenarios.

[0128] 4. Compared with existing methods, this invention achieves state-of-the-art performance on large-scale binocular near-infrared (NIR) image datasets, demonstrating the practicality of binocular NIR images in face liveness detection and improving generalization ability in real-world scenarios.

[0129] 5. This invention has low computational complexity and wide applicability. Applicable scenarios include human-computer interaction, monitoring systems, indoor access control systems, main gate access control systems, company attendance systems, mobile payment systems, payment systems for embedded devices such as vending machines, security systems for banks and financial institutions, public transportation verification systems, etc.

[0130] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0131] The above description is merely a preferred exemplary embodiment of the present invention and is not intended to limit the implementation of the present invention. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of protection claimed in the claims.

Claims

1. A face liveness detection method based on binocular images, characterized in that, The method includes: Acquire binocular images; the binocular images are generated by capturing a human face from different angles using a binocular camera, and the binocular images include a left eye image and a right eye image; The disparity of the face is estimated using the binocular images to obtain a disparity map of the face; the disparity map is used to describe the positional differences of the face at different viewpoints. Features from the disparity map and binocular image are extracted as gating components, and feature fusion performed through the gating components yields a confidence map; the confidence map is used to represent the confidence level that the face belongs to a real face or a planar attack. Liveness detection is performed on the face based on the confidence map to obtain the liveness detection result; The step of estimating disparity of the face using binocular images to obtain a disparity map of the face includes: Features of the left and right eye images of the binocular images are extracted to obtain a left feature map and a right feature map; Based on the self-attention and cross-attention of the left and right feature maps, the left and right eye images after feature matching are obtained; Attention weights are obtained based on the similarity of pixels between the left and right eye images after feature matching, and disparity maps of the face are calculated using these attention weights. The step of extracting features from the disparity map and the binocular image as a gating component, and obtaining a confidence map based on the gating component, includes: The first gating branch is used to extract features from the disparity map, and the second gating branch is used to extract features from the left or right eye image of the binocular image. During the feature extraction process, the features extracted by the first gated branch are introduced into the second gated branch and multiplied with the features extracted by the second gated branch to obtain the first image features and the second image features. The first image features and the second image features are fused to obtain a first confidence map and a second confidence map, respectively; the first confidence map is used to represent the confidence that the face belongs to a planar attack; the second confidence map is used to represent the confidence that the face belongs to a real face.

2. The method as described in claim 1, characterized in that, The self-attention and cross-attention based on the left and right feature maps, The left and right eye images obtained after feature matching include: The self-attention of the left and right feature maps is calculated using a first number of attention heads; The cross-attention between the left and right feature maps is calculated using a second number of attention heads; Based on the self-attention and cross-attention, feature matching is performed on the left and right eye images to obtain the feature-matched left and right eye images.

3. The method as described in claim 1, characterized in that, The step of performing liveness detection on the face based on the confidence map to obtain the liveness detection result includes: Global average pooling is performed on the first confidence map and the second confidence map to obtain the average pool of the confidence map; Linear calculations are performed on the average pool of the confidence graph to obtain the first probability that the face belongs to a planar attack and the second probability that the face belongs to a real face. The first probability and the second probability are compared with the set thresholds to obtain the liveness detection result.

4. The method according to any one of claims 1 to 3, characterized in that, The face liveness recognition based on binocular images is implemented by calling a face liveness recognition model, which is a trained machine learning model capable of performing face liveness recognition on the binocular images.

5. The method as described in claim 4, characterized in that, The face liveness recognition model includes a disparity estimation module and a confidence map generator; The training process of the face liveness recognition model includes: Obtain the training set; The training images in the training set are input into the face liveness recognition model for training to obtain the loss value of the disparity estimation module and the loss value of the confidence map generator. The total loss value is obtained based on the loss value of the disparity estimation module and the loss value of the confidence map generator; If the total loss value satisfies the convergence condition, the training is completed and the face liveness recognition model is obtained; otherwise, the model parameters of the machine learning model are updated, and training images from the training set are used to continue training the machine learning model until the total loss value satisfies the convergence condition.

6. The method as described in claim 5, characterized in that, The step of inputting the training images from the training set into the face liveness recognition model for training, and obtaining the loss value of the disparity estimation module and the loss value of the confidence map generator, includes: By performing 3D face alignment on the training images in the training set, a 3D face reconstruction depth map of the training images is obtained. By performing image segmentation on the human figure and background in the training image, a human body region segmentation map of the training image is obtained; Based on the 3D face reconstruction depth map and human body region segmentation map of the training image, the target depth image is obtained; Based on the disparity map, confidence map, and target depth image output by the face liveness recognition model, the loss value of the disparity estimation module and the loss value of the confidence map generator are obtained.

7. A face liveness recognition device based on binocular images, characterized in that, The apparatus for implementing the face liveness recognition method based on binocular images according to any one of claims 1 to 6, the apparatus comprising: The binocular image acquisition module is used to acquire binocular images. Binocular images are generated by taking pictures of a face from different angles using a binocular camera. Binocular images include left and right eye images. The disparity estimation module is used to estimate the disparity of a face using binocular images to obtain a disparity map of the face. The confidence acquisition module is used to extract features from the disparity map and the binocular image as a gating component, and the confidence map is obtained by feature fusion through the gating component. The result acquisition module is used to perform liveness detection on faces based on the confidence map and obtain the liveness detection result.

8. An electronic device, characterized in that, include: At least one processor, at least one memory, and at least one communication bus, wherein, The memory stores a computer program, and the processor reads the computer program from the memory via the communication bus; When the computer program is executed by the processor, it implements the face liveness recognition method based on binocular images as described in any one of claims 1 to 6.

Images