A face region-based live detection method, a terminal device, and a storage medium

Abstract

The application discloses a kind of based on face area's living body detection method, terminal equipment and storage medium, method includes: obtaining the image including face of being handled and identifying face in the image of being handled and obtaining face global image, according to preset face key point coordinate information and preset boundary frame face global image is cut, obtain multiple face local image;Face global image and multiple face local image are input into preset living body detection model and carry out face living body detection, obtain the living body detection result of face.The living body detection model of the application makes full use of the global and local features of face image, can effectively identify the image containing local occlusion, greatly improves the accuracy of living body detection, effectively avoids the attack of fake face.

Description

Technical Field

[0001] This invention relates to the field of liveness detection technology, and in particular to a liveness detection method, terminal device and storage medium based on the face region. Background Technology

[0002] Liveness detection is a method used in identity verification scenarios to determine the true physiological characteristics of an object. In facial recognition applications, liveness detection can use technologies such as facial landmark localization and face tracking to determine whether the captured face is a real face or a fake face attack by using a combination of actions such as blinking, opening the mouth, shaking the head, and nodding. It can effectively resist common attack methods such as photos, videos, face swapping, masks, occlusion, 3D animation, and screen replays, thereby helping users identify fraudulent behavior and protect their interests.

[0003] With the widespread use of facial recognition technology, facial attack techniques are constantly evolving, and the types of attacks are gradually increasing, such as partial facial occlusion, posing numerous challenges to liveness detection technology. However, existing single-modal or multi-modal near-infrared liveness detection methods typically use complete facial images of one or more corresponding modalities directly into the model for training and testing. This approach neglects the utilization of local image features, which are crucial for distinguishing between real people and spoofed images with partial occlusion. Ignoring the use of local image features significantly reduces the accuracy of liveness detection.

[0004] Therefore, existing technologies still need to be improved and enhanced. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a liveness detection method, device, terminal equipment and storage medium based on face region, in order to address the above-mentioned defects of the prior art. The aim is to solve the problem that the prior art ignores the utilization of local image features when performing liveness detection, thus reducing the accuracy of liveness detection.

[0006] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is as follows:

[0007] In a first aspect, the present invention provides a liveness detection method based on a face region, comprising: acquiring an image to be processed including a face; identifying the face in the image to be processed to obtain a global face image; cropping the global face image according to preset face key point coordinate information and preset bounding boxes to obtain multiple local face images; inputting the global face image and the multiple local face images into a preset liveness detection model for face liveness detection to obtain a liveness detection result; wherein the preset liveness detection model includes: a global feature extraction branch, multiple local feature extraction branches, a stitching module, and a binary classification module; wherein the global feature extraction branch and the multiple local feature extraction branches respectively extract features from the global face image and the multiple local face images to obtain global face features and multiple local face features; the stitching module is used to stitch together the global face features and the multiple local face features to obtain stitched features; the binary classification module is used to perform liveness detection inference on the global face features, the multiple local face features, and the stitched features respectively to obtain a liveness detection result.

[0008] Secondly, embodiments of the present invention also provide a liveness detection device based on a face region. The device includes: a face recognition module, used to acquire an image to be processed including a face, and to identify the face in the image to be processed to obtain a global face image; an image local cropping module, used to crop the global face image according to preset face key point coordinate information and preset bounding boxes to obtain multiple local face images; and a face liveness detection module, used to input the global face image and the multiple local face images into a preset liveness detection model to perform face liveness detection and obtain a face liveness detection result; wherein, the preset liveness detection model includes: a global feature extraction branch, multiple local feature extraction branches, a stitching module, and a binary classification module. The system includes a global feature extraction branch and multiple local feature extraction branches that extract features from the global face image and multiple local face images to obtain global face features and multiple local face features, respectively; a stitching module that stitches together the global face features and multiple local face features to obtain stitched features; and a binary classification module that performs liveness detection inference on the global face features, multiple local face features, and stitched features to obtain liveness detection results.

[0009] Thirdly, embodiments of the present invention also provide a terminal device, wherein the terminal device includes a data acquisition device and a processor, wherein the data acquisition device is used to acquire an image to be processed including a face, and the processor processes the image to be processed according to any of the above-described liveness detection methods based on face regions to obtain a liveness detection result.

[0010] Fourthly, embodiments of the present invention also provide a computer-readable storage medium storing a liveness detection program based on a face region. When the liveness detection program based on a face region is executed by a processor, it implements the steps of the liveness detection method based on a face region according to any of the above solutions.

[0011] Beneficial effects: Compared with existing technologies, the images used for face liveness detection in this invention include not only global face images but also local face images, making full use of local facial features and facilitating the differentiation between real faces and spoofed images with partial occlusion. Furthermore, the liveness detection model provided by this invention includes a global feature extraction branch and multiple local feature extraction branches. This liveness detection model can effectively detect local features in face images, thus effectively distinguishing images with partial occlusion, greatly improving the accuracy of liveness detection and effectively preventing spoofed face attacks. Attached Figure Description

[0012] Figure 1 This is a block diagram illustrating the internal structure of the terminal device provided in an embodiment of the present invention.

[0013] Figure 2 A flowchart illustrating a specific implementation of the liveness detection method based on face regions provided in this invention.

[0014] Figure 3 This is a schematic diagram illustrating the determination of coordinate information of key facial points in the liveness detection method based on the face region provided in this embodiment of the invention.

[0015] Figure 4 This is a schematic diagram illustrating the cropping of a global face image in a face region-based liveness detection method provided in an embodiment of the present invention.

[0016] Figure 5 This is a schematic diagram illustrating the principle of the liveness detection model in the liveness detection method based on the face region provided in this embodiment of the invention.

[0017] Figure 6 This is a schematic diagram illustrating the principle of the backbone network in the liveness detection model of the liveness detection method based on face region provided in this embodiment of the invention.

[0018] Figure 7 This is a schematic diagram of the liveness detection device based on the face region provided in the embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and effects of this invention clearer and more explicit, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0020] This embodiment provides a liveness detection method based on face regions. This liveness detection method fully utilizes local facial features, effectively identifying partially occluded facial images and effectively preventing spoofed face attacks. In specific applications, this embodiment first acquires a global face image and identifies facial key points within it. These key points may include the eyes, nose, and mouth. Next, this embodiment can crop the global face image based on the coordinate information of these key points and a preset bounding box, obtaining multiple local face images, namely, local images of the eyes, nose, and mouth. Then, this embodiment inputs both the global face image and the local face images into a preset liveness detection model for face liveness detection, thus obtaining the face liveness detection result. Since the images used in face liveness detection in this invention include not only the global face image but also local face images, this invention fully utilizes local facial features, which is beneficial for distinguishing between real people and spoofed images with partial occlusion. Furthermore, the liveness detection model in this embodiment includes a global feature extraction branch and multiple local feature extraction branches. Therefore, the liveness detection model can effectively detect local features in face images and can effectively distinguish images with partial occlusion, thus greatly improving the accuracy of liveness detection.

[0021] For example, in daily life, we often encounter scenarios that require facial recognition payment. When making a facial recognition payment, the payment terminal performs liveness detection on the captured facial image to ensure the security of the user's account. During liveness detection, the payment terminal first captures a global image of the user's face—a full-face image. Then, the payment terminal identifies the eyes, nose, and mouth in the global facial image to determine facial landmarks. Next, after determining the facial landmarks, the payment terminal further determines their coordinates, i.e., their positions within the global facial image, and then obtains a preset bounding box. Finally, based on the coordinates of the facial landmarks and the preset bounding box, the global facial image is cropped to remove the eyes, nose, and mouth, resulting in multiple partial facial images. The payment terminal inputs a global facial image and multiple partial facial images into a preset liveness detection model to perform liveness detection. The result reflects whether the facial image captured by the payment terminal is a real facial image, thus preventing criminals from using fake facial images to make payments and effectively protecting account security.

[0022] Based on this, the liveness detection method based on face regions provided in this application can be applied to terminal devices. The terminal device in this embodiment can be an intelligent product terminal with image acquisition and image analysis functions, such as a smart door lock or a payment terminal (e.g., a mobile phone). Specifically, as... Figure 1 As shown, the terminal device in this embodiment includes a data acquisition device 100, a processor 101, and a memory 102; wherein, the data acquisition device 100 is used to acquire images including human faces, the processor 101 processes the images according to the liveness detection method provided in one or more embodiments of this application to obtain liveness detection results, and the memory 102 is used to store computer programs 103 running on one or more processors 100, such as the computer programs corresponding to the liveness detection methods provided in one or more embodiments of this application.

[0023] In one embodiment, the acquisition device includes a projection module and an imaging module. The projection module projects an infrared beam onto the user's face, and the imaging module acquires an image containing the face. The projection module and imaging module are connected to a processor to transmit data, and the connection method includes wired and wireless connections. In specific applications, the imaging module can acquire infrared images of the face, and then transmit them to the processor via a network connection for further execution of corresponding tasks by the processor. The imaging module can be placed in the front of the terminal device, such as the front-facing camera on a mobile phone. When the terminal device faces the user's head and the face recognition task is activated, the imaging module can acquire infrared images containing the face. The processor is used to process the infrared images of the face. In this embodiment, the processor can also perform the following tasks: face detection, face segmentation, face liveness detection, and perform related tasks based on the liveness detection results, such as unlocking and payment. The processor can be a single dedicated processor or a combination of multiple processors, and the tasks to be executed are written into the processor in the form of software algorithms for execution.

[0024] In one embodiment, the processor 101 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0025] In one embodiment, memory 102 may be an internal storage unit of the terminal device, such as a hard drive or RAM. Memory 102 may also be an external storage device of the terminal device, such as a plug-in hard drive, smart media card (SMC), secure digital (SD) card, flash card, etc. Furthermore, memory 102 may include both internal and external storage units. Memory 102 is used to store computer programs and other programs and data required by the terminal device. Memory 102 can also be used to temporarily store data that has been output or will be output.

[0026] Those skilled in the art will understand that Figure 1 The block diagram shown is merely a partial structural diagram related to the present invention and does not constitute a limitation on the terminal device to which the present invention is applied. The specific terminal device may include more or fewer components than shown in the figure, or combine certain components, or have different component arrangements.

[0027] Exemplary methods

[0028] Figure 2 A flowchart illustrating a specific implementation of the face region-based liveness detection method provided in this invention. The method includes the following steps:

[0029] Step S100: Obtain the image to be processed, including the face, and identify the face in the image to be processed to obtain a global face image.

[0030] In practical applications, this embodiment first acquires the image of the face to be processed, and then inputs this image into a preset face detection model to obtain a global face image, which is the user's full-face image. It should be noted that this face detection model is pre-trained and is used to identify the face region from the image to be processed, that is, to extract a global face image containing only the face from the image to be processed.

[0031] In one embodiment, the face detection model in this embodiment may be YOLOV3 (the YOLOV3 algorithm model uses a single neural network to act on the image, which can divide the image into multiple regions and predict the bounding box and the probability of each region), Faster RCNN (object detection algorithm model), YOLOV7, etc., and there is no limitation here.

[0032] Step S200: Crop the global face image according to the preset facial key point coordinate information and preset bounding box to obtain multiple local face images.

[0033] In this embodiment, the facial key points are the left eye, right eye, nose tip, left corner of mouth, and right corner of mouth. Therefore, based on the preset facial key point coordinate information and preset bounding boxes, the eye, nose, and mouth portions of the global face image are cropped to obtain the eye, nose, and mouth portions. Specifically, as follows... Figure 3 As shown, Figure 3 The three rectangles in the image are preset bounding boxes, each corresponding to a different facial key point. Figure 2 By using three preset bounding boxes, corresponding partial images of the eyes, nose, and mouth can be cropped out, thus obtaining multiple partial images of the face.

[0034] In one embodiment, the preset facial key point coordinate information is obtained before step S100, and the corresponding data can be directly read when executing step S200. The preset facial key point coordinate information is obtained by: acquiring several face images and identifying the left eye, right eye, nose tip, left corner of mouth, and right corner of mouth in the several face images, and determining the facial key points in each face image; calculating the average coordinate value of the facial key points based on the facial key points in each face image, and obtaining the preset facial key point coordinate information, which is the position of each facial key point in the global face image.

[0035] Furthermore, a perturbation can be added to the mean coordinates of facial landmarks, that is, an offset is added to the original mean coordinates to obtain the coordinate information of preset facial landmarks, thereby increasing the diversity of the data. For example, Figure 4 As shown, in this embodiment, facial key points (left eye, right eye, nose tip, left corner of mouth, and right corner of mouth) are labeled in 100 facial images. The average coordinates of the coordinates of the 100 left eyes, 100 right eyes, 100 nose tips, 100 left corners of mouth, and 100 right corners of mouth are calculated to obtain the average coordinates of the 5 key points. A perturbation is added to the average coordinates of these 5 key points to obtain the preset facial key point coordinate information.

[0036] Similarly, the preset bounding box is also obtained before step S100, and the corresponding data can be directly read when executing step S200. In one embodiment, the preset bounding box is obtained based on the coordinate information of facial key points, specifically including: determining the key point region in each facial image based on the facial key points in each facial image, and calculating the average coordinate value of the outer frame of the key point region to obtain the preset bounding box. For example, based on the facial key points of 100 facial images, the coordinates of the key point regions and the outer frames of these key point regions are obtained, and the average coordinate value of the outer frames of these key point regions is calculated. The position of the average coordinate value is the position of the bounding box.

[0037] It should be noted that the preset facial key point coordinate information and preset bounding box are obtained before step S100 of the face liveness detection method. That is, the facial key points and bounding boxes of each image have been preset. When the liveness detection method is executed, multiple local facial images can be obtained by directly reading the preset facial key point coordinate information and preset bounding box and cropping the corresponding area in the global face image.

[0038] Step S300: Input the global face image and multiple local face images into the preset liveness detection model to perform face liveness detection and obtain the face liveness detection result.

[0039] Specifically, after obtaining the global face image and multiple partial face images, the global face image, the partial eye image, the partial nose image, and the partial mouth image are scaled to the same size, for example, to a 3×112×112 image. Scaled to the same size, it is beneficial to perform simultaneous analysis of the global face image and the partial face images, ensuring that the local facial features are recognized by the liveness detection model. The global face image, the partial eye image, the partial nose image, and the partial mouth image of the same size are then input into the liveness detection model to perform face liveness detection, and the liveness detection result of the face is obtained.

[0040] Figure 5 The diagram illustrates the architecture of a liveness detection model provided in this application. The liveness detection model includes: a global feature extraction branch, multiple local feature extraction branches, a concatenation module, and a binary classification module. The global feature extraction branch and the multiple local feature extraction branches extract features from a global face image and multiple local face images, respectively, to obtain global face features and multiple local face features. The concatenation module (e.g., the `concat` function) concatenates the global face features and multiple local face features to obtain concatenated features. Specifically... Figure 5 As shown, the global and local facial features obtained from the global feature extraction branch and multiple local feature extraction branches are one-dimensional vectors. Concatenating the global facial features and multiple local facial features yields a concatenated feature, which is also a one-dimensional vector. This concatenated feature has rich semantic information, not only preserving the similarities and differences between the global and local facial features, but also enabling the liveness detection model to learn abstract information with relative relationships during training, thereby increasing the network's generalization ability.

[0041] After obtaining the global facial features, multiple local facial features, and stitched features, the binary classification module performs liveness detection inference on the global facial features, multiple local facial features, and stitched features respectively to obtain the inference scores for the global facial features, multiple local facial features, and stitched features. Based on the inference scores for the global facial features, multiple local facial features, and stitched features, the liveness detection result is determined. In this embodiment, when using the liveness detection model for inference, the final classification decision of the model is jointly determined by the outputs of the global feature extraction branch and multiple local feature extraction branches. Furthermore, this model fully utilizes both global and local features of the facial image. Therefore, the overall predictive ability of the liveness detection model is not only improved, but it can also effectively identify images with partial occlusion.

[0042] In one embodiment, the global feature extraction branch and the multiple local feature extraction branches in this embodiment have different network structures. The global feature extraction branch adopts the ResNet model, and the multiple local feature extraction branches adopt any one of the VGG model, Google model, and SqueezeNet model.

[0043] In another implementation, the global feature extraction branch and multiple local feature extraction branches in this embodiment form a backbone network with the same architecture. The backbone network includes multiple feature extraction modules, feature stitching modules, dimensionality reduction modules, and downsampling modules. The feature extraction modules are used to extract facial features from the global face image and the multiple local face images to obtain different feature images. The feature stitching module is used to stitch the different feature images to obtain a stitched image. The dimensionality reduction module is used to perform dimensionality reduction processing on the stitched image to obtain a dimensionality-reduced image. The downsampling module is used to perform downsampling processing on the dimensionality-reduced image to obtain an output image for input into the stitching module of the liveness detection module.

[0044] Figure 6 This is a schematic diagram illustrating the principle of a backbone network according to this application. The backbone network includes a first feature extraction module and a second feature extraction module, which are connected in series. The first feature extraction module includes an upsampling layer and a first upsampling and downsampling layer. The first upsampling and downsampling layer is used to upsampling and downsampling the first feature image input to the first feature extraction module to obtain a second feature image. The first feature image undergoes two upsampling operations, resulting in richer extracted features. While extracting richer features, the size of the feature image is reduced, allowing the network to focus more on learning key features. The second feature extraction module in this embodiment includes a first downsampling layer and a second upsampling and downsampling layer. The first downsampling layer is used to downsampling the second feature image to filter redundant information, and the second upsampling and downsampling layer is used to upsampling the downsampled second feature image to obtain a third feature image.

[0045] The backbone network also includes a third feature extraction module and a fourth feature extraction module. The input of the third feature extraction module is the output of the first feature extraction module, and the input of the fourth feature extraction module is the output of the first downsampling layer in the second feature extraction module. In this embodiment, the third feature extraction module includes a third upsampling and downsampling layer and a fourth upsampling and downsampling layer. These two layers are used to implement upsampling and downsampling processes using different paths than those of the first feature extraction module, respectively, to obtain a fourth feature image. In this embodiment, the fourth feature extraction module includes a second downsampling layer and a fifth upsampling and downsampling layer. The fifth upsampling and downsampling layer is used to implement upsampling and downsampling processes on the downsampled second feature image using different paths than those of the second and third feature extraction modules, to obtain a fifth feature image.

[0046] For neural network models, the more dimensionality increases, the richer the extracted features. Therefore, the second, third, and fourth feature extraction modules in this embodiment produce small images with rich features obtained through different feature extraction paths. During the training of the liveness detection model, the more dimensionality increases, the higher the completeness of the features extracted by the liveness detection module, and the more parameters need to be optimized. This enhances the fitting ability of the liveness detection model, avoids gradient vanishing during training, and improves the accuracy of the liveness detection model.

[0047] The backbone network in this embodiment further includes a feature stitching module, a dimensionality reduction module, and a downsampling module. The feature stitching module stitches together the third, fourth, and fifth feature images to obtain a stitched image. The dimensionality reduction module performs dimensionality reduction processing on the stitched image to obtain a dimensionality-reduced image. The downsampling module performs downsampling processing on the dimensionality-reduced image to obtain an output image, which is then input into the stitching module of the liveness detection module. It should be noted that the inclusion of the stitching, dimensionality reduction, and downsampling modules in the backbone network of this embodiment not only reduces the computational load of the network model but also enables the trained liveness detection model to more effectively extract discriminative true / false face features from global or local face images, significantly improving the accuracy and robustness of the model's classification.

[0048] Therefore, combined Figure 5 and Figure 6 As can be seen, the liveness detection model in this embodiment has a total of 4 branches (one global feature extraction branch and three local feature extraction branches) and 5 outputs (the five outputs are global face features, local eye features, local nose features, local mouth features, and spliced ​​features, respectively). The inference score of each output of the liveness detection model after passing through a binary classification module (such as a fully connected layer) ranges from 0 to 1, and this value represents the probability of a real person.

[0049] In one embodiment, the final liveness detection result of the liveness detection model is jointly determined by the inference scores obtained from each output by the binary classification module. Specifically, this includes: accumulating or weighted accumulating the inference scores obtained from each output by the binary classification module to obtain a comprehensive score, and comparing the comprehensive score with a preset score value to obtain the final liveness detection result. For example, the preset score value can be set to 0.5. If the comprehensive score is greater than or equal to 0.5, the liveness detection result is determined to be a live face; if the comprehensive score is less than 0.5, the liveness detection result is determined to be a fake face. Furthermore, in another implementation, this embodiment can also compare the inference scores of global face features, multiple local face features, and the spliced ​​feature with the preset score value. If the inference scores of global face features, multiple local face features, and the spliced ​​feature are all greater than or equal to 0.5, the liveness detection result is determined to be a live face. If any one of the inference scores of the global facial features, multiple local facial features, and the stitched features is less than a preset score, then the liveness detection result is determined to be a fake face. This embodiment does not limit the two methods for determining the liveness detection result mentioned above; the appropriate method can be selected based on the actual situation in specific applications.

[0050] Furthermore, the liveness detection model is pre-trained, specifically including: acquiring a training set, which includes several face images and corresponding labels for those face images; identifying facial key points in the face images to obtain their coordinate information; obtaining bounding boxes based on the coordinate information of the facial key points and processing the images in the training set according to the bounding boxes to obtain several partial face images and complete face images; scaling the partial face images and complete face images to the same size (e.g., scaling them to 3×112×112 images); inputting the images of the same size into the liveness detection model to be trained to obtain predicted liveness detection results; calculating the training loss value using the predicted liveness detection results and the labels corresponding to the current image; and iteratively optimizing the liveness detection model to be trained based on the training loss value to obtain the optimal liveness detection model.

[0051] In one embodiment, facial key points in several face images are identified to obtain the coordinate information of the facial key points. The bounding box is obtained based on the coordinate information of the facial key points in the same way as the aforementioned method for obtaining the preset facial key point coordinate information and the preset bounding box, and will not be described again here.

[0052] In one embodiment, the liveness detection model to be trained has the same network architecture as the liveness detection model, which includes 5 outputs. Therefore, the training loss value is calculated by using the predicted liveness detection result and the label corresponding to the current image. This includes calculating the loss value for each output based on the predicted liveness detection result and the label corresponding to the current image using the cross-entropy loss function, to constrain network convergence. Specifically, the cross-entropy loss function L... i for:

[0053]

[0054] Where N represents the number of categories, y ik The indicator variable is 1 if the feature category of the image is the same as the preset label of the image, and 0 otherwise; p ik This represents the predicted score for the category K.

[0055] Based on the predicted liveness detection results and their corresponding labels obtained from the outputs of the liveness detection model, the final loss function of the liveness detection model to be trained in this embodiment is: L=L1+L2+L3+L4+L5, where L1~L5 are the loss values ​​calculated by the loss function for the global face features, local eye features, local nose features, local mouth features, and spliced ​​features, respectively, and their corresponding labels.

[0056] In one embodiment, a weighted loss function can also be used. Specifically, the weighted loss function is: L = aL1 + bL2 + cL3 + dL4 + eL5, where a to e are the weights corresponding to global face features, local eye features, local nose features, local mouth features, and spliced ​​features, respectively. That is, when determining the liveness detection result, this embodiment can pre-set the weights corresponding to global face features, local eye features, local nose features, local mouth features, and spliced ​​features, and then perform a weighted summation of the loss values ​​of global face features, multiple local face features, and spliced ​​features to obtain a comprehensive loss value.

[0057] It should be noted that in this embodiment, positive samples can be used to weight the loss function during training, making the model pay more attention to positive samples. During training, when the final loss value of the liveness detection model is less than a preset threshold, it indicates that the network model parameters are optimal, and training can be stopped to obtain the optimal liveness detection model. If the final loss value is greater than or equal to the preset value, the network model parameters are iteratively optimized until the loss value is less than the preset value.

[0058] In summary, the liveness detection model in this embodiment includes a global feature extraction branch and multiple local feature extraction branches. Therefore, this liveness detection model can effectively detect local features in face images, and can effectively distinguish images with partial occlusion. This greatly improves the accuracy of liveness detection and effectively avoids fake face attacks. In practical applications, the liveness detection model in this embodiment achieves a pass rate of over 99.9% for real people and can prevent attacks from high-precision 3D head models and resin masks by over 98%.

[0059] Exemplary device

[0060] Figure 7 This is a schematic diagram of a face region-based liveness detection device according to this application. The device includes a face recognition module 10, an image local cropping module 20, and a face liveness detection module 30, wherein: the face recognition module 10 is used to acquire an image to be processed including a face, and to identify the face in the image to be processed to obtain a global face image; the image local cropping module 20 is used to crop the global face image according to preset face key point coordinate information and preset bounding boxes to obtain multiple face local images; the face liveness detection module 30 is used to input the global face image and multiple face local images into a preset liveness detection model to perform face liveness detection and obtain the face liveness detection result.

[0061] In one embodiment, the preset liveness detection model includes: a global feature extraction branch, multiple local feature extraction branches, a stitching module, and a binary classification module; wherein, the global feature extraction branch and the multiple local feature extraction branches extract features from the global face image and multiple local face images respectively to obtain global face features and multiple local face features; the stitching module is used to stitch the global face features and multiple local face features to obtain stitched features; the binary classification module is used to perform liveness detection inference on the global face features, multiple local face features, and stitched features respectively to obtain liveness detection results.

[0062] It should be noted that the working principle of each functional module of the face region-based liveness detection device in this embodiment is the same as the execution process of each step in the above method embodiment, and will not be repeated here.

[0063] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments of the above methods. Any references to memory, storage, operating databases, or other media used in the embodiments provided by this invention can include non-volatile and / or volatile memory. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), dual operating data rate SDRAM (DDRSDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM), etc.

[0064] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A liveness detection method based on face region, characterized in that, The method includes: A face-containing image to be processed is acquired, and a global face image is obtained by identifying the face in the face-containing image. The face to be processed is an infrared image. The global face image is cropped based on preset facial key point coordinates and preset bounding boxes to obtain multiple partial face images; The global face image and the multiple partial face images are input into a preset liveness detection model to perform face liveness detection, and the face liveness detection result is obtained. The preset liveness detection model includes a global feature extraction branch, multiple local feature extraction branches, a stitching module, and a binary classification module. The global feature extraction branch and the multiple local feature extraction branches extract features from the global face image and the multiple local face images, respectively, to obtain global face features and multiple local face features. The stitching module is used to stitch together the global facial features and the multiple local facial features to obtain stitched features; the binary classification module is used to perform liveness detection inference on the global facial features, the multiple local facial features and the stitched features respectively to obtain the liveness detection result. The global feature extraction branch and the multiple local feature extraction branches are backbone networks with the same or different architectures; When the global feature extraction branch and the multiple local feature extraction branches form a backbone network with the same architecture, the backbone network includes multiple feature extraction modules, feature concatenation modules, dimensionality reduction modules, and downsampling modules; The feature extraction module is used to extract facial features from the global face image and the multiple local face images to obtain different feature images; The plurality of feature extraction modules include a first feature extraction module and a second feature extraction module, which are connected in series. The first feature extraction module includes an upsampling layer and a first upsampling and downsampling layer. The first upsampling and downsampling layer is used to upsampling and downsampling the first feature image input to the first feature extraction module to obtain a second feature image. The second feature extraction module includes a first downsampling layer and a second upsampling and downsampling layer. The first downsampling layer is used to downsample the second feature image to filter redundant information, and the second upsampling and downsampling layer is used to upsample the downsampled second feature image to obtain a third feature image. The plurality of feature extraction modules further include a third feature extraction module and a fourth feature extraction module, wherein the input of the third feature extraction module is the output of the first feature extraction module, and the input of the fourth feature extraction module is the output of the first downsampling layer in the second feature extraction module; The third feature extraction module includes a third upsampling and downsampling layer and a fourth upsampling and downsampling layer. The third upsampling and downsampling layer and the fourth upsampling and downsampling layer are respectively used to implement upsampling and downsampling processing on different paths from the first feature extraction module to obtain the fourth feature image. The fourth feature extraction module includes a second downsampling layer and a fifth upsampling and downsampling layer. The fifth upsampling and downsampling layer is used to perform upsampling and downsampling processing on the downsampled second feature image through a different path than the second feature extraction module and the third feature extraction module, so as to obtain the fifth feature image. The feature stitching module is used to stitch together the third feature image, the fourth feature image, and the fifth feature image to obtain a stitched image; The dimension reduction module is used to perform dimension reduction processing on the stitched image to obtain a dimension-reduced image; The downsampling module is used to downsample the dimensionality-reduced image to obtain an output image, which is then input into the stitching module of the preset liveness detection model.

2. The liveness detection method based on face region according to claim 1, characterized in that, The preset facial key points include the left eye, right eye, tip of the nose, left corner of the mouth, and right corner of the mouth. The coordinate information of the preset facial key points is obtained in advance, and the acquisition method includes: Acquire several face images and identify the left eye, right eye, nose tip, left corner of mouth, and right corner of mouth in several face images, and determine the facial key points in each face image; The average coordinates of the facial key points are calculated based on the facial key points in each facial image to obtain preset facial key point coordinate information, where the coordinate information is the position of each facial key point in the global facial image.

3. The liveness detection method based on face region according to claim 2, characterized in that, The preset bounding box is obtained in advance, and the acquisition method includes: determining the key point region in each face image based on the facial key points in each face image, and calculating the mean coordinate value of the outer box of the key point region to obtain the preset bounding box.

4. The liveness detection method based on face region according to claim 1, characterized in that, The preset facial key points include the left eye, right eye, tip of the nose, left corner of the mouth, and right corner of the mouth. The process involves obtaining a processing image containing a face and identifying the face in the processing image to obtain a global facial image. The global face image is cropped based on preset facial key point coordinates and a preset bounding box to obtain multiple partial face images, including: The process involves acquiring a face image to be processed and extracting a global face image containing only the face from the face image using a preset face detection model. Based on preset facial key point coordinates and preset bounding boxes, the eye, nose, and mouth parts of the global face image are cropped to obtain eye, nose, and mouth partial images.

5. The liveness detection method based on face region according to claim 4, characterized in that, The step of inputting the global face image and the multiple partial face images into a preset liveness detection model to perform face liveness detection and obtain the face liveness detection result includes: The global face image, the partial eye image, the partial nose image, and the partial mouth image are scaled to the same size; A global face image, a partial eye image, a partial nose image, and a partial mouth image of the same size are input into the liveness detection model. Based on the global feature extraction branch, the multiple local feature extraction branches, and the stitching module in the liveness detection model, the global face features, the multiple local face features, and the stitched features are obtained respectively. Based on the binary classification module in the liveness detection model, liveness detection inference is performed on the global face features, the multiple local face features, and the spliced ​​features to obtain the inference scores of the global face features, the multiple local face features, and the spliced ​​features. The liveness detection result is determined based on the inference scores of the global facial features, the inference scores of the multiple local facial features, and the inference score of the spliced ​​features.

6. A face region-based liveness detection device, used to implement the face region-based liveness detection method as described in any one of claims 1-5, characterized in that, The device includes: A face recognition module is used to acquire a to-be-processed image including a face, and to identify the face in the to-be-processed image to obtain a global face image; The image local cropping module is used to crop the global face image according to the preset facial key point coordinate information and the preset bounding box to obtain multiple facial local images; The face liveness detection module is used to input the global face image and the multiple partial face images into a preset liveness detection model to perform face liveness detection and obtain the face liveness detection result. The preset liveness detection model includes a global feature extraction branch, multiple local feature extraction branches, a stitching module, and a binary classification module. The global feature extraction branch and the multiple local feature extraction branches extract features from the global face image and the multiple local face images, respectively, to obtain global face features and multiple local face features. The stitching module is used to stitch together the global facial features and the multiple local facial features to obtain stitched features; the binary classification module is used to perform liveness detection inference on the global facial features, the multiple local facial features and the stitched features respectively to obtain the liveness detection result.

7. A terminal device, characterized in that, The terminal device includes a data acquisition device and a processor. The data acquisition device is used to acquire an image to be processed, including a face. The processor processes the image to be processed according to the liveness detection method based on face region as described in any one of claims 1-5 to obtain a liveness detection result.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a liveness detection program based on a face region. When the liveness detection program based on a face region is executed by a processor, it implements the steps of the liveness detection method based on a face region as described in any one of claims 1-5.

Images