Liveness detection method, electronic device, and storage medium

Abstract

Embodiments of the present application disclose a living body detection method, an electronic device and a storage medium. The method comprises the following steps: after obtaining a first image to be detected, a face region image in the first image is determined, the first image being an image comprising a face region; the first image and the face region image are preprocessed to obtain a second image, the preprocessing comprising: channel dimension splicing of the two images; the second image is input into a first living body detection model for processing to obtain a first living body detection result of the first image.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and in particular to a liveness detection method, electronic device, and storage medium. Background Technology

[0002] With the rapid development of facial recognition technology, it has brought many conveniences to people's lives and work, but it has also posed new challenges to information security. In order to resist spoofing attacks and protect the security of the entire facial recognition system, facial liveness detection technology has emerged.

[0003] Among the related technologies, the main approach is based on neural networks and deep learning methods. It uses real human data and real attack data to form a training set, and trains the network model to have the ability to distinguish between real human faces and attack faces, which can effectively prevent attacks such as screen flipping and mask attacks.

[0004] However, among various attack data, there is a type of masked face attack data. The characteristic of this type of attack data is that, in order to prevent the liveness detection model from recognizing it, the attacker will cover the face area of ​​the attack image with a mask, leaving only the face area, in order to cover up forgery traces such as copying and paper cutting. Since the models of related technologies usually focus more on the face area, and the attack features of masked face attack data in the face area are not very obvious, it is impossible to prevent masked face attack data. Summary of the Invention

[0005] This application provides a liveness detection method, electronic device, and storage medium to address the technical problem that related technologies cannot prevent masked face attack data.

[0006] According to a first aspect of this application, a method for detecting liveness is disclosed, the method comprising:

[0007] After acquiring the first image to be detected, the face region image in the first image is determined, wherein the first image is an image including the face region;

[0008] The first image and the face region image are preprocessed to obtain the second image. The preprocessing includes: stitching the two images together along the channel dimension.

[0009] The second image is input into the first liveness detection model for processing to obtain the first liveness detection result of the first image.

[0010] According to a second aspect of this application, an electronic device is disclosed, including a memory, a processor, and a computer program stored in the memory, wherein the processor executes the computer program to implement the liveness detection method as described in the first aspect.

[0011] According to a third aspect of this application, a computer-readable storage medium is disclosed having a computer program / instructions stored thereon, which, when executed by a processor, implements the liveness detection method as described in the first aspect.

[0012] According to a fourth aspect of this application, a computer program product is disclosed, comprising a computer program / instructions that, when executed by a processor, implement the liveness detection method as described in the first aspect.

[0013] In this embodiment, for the first image requiring liveness detection, the first image and the face region image within it can be preprocessed and then input into the first liveness detection model for processing to obtain the first liveness detection result. Since the first liveness detection model inputs both the entire first image and the face region image simultaneously during liveness detection, it references not only the overall features of the entire first image but also the fine features of the face region. Therefore, it can obtain more obvious, comprehensive, and refined mask attack features. Liveness detection based on these mask attack features can prevent masked face attack data, improving the defense capability against attack data. Attached Figure Description

[0014] Figure 1 This is a flowchart of a liveness detection method provided in an embodiment of this application;

[0015] Figure 2 This is an example diagram of a liveness detection model provided in an embodiment of this application;

[0016] Figure 3 This is one of the example diagrams of image enhancement processing based on gradient operators provided in the embodiments of this application;

[0017] Figure 4 This is the second example diagram of image enhancement processing based on gradient operators provided in the embodiments of this application;

[0018] Figure 5 This is the third example diagram of image enhancement processing based on gradient operators provided in the embodiments of this application;

[0019] Figure 6 This is an example diagram of the liveness detection method provided in the embodiments of this application;

[0020] Figure 7 This is a flowchart of a model training method provided in an embodiment of this application;

[0021] Figure 8 This is a flowchart of a method for generating sample images of mask attacks provided in an embodiment of this application;

[0022] Figure 9 This is an example diagram of the mask generation process provided in the embodiments of this application;

[0023] Figure 10 This is a schematic diagram of the structure of a liveness detection device provided in an embodiment of this application;

[0024] Figure 11 This is a structural block diagram of an electronic device provided in an embodiment of this application. Detailed Implementation

[0025] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0026] It should be noted that, for the sake of simplicity, the method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this application are not limited to the described order of actions, because according to the embodiments of this application, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of this application.

[0027] In recent years, significant progress has been made in research on technologies based on artificial intelligence, such as computer vision, deep learning, machine learning, image processing, and image recognition. Artificial intelligence (AI) is an emerging science and technology that studies and develops theories, methods, technologies, and application systems to simulate and extend human intelligence. AI is a comprehensive discipline involving numerous technologies, including chips, big data, cloud computing, the Internet of Things, distributed storage, deep learning, machine learning, and neural networks. Computer vision, as an important branch of AI, specifically enables machines to recognize the world. Computer vision technologies typically include face recognition, liveness detection, fingerprint recognition and anti-counterfeiting verification, biometric recognition, face detection, pedestrian detection, object detection, image processing, image recognition, image semantic understanding, image retrieval, text recognition, video processing, video content recognition, 3D reconstruction, virtual reality, augmented reality, simultaneous localization and mapping (SLAM), computational photography, and robot navigation and localization. With the research and advancement of artificial intelligence technology, this technology has been applied in numerous fields, such as security, urban management, traffic management, building management, park management, facial recognition access control, facial recognition attendance, logistics management, warehouse management, robotics, intelligent marketing, computational photography, mobile imaging, cloud services, smart homes, wearable devices, autonomous driving, autonomous driving, smart healthcare, facial payment, facial unlocking, fingerprint unlocking, identity verification, smart screens, smart TVs, cameras, mobile internet, live streaming, beautification, makeup, medical aesthetics, and intelligent temperature measurement.

[0028] It should be noted that the data obtained in this application was accessed, collected, stored, and used for subsequent analysis and processing after the user or relevant data owner was clearly informed of the content of the data collection, the purpose of the data, the processing method, etc., and with the consent and authorization of the user or relevant data owner. Furthermore, the application can provide the user or relevant data owner with the means to access, correct, or delete the data, as well as the method to revoke consent or authorization.

[0029] This application provides a liveness detection method, an electronic device, and a storage medium. The liveness detection method provided by this application will be described below with reference to the accompanying drawings.

[0030] It should be noted that the liveness detection method provided in this application is applicable to electronic devices. In practical applications, these electronic devices include, but are not limited to, mobile terminals, attendance machines, identity verification devices, smartphones, tablets, ATMs, payment devices, and backend servers. For example, when the electronic device is a smartphone or tablet, the liveness detection method can be used for unlocking scenarios; when the electronic device is an ATM, the liveness detection method can be applied to deposit and withdrawal scenarios; and when the electronic device is a payment device, the liveness detection method can be used for payment scenarios.

[0031] Figure 1 This is a flowchart of a liveness detection method provided in an embodiment of this application, such as... Figure 1 As shown, the method may include the following steps: step 101, step 102 and step 103;

[0032] In step 101, after acquiring the first image to be detected, the face region image in the first image is determined, wherein the first image is an image that includes the face region.

[0033] In this embodiment of the application, taking into account the characteristic that mask attack data only retains the face region, a first liveness detection model is designed by combining traditional computer vision technology with neural network and deep learning technology to ensure accurate and efficient prevention of mask data attacks.

[0034] In this embodiment, considering that only the full image is used, although the information in the full image is complete, if the full image does not meet the model's size requirements for the input image, it may be necessary to adjust the size of the full image to the size required by the model. Since size adjustment will cause proportional distortion of the image, the face in the image may appear anywhere in the image space, increasing the learning difficulty for the network. Therefore, while using the full image, the face region image in the full image is also used. This ensures that the network focuses more on the face and its surrounding information closely related to the liveness detection task through this part of the input. This can be understood as performing attention processing on the model's input. Therefore, after obtaining the first image to be detected, it is also necessary to determine the face region image in the first image.

[0035] In this embodiment of the application, any object detection method in the related technology can be used to detect the face position in the first image, and then the face region image in the first image can be determined based on the detected face position information.

[0036] In step 102, the first image and the face region image are preprocessed to obtain the second image. The preprocessing includes: stitching the two images together along the channel dimension.

[0037] In this embodiment, the existing liveness detection model can be improved to obtain the first liveness detection model in this embodiment.

[0038] In related technologies, the input of the liveness detection model is an RGB image with dimensions (3, W, H), where 3 represents the number of channels of the image, W represents the width of the image, and H represents the height of the image. (3, W, H) represents a three-channel RGB image.

[0039] The input dimension of the first liveness detection model in this embodiment is (6,W,H). (6,W,H) is formed by stitching together two images with dimensions (3,W,H) in the channel dimension. (6,W,H) represents a six-channel image.

[0040] In step 103, the second image is input into the first liveness detection model for processing to obtain the first liveness detection result of the first image.

[0041] In this embodiment of the application, the first liveness detection result includes: real person or mask attack.

[0042] As can be seen from the above embodiments, in this embodiment, for the first image that needs to be detected for liveness detection, the first image and the face region image in the first image can be preprocessed and then input into the first liveness detection model for processing to obtain the first liveness detection result. Since the entire first image and the face region image are input simultaneously when using the first liveness detection model for liveness detection, the first liveness detection model, in addition to referring to the overall features of the entire first image, also refers to the fine features of the face region in the first image during processing. Therefore, it can obtain more obvious, comprehensive, and refined mask attack features. Based on the above mask attack features, liveness detection can be performed to prevent mask face attack data and improve the defense capability against attack data.

[0043] In another embodiment provided in this application, the network structure of the first liveness detection model used in this embodiment is described.

[0044] In this embodiment, the existing liveness detection model can be improved to obtain the first liveness detection model in this embodiment. The differences between the first liveness detection model in this embodiment and the liveness detection model in related technologies are mainly reflected in the following three aspects: network input, network structure and network intermediate processing.

[0045] I. Network Input:

[0046] In related technologies, the input of a liveness detection model is an RGB image with dimensions (B, 3, W, H), where B represents the model's batch size, 3 represents the number of channels in the image, W represents the image width, and H represents the image height. (3, W, H) represents a three-channel RGB image.

[0047] The input dimension of the first liveness detection model in this embodiment is (B, 6, W, H). (6, W, H) is formed by stitching together two images with dimensions (3, W, H) in the channel dimension. (6, W, H) represents a six-channel image.

[0048] II. Network Structure:

[0049] The liveness detection model in related technologies mainly includes a feature extraction network and a classification network. The feature extraction network is used to extract features from the input image, and the classification network is used to classify based on the output features of the feature extraction network to obtain the liveness detection result.

[0050] The first liveness detection model in the embodiments of this application, such as Figure 2 As shown, it may include: a feature extraction network, a first classification network, a second classification network, and a third classification network; wherein, the first and second classification networks are both connected to the feature extraction network and are located after it; the first and second classification networks are both connected to the third classification network and are located before it; the feature extraction network is used to extract target features from the second image; the first classification network is used to generate a first classification result based on the features corresponding to the first image in the target features; the second classification network is used to generate a second classification result based on the features corresponding to the face region image in the target features; and the third classification network is used to generate a first liveness detection result based on the second and third classification results.

[0051] III. Intermediate processing steps in the network:

[0052] The processing steps of the liveness detection model in related technologies are as follows: the feature extraction network extracts a feature map with n channels from the input image, the classification network performs classification processing based on the feature map with n channels, and outputs a liveness detection result with dimension (B,1).

[0053] The processing procedure of the first liveness detection model in this embodiment is as follows: the feature extraction network extracts a feature map with n channels from the input image, and divides the feature map with n channels into two feature maps with n / 2 channels each. The two feature maps with n / 2 channels correspond to the input full image and the face region image of the full image, respectively. The first classification network performs classification processing based on one feature map with n / 2 channels to obtain a first classification result with dimension (B,1). The second classification network performs classification processing based on the other feature map with n / 2 channels to obtain a second classification result with dimension (B,1). The first classification result and the second classification result are concatenated into a result with dimension (B,2), which is then input into the third classification network for processing to obtain a first liveness detection result with dimension (B,1).

[0054] In some embodiments of this application, the first classification network may include: a first pooling layer and a first fully connected layer, wherein the first pooling layer is connected to the feature extraction network and the first fully connected layer is connected to the first pooling layer; the second classification network may include: a second pooling layer and a second fully connected layer, wherein the second pooling layer is connected to the feature extraction network and the second fully connected layer is connected to the second pooling layer; the third classification network may include: a third fully connected layer.

[0055] In this embodiment of the application, both the first classification network and the second classification network include pooling layers and fully connected layers. The pooling layer can be used to reduce the dimensionality of the input features to reduce the amount of data processing and improve the processing speed; the fully connected layer is used to classify based on the dimensionality-reduced features.

[0056] In some embodiments of this application, considering that models typically have a certain bias towards the size of input images—for example, the model processes images that meet the size requirements better; while it can process images that do not meet the size requirements, the processing effect is generally poor or even negligible—the image size can be adjusted to the size required by the first liveness detection model before inputting the image into the model. Accordingly, the preprocessing described above may further include: adjusting the size of the two images to a target size before concatenating them along the channel dimension; wherein the target size refers to the size required by the first liveness detection model.

[0057] In some embodiments of this application, considering that for image classification algorithms trained on neural networks, there are two types of images (real-person images and masked images) that need to be classified, preprocessing them into a form with more obvious visual features before inputting them into the network would greatly help the network classification. Accordingly, the above preprocessing may also include: enhancing the two images before concatenating them along the channel dimension to make the image features of the input image more obvious, thereby benefiting the model processing.

[0058] In this embodiment of the application, considering that when the human eye distinguishes whether there is a mask on an image, the vision often notices the difference in sharpness between the normal image of the face and the blurred image of the mask, this difference can be defined as a gradient in the features of the digital image. Therefore, the two images can be enhanced based on the gradient operator, wherein the gradient operator can include any of the following: Scharr operator, Sobel operator and Laplacian operator.

[0059] The following is a brief introduction to the three gradient operators:

[0060] Sobel operator: The Sobel operator can calculate image gradients, which is used to extract boundaries. Taking a 3×3 convolution kernel to calculate the Sobel operator as an example... Figure 3 Gx in the diagram represents the convolution kernel used to calculate the horizontal gradient. Simply put, it's the difference between the right and left columns of the target pixel, with the weights determined by the kernel. When the difference between the right and left columns of the target pixel P5 is particularly large, the gradient value of that pixel will be very large, indicating that this point is a boundary; similarly... Figure 3 The Gy in the diagram represents the convolution kernel used to calculate the gradient in the vertical direction. A large difference in the vertical direction indicates the presence of a boundary. Taking pixel P5 as an example, the gradient of P5 in the x-direction is denoted as P5x, P5x = (P3-P1) + 2*(P6-P4) + (P9-P7); the gradient of P5 in the y-direction is denoted as P5y, P5y = (P7-P1) + 2*(P8-P2) + (P9-P3).

[0061] It should be noted that the horizontal gradient operation extracts the vertical boundary, and the vertical gradient operation obtains the horizontal boundary. In OpenCV, the pixel value is truncated by default, that is, less than 0 is counted as 0, and greater than 255 is counted as 255. The image calculated in this way may lose information. Therefore, in this embodiment, it is modified to: for values ​​less than 0, the absolute value is taken, and values ​​greater than 255 can be counted as 255 (i.e., the maximum range is taken).

[0062] Scharr operator: The Scharr operator shares the same concept as the Sobel operator, such as... Figure 4 As shown, Figure 4 In the diagram, Gx represents the convolution kernel used to calculate the horizontal gradient, and Gy represents the convolution kernel used to calculate the vertical gradient. The only difference is the coefficients of the convolution kernels. The Scharr operator is also more sensitive to boundary extraction and can extract finer boundaries. Taking pixel P5 as an example, the gradient of P5 in the x-direction is denoted as P5x, P5x = 3*(P3-P1) + 10*(P6-P4) + 3*(P9-P7); the gradient of P5 in the y-direction is denoted as P5y, P5y = 3*(P7-P1) + 10*(P8-P2) + 3*(P9-P3).

[0063] Laplacian operator: The Laplacian operator is also used to calculate image gradients, and its function is also to extract boundaries. The convolution kernel is set as follows... Figure 5As shown. The Laplacian operator differs from the two operators mentioned above in that the Sobel and Scharr operators typically calculate a horizontal gradient first, then a vertical gradient, and then fuse the two results with a weight of 0.5 to obtain a complete boundary. However, the Laplacian operator is different; it is a second-order operator. Its operation rule is to perform two horizontal and two vertical operations, and then superimpose the two results to replace the pixel value (grayscale value) of the center point. The Laplacian operator is similar to the second-order Sobel operator; in fact, in OpenCV, the Laplacian operator is calculated by calling the Sobel operator. Taking pixel P5 as an example, the gradient of P5 is denoted as P5. L P5 L = (P2+P4+P6+P8)-4*P5.

[0064] In practical applications, the more detailed Scharr operator can be preferred.

[0065] In this embodiment, the second image is processed by the feature extraction network, the first classification network, the second classification network and the third classification network of the first liveness detection model to obtain the first liveness detection result of the first image.

[0066] from Figure 2 As can be seen from the above, the network structure of the first liveness detection model in this embodiment contains two branches, so the first liveness detection model can be called a dual-stream input network model.

[0067] In this embodiment, a feature extraction network receives a second image and extracts target features from the second image; a first classification network generates a first classification result based on the features corresponding to the first image in the target features; a second classification network generates a second classification result based on the features corresponding to the face region image in the target features; and a third classification network generates a liveness detection result based on the second and third classification results. The first classification result is a classification result based on the full-image features of the first image, and the second classification result is a classification result based on the face region features of the first image; the first liveness detection result is a detection result combining the second and third classification results, and the first liveness detection result is used to indicate whether the first image belongs to a real person image or a masked attack image.

[0068] In one example, after acquiring the first image, the Scharr operator is first used to enhance the first image, resulting in an enhanced image, such as... Figure 6As shown, the enhanced image is image 60. Then, the entire image of image 60 is resized to obtain image 61, and the face region image in image 60 is resized to obtain image 62. The dimensions of image 61 are (B,3,W,H), and the dimensions of image 62 are also (B,3,W,H). Images 61 and 62 are concatenated along their channel dimensions to obtain a second image with dimensions (B, 6, W, H). This second image is then input into a feature extraction network for feature extraction, resulting in a feature map with n channels. This feature map with n channels is then divided into two feature maps, each with n / 2 channels. These two feature maps correspond to the input first image and the face region image of the first image, respectively. The first pooling layer in the first classification network performs dimensionality reduction on one feature map with n / 2 channels, and then inputs it into the first fully connected layer for classification, resulting in a first classification result with dimension (B, 1). Similarly, the second pooling layer in the second classification network performs dimensionality reduction on the other feature map with n / 2 channels, and then inputs it into the second fully connected layer for classification, resulting in a second classification result with dimension (B, 1). The first and second classification results are concatenated into a result with dimension (B, 2), which is then input into the third classification network for processing, resulting in a first liveness detection result with dimension (B, 1).

[0069] As can be seen, in this embodiment, for the first image requiring liveness detection, the first image and the face region image within it can be preprocessed before being input into the first liveness detection model for processing to obtain the first liveness detection result. Since the first liveness detection model references not only the overall features of the entire image during training but also the fine features of the face within the image, its feature extraction network can learn more obvious, comprehensive, and refined masking attack features. The various classification networks of the liveness detection model, based on the features extracted by the feature extraction network, cooperate to obtain the final first liveness detection result, thus preventing masked face attack data and improving the defense capability against attack data.

[0070] Figure 7 This is a flowchart of a model training method provided in an embodiment of this application. This method is used for training... Figure 1 The first liveness detection model in the illustrated embodiment, such as Figure 7 As shown, the method may include the following steps: step 701, step 702, step 703 and step 704;

[0071] In step 701, a training set is obtained, which includes: multiple sample images and annotation information for each sample image. The annotation information includes: category information and face location information. The sample images are divided into real person sample images and mask attack sample images.

[0072] In this embodiment of the application, in order to ensure that the trained model has a strong defense capability against mask attacks, the training set may include a massive number of sample images.

[0073] In some embodiments, the sample images and corresponding annotation information in the training set can be manually annotated datasets.

[0074] In this embodiment of the application, face location information is used to determine the location of the face region in the sample image. The face location information can be the position coordinates of the face region in the image.

[0075] In this embodiment of the application, the category information is used to indicate whether the sample image belongs to a real person image or a mask attack image. For example, when the sample image belongs to a real person image, the category label information is 1; when the sample image belongs to a mask attack image, the category label information is 0.

[0076] In some embodiments, considering the difficulty of obtaining a large amount of attack data to construct a training set when a large number of masked attack sample images are required, masked attack sample images can be automatically synthesized. The generation process can be found in [reference needed]. Figure 8 The example shown.

[0077] In step 702, the network to be trained and its corresponding loss function are constructed.

[0078] In this embodiment, the network structure of the network to be trained is the same as that of the first liveness detection model, including: a feature extraction network, a first classification network, a second classification network, and a third classification network. The loss function in this embodiment can be the same as the loss function of a liveness detection model in related technologies.

[0079] In step 703, based on the face location information in the annotation information, the face region image in the sample image is determined, and the sample image and the corresponding face region image are preprocessed.

[0080] In this embodiment of the application, the preset processing may include: first enhancing the two images, then adjusting the size of the two images to the target size, and finally stitching the two images of the target size together by channel dimension.

[0081] In step 704, the network to be trained is trained iteratively multiple times based on the preprocessed image. During each iteration, the parameters in the network to be trained are updated by backpropagation based on the prediction information, category information in the annotation information, and loss function output by the network to be trained. The above iterative process is repeated until the model converges, and the first liveness detection model is obtained.

[0082] As can be seen from the above embodiments, in this embodiment, since the first liveness detection model refers not only to the overall features of the entire image but also to the fine features of the face region in the image during the training process, the feature extraction network of the first liveness detection model can learn more obvious, comprehensive and fine mask attack features. The various classification networks of the first liveness detection model cooperate with each other based on the features extracted by the feature extraction network to obtain the final first liveness detection result, which can prevent mask face attack data and improve the defense capability against attack data.

[0083] In related technologies, the amount of real attack data is limited, and it is difficult to obtain a large amount of attack data to construct a training set. In order to solve the above technical problems, this application provides a method for generating masked attack sample images.

[0084] Figure 8 This is a flowchart of a method for generating sample images for mask attacks provided in an embodiment of this application, such as... Figure 8 As shown, the process of synthesizing a mask attack sample image may include the following steps: step 801, step 802, step 803 and step 804;

[0085] In step 801, a mask is generated based on a standard set of facial key points, wherein the standard set of facial key points includes multiple facial key points.

[0086] In this embodiment of the application, some masks can be generated based on the actual mask data attack situation.

[0087] In some embodiments of this application, step 801 may include the following steps: step 8011 and step 8012;

[0088] In step 8011, a facial contour is generated on the background image based on a standard set of facial key points.

[0089] In this embodiment of the application, the standard facial landmark set may include a series of standard dense facial landmarks, such as 68 facial landmarks or 81 facial landmarks.

[0090] In step 8012, a foreground region is generated on the background image to obtain a mask based on the position of the face contour; wherein, the overlap area between the foreground region and the face contour is greater than the target threshold; when the pixel value of the background image is 255, the pixel value of the foreground region is 0; when the pixel value of the background image is 0, the pixel value of the foreground region is 255.

[0091] In this embodiment, when the background image is a black image, the foreground area is a white image area; when the background image is a white image, the foreground area is a black image area.

[0092] In this embodiment, the shape of the foreground region can be circular, elliptical, rectangular, etc. Considering the shape of the human face, the shape of the foreground region is preferably elliptical.

[0093] In one example, a series of standard dense facial landmarks can be obtained first, including but not limited to 68 landmarks and 81 landmarks, such as... Figure 9 As shown, place the standard facial key points at arbitrary positions on a black background image with pixel values ​​of 0. Then, draw a white ellipse with pixel values ​​of 255 based on its position. The ratio of the major and minor axes of the ellipse, its size, and its position relative to the face should have a certain degree of randomness. Construct multiple masks in this way, for example, four.

[0094] In step 802, the mask is aligned with the position of the target face in the reference image to obtain the mask.

[0095] In this embodiment, all pre-prepared masks have a common standard facial key point. The reference image will also output the same number of facial key points according to the facial key point detection algorithm. Using the key points of the target face in the reference image as a template, the mask ellipse is aligned to the position of the target face to ensure that the spatial position of the mask ellipse and the target face is consistent, thus obtaining the aligned mask.

[0096] In this embodiment of the application, the reference image may be derived from a database, which contains multiple real images containing faces.

[0097] In step 803, the reference image is blurred to obtain a blurred image.

[0098] In this embodiment of the application, considering that the actual mask data will have a certain gradual transition around the face, in order to achieve this effect, the reference image can be processed by multi-round multi-scale Gaussian blur. For example, a random seed is designed to perform Gaussian blur of random number of times, and the Gaussian kernel for each blur is also randomly selected within a certain range (in the live face scene, in order to more closely approximate the effect of the real mask attack data, the Gaussian kernel is usually selected as a positive odd number greater than 90) to obtain the blurred image.

[0099] In step 804, the reference image and the blurred image are fused based on the mask to obtain the mask attack sample image.

[0100] In this embodiment of the application, based on a face-aligned mask, a weighted sum is calculated on the reference image and its blurred image to obtain the final mask attack sample image.

[0101] For example, when the background image is a black image and the foreground region is a white elliptical region, the weighted summation rule is: output = img * mask + blur_img * (1 - mask); where output represents the pixel value of the pixel in the mask attack sample image, img represents the pixel value of the corresponding pixel in the base image, blur_img represents the pixel value of the corresponding pixel in the blurred image, and mask represents the pixel value of the corresponding pixel in the mask.

[0102] In one example, the face in the reference image is first aligned with the mask to obtain the mask. The reference image is then blurred to obtain the blurred image. Based on the mask, the reference image and the blurred image are weighted and summed to obtain the mask attack sample image.

[0103] As can be seen, in response to the problem of not being able to obtain a large amount of attack data, this embodiment of the application can generate a large number of robust masked attack sample images with high matching degree with real attacks based on the characteristics of masked attack data. These images, together with real human images, form a training set to train the network model. Since the training data has a high degree of diversity, this defense capability can remain at a high level for a long period of time and will not decline rapidly due to slight changes in the attack method. Thus, it achieves the task of active defense against masked face attacks without relying on massive amounts of real attack data.

[0104] In another embodiment provided in this application, the first liveness detection model can also be integrated in parallel with the second liveness detection model in related technologies during deployment. Correspondingly, the provided liveness detection method can also be used in... Figure 1 Based on the illustrated embodiment, the following steps are added:

[0105] The first image is input into the second liveness detection model for processing to obtain the second liveness detection result of the first image;

[0106] Based on the first and second liveness detection results, the target liveness detection result is generated.

[0107] In this embodiment, the second liveness detection model can be a model used to detect other attack types besides mask attacks (such as screen flip attacks, mask attacks, etc.). By combining the detection results of the first liveness detection model and the detection results of the second liveness detection model, the final target liveness detection result is obtained, which can cover various liveness attack types and improve the defense capability against liveness attacks.

[0108] Figure 10 This is a schematic diagram of the structure of a liveness detection device provided in an embodiment of this application, as shown below. Figure 10As shown, the liveness detection device 1000 may include: a determination module 1001, a first processing module 1002, and a second processing module 1003;

[0109] The determining module 1001 is used to determine the face region image in the first image after acquiring the first image to be detected, wherein the first image is an image including the face region;

[0110] The first processing module 1002 is used to preprocess the first image and the face region image to obtain a second image. The preprocessing includes: stitching the two images together along the channel dimension.

[0111] The second processing module 1003 is used to input the second image into the first liveness detection model for processing, and obtain the first liveness detection result of the first image.

[0112] As can be seen from the above embodiments, in this embodiment, for the first image that needs to be detected for liveness detection, the first image and the face region image in the first image can be preprocessed and then input into the first liveness detection model for processing to obtain the first liveness detection result. Since the entire first image and the face region image are input simultaneously when using the first liveness detection model for liveness detection, the first liveness detection model, in addition to referring to the overall features of the entire first image, also refers to the fine features of the face region in the first image during processing. Therefore, it can obtain more obvious, comprehensive, and refined mask attack features. Based on the above mask attack features, liveness detection can be performed to prevent mask face attack data and improve the defense capability against attack data.

[0113] Optionally, as an embodiment, the preprocessing may further include: adjusting the size of the two images to the target size before stitching the two images together by channel dimension.

[0114] Optionally, as an embodiment, the preprocessing may further include: enhancing the two images based on gradient operators before concatenating them along the channel dimension, wherein the gradient operators include any one of the following: Scharr operator, Sobel operator, and Laplacian operator.

[0115] Optionally, as an embodiment, the first liveness detection model may include: a feature extraction network, a first classification network, a second classification network, and a third classification network; the first classification network and the second classification network are both connected to the feature extraction network and are located after it; the first classification network and the second classification network are both connected to the third classification network and are located before it;

[0116] The feature extraction network is used to extract target features from the second image;

[0117] The first classification network is used to generate a first classification result based on the features corresponding to the first image in the target features;

[0118] The second classification network is used to generate a second classification result based on the features corresponding to the face region image in the target features;

[0119] The third classification network is used to generate a first liveness detection result based on the second classification result and the third classification result.

[0120] Optionally, as an embodiment, the first classification network may include: a first pooling layer and a first fully connected layer, wherein the first pooling layer is connected to the feature extraction network, and the first fully connected layer is connected to the first pooling layer;

[0121] The second classification network includes: a second pooling layer and a second fully connected layer, wherein the second pooling layer is connected to the feature extraction network, and the second fully connected layer is connected to the second pooling layer;

[0122] The third classification network includes: a third fully connected layer.

[0123] Optionally, as an embodiment, the liveness detection device 1000 may further include:

[0124] The third processing module is used to input the first image into the second liveness detection model for processing, and obtain the second liveness detection result of the first image;

[0125] The first generation module is used to generate a target liveness detection result based on the first liveness detection result and the second liveness detection result.

[0126] Optionally, as an embodiment, the first liveness detection model is trained based on real human sample images and synthetic mask attack sample images;

[0127] The liveness detection device 1000 may further include: a second generation module;

[0128] The second generation module may include:

[0129] A generation submodule is used to generate a mask based on a standard set of facial key points, wherein the standard set of facial key points includes multiple facial key points;

[0130] The alignment submodule is used to align the mask to the position of the target face in the reference image to obtain the mask.

[0131] The blurring submodule is used to blur the reference image to obtain a blurred image.

[0132] The fusion submodule is used to fuse the reference image and the blurred image based on the mask to obtain a mask attack sample image.

[0133] Optionally, as an embodiment, the generation submodule may include:

[0134] The first generation unit is used to generate a face outline on a background image based on a standard set of facial key points.

[0135] The second generation unit is used to generate a foreground region mask on the background image based on the position of the face contour.

[0136] Wherein, the overlap area between the foreground region and the face contour is greater than the target threshold; when the pixel value of the background image is 255, the pixel value of the foreground region is 0; when the pixel value of the background image is 0, the pixel value of the foreground region is 255.

[0137] Any step and specific operation in any step of the liveness detection method provided in this application can be completed by the corresponding module in the liveness detection device. The process of the corresponding operation completed by each module in the liveness detection device is referred to the process of the corresponding operation described in the embodiment of the liveness detection method.

[0138] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0139] Figure 11 This is a structural block diagram of an electronic device provided in an embodiment of this application. The electronic device includes a processing component 1122, which further includes one or more processors, and memory resources represented by a memory 1132 for storing instructions executable by the processing component 1122, such as application programs. The application programs stored in the memory 1132 may include one or more modules, each corresponding to a set of instructions. Furthermore, the processing component 1122 is configured to execute instructions to perform the methods described above.

[0140] The electronic device may also include a power supply component 1126 configured to perform power management of the electronic device, a wired or wireless network interface 1150 configured to connect the electronic device to a network, and an input / output (I / O) interface 1158. The electronic device may operate on an operating system stored in memory 1132, such as Windows Server™, MacOS X™, Unix™, Linux™, FreeBSD™, or similar.

[0141] According to yet another embodiment of this application, this application also provides a computer-readable storage medium having a computer program / instructions stored thereon, which, when executed by a processor, implements the steps in the liveness detection method as described in any of the above embodiments.

[0142] According to another embodiment of this application, this application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the steps in the liveness detection method as described in any of the above embodiments.

[0143] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0144] Those skilled in the art will understand that embodiments of this application can be provided as methods, apparatus, or computer program products. Therefore, embodiments of this application can take the form of entirely hardware embodiments, entirely software embodiments, or embodiments combining software and hardware aspects. Furthermore, embodiments of this application can take the form of computer program products implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0145] This application describes embodiments with reference to flowchart illustrations and / or block diagrams of methods, terminal devices (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing terminal device to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing terminal device, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0146] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing terminal device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0147] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0148] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0149] The present application provides a detailed description of a liveness detection method, electronic device, and storage medium. Specific examples have been used to illustrate the principles and implementation methods of the present application. The descriptions of the above embodiments are only for the purpose of helping to understand the method and its core ideas. At the same time, those skilled in the art will recognize that there will be changes in the specific implementation methods and application scope based on the ideas of the present application. Therefore, the content of this specification should not be construed as a limitation of the present application.

Claims

1. A method for detecting liveness, characterized in that, The method includes: After acquiring the first image to be detected, the face region image in the first image is determined. The first image is an image that includes the face region. The Scharr operator is used to enhance the first image to obtain the enhanced image. The first image and the face region image are preprocessed to obtain the second image. The preprocessing includes: stitching the two images together along the channel dimension. The second image is input into the first liveness detection model for processing to obtain the first liveness detection result of the first image; The first liveness detection model was trained based on real human sample images and synthetic mask attack sample images; The mask attack sample image is synthesized through the following process: A mask is generated based on a standard set of facial key points, wherein the standard set of facial key points includes multiple facial key points; Align the mask to the position of the target face in the reference image to obtain the mask film; The reference image is blurred to obtain a blurred image; Based on the mask, the reference image and the blurred image are fused to obtain the mask attack sample image.

2. The method according to claim 1, characterized in that, The preprocessing also includes: adjusting the size of the two images to the target size before stitching them together by channel dimension.

3. The method according to claim 1, characterized in that, The preprocessing further includes: enhancing the two images based on gradient operators before concatenating them along the channel dimension, wherein the gradient operators include any one of the following: Scharr operator, Sobel operator, and Laplacian operator.

4. The method according to any one of claims 1-3, characterized in that, The first liveness detection model includes: a feature extraction network, a first classification network, a second classification network, and a third classification network; the first classification network and the second classification network are both connected to the feature extraction network and are located after it; the first classification network and the second classification network are both connected to the third classification network and are located before it. The feature extraction network is used to extract target features from the second image; The first classification network is used to generate a first classification result based on the features corresponding to the first image in the target features; The second classification network is used to generate a second classification result based on the features corresponding to the face region image in the target features; The third classification network is used to generate a first liveness detection result based on the first classification result and the second classification result.

5. The method according to claim 4, characterized in that, The first classification network includes: a first pooling layer and a first fully connected layer, wherein the first pooling layer is connected to the feature extraction network, and the first fully connected layer is connected to the first pooling layer; The second classification network includes: a second pooling layer and a second fully connected layer, wherein the second pooling layer is connected to the feature extraction network, and the second fully connected layer is connected to the second pooling layer; The third classification network includes: a third fully connected layer.

6. The method according to claim 1, characterized in that, The method further includes: The first image is input into the second liveness detection model for processing to obtain the second liveness detection result of the first image; Based on the first liveness detection result and the second liveness detection result, a target liveness detection result is generated.

7. The method according to claim 1, characterized in that, The process of generating a mask based on a standard set of facial key points includes: Based on a standard set of facial key points, a facial outline is generated on a background image; Based on the position of the facial contour, a foreground region is generated on the background image to obtain a mask; Wherein, the overlap area between the foreground region and the face contour is greater than the target threshold; when the pixel value of the background image is 255, the pixel value of the foreground region is 0; when the pixel value of the background image is 0, the pixel value of the foreground region is 255.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that, The processor executes the computer program to implement the method according to any one of claims 1-7.

9. A computer-readable storage medium having a computer program / instructions stored thereon, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-7.

10. A computer program product comprising a computer program / instructions, characterized in that, When the computer program / instructions are executed by the processor, they implement the method described in any one of claims 1-7.

Images