Method, device, equipment and storage medium for living body recognition

By generating and using three-dimensional face images for face live detection, the problems of poor generalization and poor detection of unknown attacks in the prior art are solved, and the accuracy and performance of face live recognition are improved.

CN113255511BInactive Publication Date: 2024-12-06BEIJING BAIDU NETCOM SCI & TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110557990.9
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-05-21
Publication Date
2024-12-06
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing facial live detection technology has poor generalization and poor detection of unknown attack methods, which affects the actual application performance.

Method used

By acquiring multiple pending images of the target object under different poses, a pre-trained three-dimensional image generation model is used to generate three-dimensional face images, and live recognition is performed based on this, which improves the validity of identification of complex and diverse attack methods and generalization Metamorphosis.

Benefits of technology

It improves the accuracy of live body recognition and the ability to identify attack methods, improves the performance and user experience of face live body recognition technology, and is suitable for security, attendance, finance and other scenarios.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113255511B_ABST
    Figure CN113255511B_ABST
Patent Text Reader

Abstract

The present disclosure provides methods, devices, equipment and storage media for liveness recognition, which relate to the field of artificial intelligence technology, especially to the field of computer vision and deep learning technology, and can be applied to smart cities and financial scenarios. The specific implementation scheme is: obtaining multiple images to be processed of the target object, each image to be processed corresponds to a different posture of the target object; inputting the multiple images to be processed into a pre-trained three-dimensional image generation model to obtain a three-dimensional face image of the target object; based on the three-dimensional face image, obtaining the liveness recognition result of the target object. According to the technology disclosed in the present disclosure, the accuracy of liveness recognition of the target object is improved, and the effectiveness and generalization of identifying complex and diverse attack methods are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence technology, in particular to the field of computer vision and deep learning technology, and can be applied in smart city and financial scenarios. Background Art

[0002] Face liveness detection is a basic component of the face recognition system, ensuring the security of the face recognition system. Face liveness detection technology specifically refers to the computer's determination of whether the detected face is a real face or a fake face attack. Face attacks can be legitimate user pictures, videos taken in advance, etc. Among related technologies, face liveness detection algorithms using deep learning technology are the mainstream methods in this field, and have greatly improved accuracy compared to traditional algorithms. However, for some application scenarios, there are still problems with poor generalization and poor detection of unknown attack methods, which affect actual application performance. Summary of the invention

[0003] The present disclosure provides a method, an apparatus, a device and a storage medium for living body recognition.

[0004] According to one aspect of the present disclosure, a method for living body recognition is provided, comprising:

[0005] Acquire multiple images to be processed of the target object, each of which corresponds to a different posture of the target object;

[0006] Inputting a plurality of images to be processed into a pre-trained three-dimensional image generation model to obtain a three-dimensional face image of a target object;

[0007] Based on the three-dimensional face image, the liveness recognition result of the target object is obtained.

[0008] According to another aspect of the present disclosure, a method for training a three-dimensional image generation model is provided, comprising:

[0009] Using the sample image, determine the target three-dimensional face image;

[0010] Inputting the sample image into the 3D image generation model to be trained to obtain a predicted 3D face image;

[0011] The difference between the target 3D face image and the predicted 3D face image is determined, and the 3D image generation model to be trained is trained according to the difference until the difference is within an allowable range.

[0012] According to another aspect of the present disclosure, there is provided a device for living body recognition, comprising:

[0013] The to-be-processed image acquisition module is used to acquire multiple to-be-processed images of the target object, each to-be-processed image corresponds to a different posture of the target object;

[0014] A three-dimensional face image generation module is used to input a plurality of images to be processed into a pre-trained three-dimensional image generation model to obtain a three-dimensional face image of a target object;

[0015] The liveness recognition module is used to obtain the liveness recognition result of the target object based on the three-dimensional face image.

[0016] According to another aspect of the present disclosure, a training device for a three-dimensional image generation model is provided, comprising:

[0017] A target three-dimensional face image determination module is used to determine a target three-dimensional face image using a sample image;

[0018] A predicted three-dimensional face image generation module is used to input a sample image into a three-dimensional image generation model to be trained to obtain a predicted three-dimensional face image;

[0019] The training module is used to determine the difference between the target three-dimensional face image and the predicted three-dimensional face image, and train the three-dimensional image generation model to be trained according to the difference until the difference is within an allowable range.

[0020] According to another aspect of the present disclosure, there is provided an electronic device, comprising:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute the method in any embodiment of the present disclosure.

[0024] According to another aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided. The computer instructions are used to enable a computer to execute the method in any embodiment of the present disclosure.

[0025] According to another aspect of the present disclosure, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the method in any embodiment of the present disclosure is implemented.

[0026] According to the technology disclosed in the present invention, by acquiring multiple images to be processed in different postures of the target object, and then using a pre-trained three-dimensional image generation model to generate a three-dimensional face image of the target object, and finally determining the liveness recognition result of the target object based on the three-dimensional face image, the accuracy of liveness recognition of the target object is improved, and the effectiveness and generalization of identifying complex and diverse attack methods are improved.

[0027] It should be understood that the content described in this section is not intended to identify the key or important features of the embodiments of the present disclosure, nor is it intended to limit the scope of the present disclosure. Other features of the present disclosure will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The accompanying drawings are used to better understand the present solution and do not constitute a limitation of the present disclosure.

[0029] Figure 1 A flow chart showing a method for living body recognition according to an embodiment of the present disclosure;

[0030] Figure 2 A specific flow chart showing a method for obtaining a three-dimensional face image according to an embodiment of the present disclosure for living body recognition is shown;

[0031] Figure 3 A specific flow chart showing a liveness recognition result of a target object in a method for liveness recognition according to an embodiment of the present disclosure;

[0032] Figure 4 A specific flow chart showing preprocessing of a three-dimensional face image in a method for living body recognition according to an embodiment of the present disclosure;

[0033] Figure 5 A flowchart showing a method for training a three-dimensional image generation model according to an embodiment of the present disclosure;

[0034] Figure 6 A specific flow chart showing the method for training a three-dimensional image generation model according to an embodiment of the present disclosure to obtain a predicted three-dimensional face image;

[0035] Figure 7 A specific flow chart showing the determination of a target three-dimensional face image in a training method for a three-dimensional image generation model according to an embodiment of the present disclosure;

[0036] Figure 8 A flowchart showing preprocessing of a face region image in a training method for a three-dimensional image generation model according to an embodiment of the present disclosure;

[0037] Fig. 9 A specific flow chart showing the determination difference of the training method of the three-dimensional image generation model according to an embodiment of the present disclosure;

[0038] Fig.10 A schematic diagram showing a device for living body recognition according to an embodiment of the present disclosure;

[0039] Fig.11 A schematic diagram showing a three-dimensional image training device according to an embodiment of the present disclosure;

[0040] Fig.12The block diagram is a block diagram of an electronic device for implementing the method for living body identification according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0041] The following is a description of exemplary embodiments of the present disclosure in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding, which should be considered as merely exemplary. Therefore, it should be recognized by those of ordinary skill in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0042] Figure 1 A flow chart of a method for living body recognition according to an embodiment of the present disclosure is shown.

[0043] like Figure 1 As shown, the method for living body recognition in the embodiment of the present disclosure specifically includes the following steps:

[0044] S101: Acquire multiple images to be processed of a target object, each image to be processed corresponds to a different posture of the target object;

[0045] S102: Inputting a plurality of images to be processed into a pre-trained three-dimensional image generation model to obtain a three-dimensional face image of a target object;

[0046] S103: Obtaining a liveness recognition result of the target object based on the three-dimensional face image.

[0047] In the embodiments of the present disclosure, the target object can be understood as the target discrimination object in a specific scenario, and the liveness recognition result of the target object can be whether the carrier of the target object is the main body of the target object. For example, in the face recognition scenario, the target object can be a face, and the liveness recognition result of the target object can be whether the carrier of the target object is the main body or an attack, where the attack refers to a picture or video containing the target object.

[0048] The multiple images to be processed of the target object can be understood as multiple images to be processed containing the target object, and the target object has different positions and postures in different images to be processed. In other words, the target object has different perspectives in different images to be processed. It should be noted that the position and posture of the target object can be composed of three degrees of freedom of the target object in three-dimensional space and three degrees of freedom of spatial rotation. For example, in a face recognition scenario, different positions and postures of the target object can be that the face of the target object faces different directions.

[0049] For example, in step S101, the target object in different positions can be captured by a terminal device to obtain a plurality of images to be processed. The terminal device can be various image capture devices, such as a camera, a video camera, etc.

[0050] Exemplarily, in step S102, the 3D image generation model may adopt an SFM (Structure From Motion) model, an SFS (Shape From Shading, a method for restoring 3D information from a single image) model, or a 3DMM (3D Morphable models, 3D deformable face models) model.

[0051] Specifically, taking 3DMM as an example, its core idea is that the key points of a face can be matched one by one in three-dimensional space, and can be calculated by weighted or linear addition of many other orthogonal bases of faces. Based on the key points of the target object in each image to be processed, a fixed number of key points are used to represent the face, where the coordinates of each key point can be represented by the basis of the key point in three directions in three-dimensional space. For example, the coordinates of the key point (x, y, z) can be calculated by weighted addition of the basis of three directions in three-dimensional space (1, 0, 0), (0, 1, 0) and (0, 0, 1). Among them, the basic attributes of a three-dimensional face image include shape and texture, and each three-dimensional face image can be represented by the linear superposition of shape vectors and texture vectors.

[0052] Exemplarily, in step S103, the liveness recognition result of the target object can be obtained by inputting the three-dimensional face image into the pre-trained liveness recognition model. Specifically, the three-dimensional face image can adopt convolutional neural network (CNN), which is a type of feedforward neural network (FNN) with deep structure and convolution calculation, and can realize supervised learning or unsupervised learning of liveness recognition.

[0053] The following describes the method for living body recognition according to an embodiment of the present disclosure in conjunction with a specific application scenario.

[0054] In a face recognition scenario, in response to a face recognition request, facial images of the target object in different positions are obtained, for example, multiple images to be processed with the face of the target object facing different horizontal angles, or multiple images to be processed with the face of the target object facing different vertical angles. Then, the multiple images to be processed of the target object are input into a pre-trained three-dimensional image generation model to obtain a three-dimensional face image of the target object. Finally, the three-dimensional face image is input into a pre-trained liveness recognition model to obtain a liveness recognition result of the target object. If the liveness recognition result of the target object is live, the next face recognition process is performed; if the liveness recognition result of the target object is not live, the subsequent face recognition process is terminated.

[0055] Compared with the related art that only uses a single image of the target object as the input of the convolutional neural network, the method for liveness recognition in the embodiment of the present disclosure obtains multiple images to be processed in different postures of the target object, and then uses a pre-trained three-dimensional image generation model to generate a three-dimensional face image of the target object, and finally determines the liveness recognition result of the target object based on the three-dimensional face image. For the situation where the posture of the target object changes too much in real scenes, the method for liveness recognition in the embodiment of the present disclosure has high robustness, which is conducive to improving the accuracy of liveness recognition, and improves the effectiveness and generalization of identification against complex and diverse attack methods. In particular, in many application scenarios such as security, attendance, finance, and access control, the technical performance of face liveness recognition is effectively improved, and the use effect and user experience of many applications based on face liveness recognition technology are effectively improved, which is conducive to the further promotion of business projects.

[0056] like Figure 2 As shown, in one implementation, step S102 specifically includes the following steps:

[0057] S201: Input a plurality of images to be processed into a feature extraction layer in a three-dimensional image generation model, and receive a three-dimensional face image of a target object from an image generation layer in the three-dimensional image generation model.

[0058] Among them, the feature extraction layer is configured to extract parameter information of each image to be processed and input the parameter information of the image to be processed into the image generation layer; the image generation layer is configured to receive the parameter information of the image to be processed and output the three-dimensional face image of the target object.

[0059] Exemplarily, the feature extraction layer can use ResNet34 (a deep neural network). Specifically, ResNet34 uses a 34-layer residual network structure (Residual Unit). This residual jumping structure breaks the convention that the output of the n-1 layer of the traditional neural network can only be used as input to the n layer, so that the output of a certain layer can directly skip several layers as the input of a certain layer behind, which provides a new direction for the problem that the error rate of the entire learning model increases instead of decreases due to the superposition of multiple layers of networks. Therefore, the number of layers of the neural network can exceed the previous constraints and reach dozens, hundreds or even thousands of layers, which provides feasibility for high-level semantic feature extraction and classification. Based on this, not only the accuracy of parameter information extraction is improved, but also it is beneficial to improve the training efficiency of the three-dimensional face image generation model.

[0060] Based on the parameter information of each image to be processed extracted by the feature extraction layer, the image generation layer constructs a three-dimensional face contour of the target object, and finally obtains a three-dimensional face image of the target object by rendering the three-dimensional face contour.

[0061] According to the above implementation, by extracting parameter information of each image to be processed and generating a three-dimensional face image of the target object based on the parameter information, the accuracy and precision of the three-dimensional face image are improved, thereby further improving the accuracy and generalization of the liveness recognition result.

[0062] In one embodiment, the parameter information includes at least one of a shape parameter, an expression parameter, a texture parameter, and at least one of a posture parameter, a lighting parameter, and a camera parameter. The image generation layer is configured as follows:

[0063] Receive parameter information of an image to be processed, and obtain a three-dimensional contour image of a target object based on at least one of a shape parameter, an expression parameter, and a texture parameter; and render the three-dimensional contour video based on at least one of a posture parameter, an illumination parameter, and a camera parameter to obtain a three-dimensional face image of the target object.

[0064] It can be understood that shape parameters are used to characterize the basic shape parameters of the target object's face; expression parameters are used to characterize the emotional parameters of the target object's face; texture parameters are used to characterize the parameters of the arrangement and combination of multiple texture primitives that make up the target object's face, and texture primitives refer to image primitives with a certain size and shape composed of pixels; posture parameters are used to characterize the position of the target object's face in three-dimensional space and related parameters of facial orientation; lighting parameters are used to characterize related parameters such as lighting intensity and glossiness of the target object's face; and camera parameters are used to characterize the related shooting parameters used by the image acquisition device when acquiring the image to be processed.

[0065] Exemplarily, in the process where the image generation layer receives parameter information of the image to be processed from the feature extraction layer and outputs a three-dimensional face image of the target object, a three-dimensional contour image of the target object can be generated based on the shape parameters, expression parameters and texture parameters in the parameter information, and then the three-dimensional contour image can be differentiably rendered based on the posture parameters, lighting parameters and camera parameters to finally obtain the three-dimensional face image of the target object.

[0066] According to the above implementation, the image generation layer receives parameter information in multiple dimensions, constructs a three-dimensional contour image based on the parameter information in different dimensions, and renders the three-dimensional contour image, thereby improving the construction precision and accuracy of the three-dimensional face image in multiple dimensions, and further improving the robustness of subsequent living body recognition.

[0067] like Figure 3 As shown, in one implementation, step S103 includes:

[0068] S301: preprocessing the three-dimensional face image to obtain a preprocessed three-dimensional face image;

[0069] S302: Input the preprocessed three-dimensional face image into a pre-trained liveness recognition model to obtain a liveness recognition result of the target object.

[0070] It should be noted that in step S301, after preprocessing the three-dimensional face image, the data of the preprocessed three-dimensional face image can meet the input requirements of the liveness recognition model, and it is beneficial to simplify the data of the three-dimensional face image and improve the detection efficiency and stability of the subsequent liveness recognition model.

[0071] Exemplarily, in step S302, the living body recognition model may adopt various models known to those skilled in the art or to be known in the future.

[0072] Taking the liveness recognition model using the convolutional neural network model as an example, specifically, the convolutional neural network includes a feature extraction layer, a fully connected layer (FC) and an image normalization layer. The feature extraction layer is used to extract features from the three-dimensional face image, and then the extracted features are input into the fully connected layer. The fully connected layer classifies the three-dimensional face image according to the extracted features, and inputs the classification results into the image normalization layer. After the image normalization processing of the image normalization layer, the liveness recognition result is finally output. Among them, the feature extraction layer can use MobileNetV2 (a deep separable convolution) as the backbone network of the convolutional neural network; the image normalization layer can use the Softmax layer (a logistic regression model).

[0073] In addition, during the training process of the liveness recognition model, the liveness training samples and the attack training samples can be respectively input into the three-dimensional image generation model to obtain the liveness three-dimensional face image and the attack three-dimensional face image. The liveness recognition model is trained using batches of liveness three-dimensional face images and attack three-dimensional face images.

[0074] According to the above implementation, by inputting a three-dimensional face image into a pre-trained liveness recognition model to obtain the liveness recognition result of the target object, on the one hand, the detection precision and accuracy of the liveness recognition result are improved, and the generalization of attack detection is improved; on the other hand, the recognition efficiency of liveness recognition is improved, which is conducive to the further promotion of business projects that adopt the method of the embodiment of the present disclosure.

[0075] like Figure 4 As shown, in one implementation, step S301 includes:

[0076] S401: determining a face region in the three-dimensional face image based on the three-dimensional face image;

[0077] S402: extracting a partial image of the face region in the three-dimensional face image based on the face region as a face region image;

[0078] S403: performing image normalization processing on the face region image to obtain a pre-processed three-dimensional face image.

[0079] Exemplarily, in step S401, a face region in a three-dimensional face image may be determined by using a pre-trained face detection model.

[0080] Exemplarily, in step S402, the coordinate values ​​of each key point in the face area are detected by a pre-trained facial key point detection model, and the coordinate range of the face frame corresponding to the face area is determined based on the coordinate value of each key point. According to the coordinate range of the face frame, a partial image of the face area in the three-dimensional face image is captured to obtain a face area image.

[0081] For example, the facial key point detection model is used to detect the predefined 72 facial key point coordinates (x1, y1)…(x72, y72), and determine the two key point coordinates with the largest and smallest horizontal coordinates and the two key point coordinates with the largest and smallest vertical coordinates. Based on the above four coordinates, the coordinate range of the face frame corresponding to the face area is determined, and the three-dimensional face image is intercepted based on the coordinate range of the face frame to obtain the face area template image. Finally, the face area template image is enlarged three times, and the face frame in the face area template image is intercepted twice, and the final image is used as the face area image.

[0082] Exemplarily, in step S403, the pixel value of each pixel in the face region image is calculated according to a preset rule, and a preprocessed three-dimensional face image is determined based on the calculation result of each pixel value.

[0083] For example, for each pixel value in the face area image, the difference between each pixel value and 128 is divided by 256 to obtain a calculation result, wherein the calculation result of each pixel value has a value range between [-0.5, 0.5].

[0084] In addition, after performing image normalization processing on the face region graphics, random data enhancement processing can be performed on the preprocessed three-dimensional face image, and the processed three-dimensional face image can be used as an input of the living body recognition model.

[0085] Through the above implementation, the relevant data of the preprocessed three-dimensional face image can be limited to a preset range, thereby eliminating the influence of singular data in the relevant data on the liveness recognition result finally generated, and further improving the accuracy of the liveness recognition result.

[0086] In one embodiment, the plurality of images to be processed include a first image to be processed, a second image to be processed, and a third image to be processed. The first image to be processed corresponds to the position of the target object when the face is turned left by a first preset angle, the second image to be processed corresponds to the position of the target object when the face is facing forward, and the third image to be processed corresponds to the position of the target object when the face is turned right by a second preset angle.

[0087] Exemplarily, in the first image to be processed, the face of the target object is 45 degrees to the left relative to the image acquisition device; in the second image to be processed, the face of the target object is facing the image acquisition device; in the third image to be processed, the face of the target object is 45 degrees to the right relative to the image acquisition device.

[0088] Through the above implementation, multiple images to be processed respectively capture the states of the target object in different postures, so that the three-dimensional face image generated by the three-dimensional image generation model can be more consistent with the real face image of the target object, thereby improving the accuracy and precision of the three-dimensional face image, and being more conducive to the recognition of attack images in the subsequent liveness recognition process.

[0089] As another aspect of the embodiments of the present disclosure, a method for training a three-dimensional image generation model is also provided.

[0090] Figure 5 A training method for a three-dimensional image generation model according to an embodiment of the present disclosure is shown.

[0091] like Figure 5 As shown, the training method of the three-dimensional image generation model includes the following steps:

[0092] S501: using the sample image, determining a target three-dimensional face image;

[0093] S502: Inputting the sample image into the 3D image generation model to be trained to obtain a predicted 3D face image;

[0094] S503: Determine the difference between the target three-dimensional face image and the predicted three-dimensional face image, and train the three-dimensional image generation model to be trained according to the difference until the difference is within an allowable range.

[0095] For example, in step S501, the sample image can be obtained by respectively collecting multiple images of the living object in different positions and images of the attack object in different positions, wherein the attack object can be a photo or video of the living object. The target three-dimensional face image can be obtained by annotating the sample image.

[0096] Exemplarily, in step S503, the difference between the target three-dimensional face image and the predicted three-dimensional face image may be determined by calculating the loss function of each pixel value between the two.

[0097] According to the method of the embodiment of the present disclosure, the training of the three-dimensional image generation model is realized, and by using the images of the living object or the attacking object in different postures as sample images, the trained three-dimensional image generation model has high accuracy and generalization.

[0098] like Figure 6 As shown, in one implementation, step S502 includes:

[0099] S601: Inputting a sample image into a feature extraction layer in a 3D image generation model to be trained, and receiving a predicted 3D face image from an image generation layer in the 3D image generation model to be trained;

[0100] Among them, the feature extraction layer is configured to extract parameter information of the sample image and input the parameter information of the sample image into the image generation layer; the image generation layer is configured to receive the parameter information of the sample image and output a predicted three-dimensional face image.

[0101] Exemplarily, the feature extraction layer may use ResNet34. By using ResNet34 as the feature extraction layer, not only the accuracy of parameter information extraction is improved, but also the training efficiency of the three-dimensional face image generation model is improved.

[0102] Based on the parameter information of each image to be processed extracted by the feature extraction layer, the image generation layer constructs a three-dimensional face contour of the target object, and finally obtains a three-dimensional face image of the target object by rendering the three-dimensional face contour.

[0103] Through the above implementation, the trained 3D image generation model can extract parameter information of each image to be processed and generate a 3D face image of the target object based on the parameter information, thereby improving the accuracy and precision of the 3D face image output by the 3D image generation model.

[0104] like Figure 7 As shown, in one implementation, step S501 includes:

[0105] S701: Based on the sample image, determine the facial key point data in the sample image;

[0106] S702: performing segmentation and masking processing on the sample image based on the facial key point data to obtain a facial region image;

[0107] S703: Preprocess the face region image to obtain a target three-dimensional face image.

[0108] Exemplarily, the coordinate values ​​of the facial key points in the sample image are detected through a pre-trained facial key point dataset, and the coordinate range of the face frame corresponding to the face area is determined based on the coordinate value of each facial key point. According to the coordinate range of the face frame, a partial image of the face area in the three-dimensional face image is segmented and masked to obtain a face area image.

[0109] For example, the facial key point dataset may be an ibug key point dataset, which is used to detect the predefined 68 facial key point coordinates (x1, y1)…(x68, y68), and determine the two facial key point coordinates with the largest and smallest horizontal coordinates and the two facial key point coordinates with the largest and smallest vertical coordinates. Based on the above four facial key point coordinates, the coordinate range of the face frame corresponding to the face area is determined, and the three-dimensional face image is segmented and masked based on the coordinate range of the face frame to obtain a face area template image. Finally, the face area template image is enlarged three times, and the face frame in the face area template image is intercepted twice, and the final image is used as the face area image.

[0110] Through the above implementation, the face area image can be accurately extracted from the sample image as the input of the three-dimensional image generation model to be trained, thereby eliminating the influence of singular data in the sample image data on the training process.

[0111] like Figure 8 As shown, in one implementation, step S703 includes:

[0112] S801: performing alignment processing, cropping processing, size adjustment processing and image normalization processing on the face region image to obtain a target three-dimensional face image.

[0113] For example, in step S801, a pre-established high-definition face dataset (Flickr Faces High Quality, FFHQ) can be used to align and crop the face region image according to the matching result of the face region image in the high-definition face dataset, and the size of the face region image is adjusted to 224*224.

[0114] In addition, the normalization processing of the face area image can be performed by calculating the pixel value of each pixel in the face area image according to preset rules, and determining the pre-processed three-dimensional face image based on the calculation result of each pixel value.

[0115] For example, for each pixel value in the face area image, the difference between each pixel value and 128 is divided by 256 to obtain a calculation result, wherein the calculation result of each pixel value has a value range between [-0.5, 0.5].

[0116] Through the above implementation, it can be ensured that the determined target three-dimensional face image has high accuracy, and the human labeling cost of manual labeling is saved, thereby reducing the difficulty of obtaining the target three-dimensional face image.

[0117] like Fig. 9 As shown, in one implementation, step S503 includes:

[0118] S901: Calculating a first loss value based on pixel values ​​of the predicted three-dimensional face image and pixel values ​​of the target three-dimensional face image;

[0119] S902: Calculating a second loss value based on the key point coordinate values ​​of the predicted 3D face image and the key point coordinate values ​​of the sample image;

[0120] S903: Sum the first loss value and the second loss value to obtain a difference between the target three-dimensional face image and the predicted three-dimensional face image.

[0121] Exemplarily, in step S901, the specific steps of calculating the first loss value include: rendering the predicted three-dimensional face image to obtain a two-dimensional image corresponding to the predicted three-dimensional face image, bit-wise subtracting the pixel values ​​of the two-dimensional image from the pixel values ​​of the sample image, bit-wise multiplying the obtained pixel difference with the pixel value of the face area image, summing the absolute value of the product of each pixel obtained, and obtaining the first loss value.

[0122] Exemplarily, in step S902, the key point data set can be used to determine the key point coordinates of the predicted face image, and then the L1 loss function between the pixel values ​​of the key point coordinates of the predicted face image and the pixel values ​​of the key point coordinates of the sample image is calculated to obtain a second loss value.

[0123] Exemplarily, in step S903, the first loss value, the second loss value, and regularization terms of other parameters such as shape parameters, expression parameters, texture parameters, and posture parameters of the sample image are summed to obtain the difference between the target three-dimensional face image and the predicted three-dimensional face image.

[0124] Through the above implementation, the difference between the target three-dimensional face image and the predicted three-dimensional face image can be accurately obtained, thereby improving the optimization effect of the three-dimensional image generation model during the training process.

[0125] According to another aspect of the embodiments of the present disclosure, a device for living body recognition is also provided.

[0126] like Fig.10 As shown, the device for living body recognition includes:

[0127] The to-be-processed image acquisition module 1001 is used to acquire multiple to-be-processed images of the target object, each to-be-processed image corresponds to a different posture of the target object;

[0128] The 3D face image generation module 1002 is used to input a plurality of images to be processed into a pre-trained 3D image generation model to obtain a 3D face image of a target object;

[0129] The liveness recognition module 1003 is used to obtain the liveness recognition result of the target object based on the three-dimensional face image.

[0130] In one implementation, the 3D face image generation module 1002 is further configured to:

[0131] Inputting a plurality of images to be processed into a feature extraction layer in a three-dimensional image generation model, and receiving a three-dimensional face image of a target object from an image generation layer in the three-dimensional image generation model;

[0132] Among them, the feature extraction layer is configured to extract parameter information of each image to be processed and input the parameter information of the image to be processed into the image generation layer; the image generation layer is configured to receive the parameter information of the image to be processed and output the three-dimensional face image of the target object.

[0133] In one embodiment, the parameter information includes at least one of a shape parameter, an expression parameter, and a texture parameter, and at least one of a posture parameter, an illumination parameter, and a camera parameter; and the image generation layer is configured as follows:

[0134] receiving parameter information of an image to be processed, and obtaining a three-dimensional contour image of a target object based on at least one of a shape parameter, an expression parameter, and a texture parameter; and

[0135] Based on at least one of the posture parameter, the illumination parameter and the camera parameter, the three-dimensional contour video is rendered to obtain a three-dimensional face image of the target object.

[0136] In one embodiment, the living body recognition module 1003 includes:

[0137] A preprocessing submodule, used for preprocessing the three-dimensional face image to obtain a preprocessed three-dimensional face image;

[0138] The liveness recognition submodule is used to input the preprocessed three-dimensional face image into the pre-trained liveness recognition model to obtain the liveness recognition result of the target object.

[0139] In one embodiment, the preprocessing submodule includes:

[0140] A face region determination unit, used to determine a face region in the three-dimensional face image based on the three-dimensional face image;

[0141] A face region image determination unit, used to extract a partial image of the face region in the three-dimensional face image based on the face region as the face region image;

[0142] The image normalization processing unit is used to perform image normalization processing on the face area image to obtain a pre-processed three-dimensional face image.

[0143] In one embodiment, the plurality of images to be processed include a first image to be processed, a second image to be processed, and a third image to be processed;

[0144] The first image to be processed corresponds to the position of the target object when the face is turned left by a first preset angle, the second image to be processed corresponds to the position of the target object when the face is facing forward, and the third image to be processed corresponds to the position of the target object when the face is turned right by a second preset angle.

[0145] According to another aspect of the present disclosure, a training device for a three-dimensional image generation model is also provided.

[0146] like Fig.11 As shown, the training device of the three-dimensional image generation model includes:

[0147] A target 3D face image determination module 1101 is used to determine a target 3D face image using a sample image;

[0148] The predicted 3D face image generation module 1102 is used to input the sample image into the 3D image generation model to be trained to obtain a predicted 3D face image;

[0149] The training module 1103 is used to determine the difference between the target 3D face image and the predicted 3D face image, and train the 3D image generation model to be trained according to the difference until the difference is within an allowable range.

[0150] In one implementation, the predicted 3D face image generation module 1102 is further configured to:

[0151] Inputting the sample image into a feature extraction layer in a three-dimensional image generation model to be trained, and receiving a predicted three-dimensional face image from an image generation layer in the three-dimensional image generation model to be trained;

[0152] Among them, the feature extraction layer is configured to extract parameter information of the sample image and input the parameter information of the sample image into the image generation layer; the image generation layer is configured to receive the parameter information of the sample image and output a predicted three-dimensional face image.

[0153] In one implementation, the target 3D face image determination module 1101 includes:

[0154] A facial key point data determination submodule, used to determine facial key point data in a sample image based on the sample image;

[0155] The face area image determination submodule is used to perform segmentation and mask processing on the sample image based on the face key point data to obtain the face area image;

[0156] The target three-dimensional face image determination submodule is used to pre-process the face area image to obtain the target three-dimensional face image.

[0157] In one embodiment, the target 3D face image determination submodule is further used for:

[0158] The face region image is aligned, cropped, resized and normalized to obtain a target three-dimensional face image.

[0159] In one embodiment, the training module 1103 includes:

[0160] A first loss value calculation submodule, used to calculate a first loss value based on the pixel values ​​of the predicted three-dimensional face image and the pixel values ​​of the target three-dimensional face image;

[0161] A second loss value calculation submodule, used to calculate a second loss value based on the key point coordinate values ​​of the predicted three-dimensional face image and the key point coordinate values ​​of the sample image;

[0162] The difference calculation submodule is used to sum the first loss value and the second loss value to obtain the difference between the target three-dimensional face image and the predicted three-dimensional face image.

[0163] The functions of each unit, module or sub-module in each device of the embodiments of the present disclosure can be found in the corresponding description in the above method embodiments, and will not be repeated here.

[0164] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium and a computer program product.

[0165] Fig.12 A schematic block diagram of an example electronic device 1200 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0166] like Fig.12 As shown, the electronic device 1200 includes a computing unit 1201, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 1202 or a computer program loaded from a storage unit 1208 into a random access memory (RAM) 1203. In the RAM 1203, various programs and data required for the operation of the electronic device 1200 can also be stored. The computing unit 1201, the ROM 1202, and the RAM 1203 are connected to each other via a bus 1204. An input / output (I / O) interface 1205 is also connected to the bus 1204.

[0167] Multiple components in the electronic device 1200 are connected to the I / O interface 1205, including: an input unit 1206, such as a keyboard, a mouse, etc.; an output unit 1207, such as various types of displays, speakers, etc.; a storage unit 1208, such as a disk, an optical disk, etc.; and a communication unit 1209, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 1209 allows the electronic device 1200 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0168] The computing unit 1201 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 1201 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 1201 performs the various methods and processes described above, such as a method for live body recognition and / or a training method for a three-dimensional image generation model. For example, in some embodiments, the method for live body recognition and / or the training method for a three-dimensional image generation model may be implemented as a computer software program, which is tangibly included in a machine-readable medium, such as a storage unit 1208. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 1200 via the ROM 1202 and / or the communication unit 1209. When the computer program is loaded into the RAM 1203 and executed by the computing unit 1201, one or more steps of the method for live body recognition and / or the training method for a three-dimensional image generation model described above may be performed. Alternatively, in other embodiments, the computing unit 1201 may be configured in any other appropriate manner (eg, by means of firmware) to execute the method for living body recognition and / or the training method for the three-dimensional image generation model.

[0169] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0170] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0171] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0172] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0173] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0174] A computer system may include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship to each other.

[0175] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0176] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for living body recognition, comprising: Acquire a plurality of to-be-processed images of the target object, each of the to-be-processed images corresponding to a different posture of the target object; Inputting the plurality of images to be processed into a pre-trained three-dimensional image generation model to obtain a three-dimensional face image of the target object; Based on the three-dimensional face image, obtaining a liveness recognition result of the target object; Wherein, determining the liveness recognition result of the target object based on the three-dimensional face image includes: Based on the three-dimensional face image, determining a face area in the three-dimensional face image; The facial key point detection model trained in advance is used to detect the coordinate values ​​of each key point in the facial region, and the coordinate range of the facial frame corresponding to the facial region is determined according to the coordinate values ​​of each key point, and a partial image of the facial region in the three-dimensional facial image is intercepted according to the coordinate range of the facial frame to obtain a facial region image; For each pixel value in the face region image, a calculation process is performed according to a preset rule, and a preprocessed three-dimensional face image is determined based on the calculation result of each pixel value, wherein the calculation result of each pixel value is within a preset range; The preprocessed three-dimensional face image is input into a pre-trained liveness recognition model to obtain a liveness recognition result of the target object.

2. The method according to claim 1, wherein: The step of inputting the plurality of images to be processed into a pre-trained three-dimensional image generation model to obtain a three-dimensional face image of the target object comprises: Inputting the plurality of images to be processed into a feature extraction layer in the three-dimensional image generation model, and receiving a three-dimensional face image of the target object from an image generation layer in the three-dimensional image generation model; Among them, the feature extraction layer is configured to extract parameter information of each image to be processed and input the parameter information of the image to be processed into the image generation layer; the image generation layer is configured to receive the parameter information of the image to be processed and output the three-dimensional face image of the target object.

3. The method according to claim 2, wherein: The parameter information includes at least one of a shape parameter, an expression parameter, a texture parameter, and at least one of a posture parameter, a lighting parameter, and a camera parameter; The image generation layer is configured as: Receiving parameter information of the image to be processed, and obtaining a three-dimensional contour image of the target object based on at least one of the shape parameter, the expression parameter, and the texture parameter; as well as, Based on at least one of the posture parameter, the illumination parameter and the camera parameter, the three-dimensional contour video is rendered to obtain a three-dimensional face image of the target object.

4. The method according to any one of claims 1 to 3, wherein the plurality of images to be processed include a first image to be processed, a second image to be processed and a third image to be processed; The first image to be processed corresponds to the position of the target object when the face is turned left by a first preset angle, the second image to be processed corresponds to the position of the target object when the face is facing forward, and the third image to be processed corresponds to the position of the target object when the face is turned right by a second preset angle.

5. A training method for a three-dimensional image generation model, comprising: Using the sample image, determine the target three-dimensional face image; Inputting the sample image into a three-dimensional image generation model to be trained to obtain a predicted three-dimensional face image; Determining a difference between the target three-dimensional face image and the predicted three-dimensional face image, and training the three-dimensional image generation model to be trained according to the difference until the difference is within an allowable range; Wherein, the step of using the sample image to determine the target three-dimensional face image includes: Based on the sample image, determining facial key point data in the sample image; Using a pre-trained facial key point data set, the coordinate values ​​of the facial key points in the sample image are detected, and according to the coordinate values ​​of each facial key point, the coordinate range of the face frame corresponding to the face region is determined, and according to the coordinate range of the face frame, a partial image of the face region in the three-dimensional face image is segmented and masked to obtain a face region image; The pixel value of each pixel in the face area image is calculated and processed according to preset rules, and a preprocessed three-dimensional face image is determined based on the calculation result of each pixel value to obtain a target three-dimensional face image, wherein the calculation result of each pixel value is within a preset range.

6. The method according to claim 5, wherein: The step of inputting the sample image into a three-dimensional image generation model to be trained to obtain a predicted three-dimensional face image comprises: Inputting the sample image into a feature extraction layer in a three-dimensional image generation model to be trained, and receiving a predicted three-dimensional face image from an image generation layer in the three-dimensional image generation model to be trained; Among them, the feature extraction layer is configured to extract parameter information of the sample image and input the parameter information of the sample image into the image generation layer; the image generation layer is configured to receive the parameter information of the sample image and output the predicted three-dimensional face image.

7. The method according to claim 5, wherein: The determining the difference between the target three-dimensional face image and the predicted three-dimensional face image includes: Calculating a first loss value based on the pixel values ​​of the predicted three-dimensional face image and the pixel values ​​of the target three-dimensional face image; Calculating a second loss value based on the key point coordinate values ​​of the predicted three-dimensional face image and the key point coordinate values ​​of the sample image; The first loss value and the second loss value are summed to obtain a difference between the target three-dimensional face image and the predicted three-dimensional face image.

8. A device for living body recognition, comprising: A module for acquiring images to be processed, used for acquiring a plurality of images to be processed of a target object, each of the images to be processed corresponds to a different posture of the target object; A three-dimensional face image generation module, used for inputting a plurality of the images to be processed into a pre-trained three-dimensional image generation model to obtain a three-dimensional face image of the target object; A liveness recognition module, used to obtain a liveness recognition result of the target object based on the three-dimensional face image; The living body recognition module includes: a preprocessing submodule and a living body recognition submodule; the preprocessing submodule includes: a face area determination unit, a face area image determination unit and an image normalization processing unit; The face region determining unit is used to determine the face region in the three-dimensional face image based on the three-dimensional face image; The face area image determination unit is used to detect the coordinate values ​​of each key point in the face area through a pre-trained face key point detection model, determine the coordinate range of the face frame corresponding to the face area according to the coordinate values ​​of each key point, and intercept a partial image of the face area in the three-dimensional face image according to the coordinate range of the face frame to obtain a face area image; The image normalization processing unit is used to calculate the pixel value of each pixel in the face area image according to a preset rule, and determine a preprocessed three-dimensional face image based on the calculation result of each pixel value, wherein the calculation result of each pixel value is within a preset range; The liveness recognition submodule is used to input the preprocessed three-dimensional face image into a pre-trained liveness recognition model to obtain a liveness recognition result of the target object.

9. The device according to claim 8, wherein: The three-dimensional face image generation module is also used for: Inputting the plurality of images to be processed into a feature extraction layer in the three-dimensional image generation model, and receiving a three-dimensional face image of the target object from an image generation layer in the three-dimensional image generation model; Among them, the feature extraction layer is configured to extract parameter information of each image to be processed and input the parameter information of the image to be processed into the image generation layer; the image generation layer is configured to receive the parameter information of the image to be processed and output the three-dimensional face image of the target object.

10. The device according to claim 9, wherein: The parameter information includes at least one of a shape parameter, an expression parameter, a texture parameter, and at least one of a posture parameter, a lighting parameter, and a camera parameter; The image generation layer is configured as: Receiving parameter information of the image to be processed, and obtaining a three-dimensional contour image of the target object based on at least one of the shape parameter, the expression parameter, and the texture parameter; as well as, Based on at least one of the posture parameter, the illumination parameter and the camera parameter, the three-dimensional contour video is rendered to obtain a three-dimensional face image of the target object.

11. The device according to any one of claims 8 to 10, wherein the plurality of images to be processed include a first image to be processed, a second image to be processed and a third image to be processed; The first image to be processed corresponds to the position of the target object when the face is turned left by a first preset angle, the second image to be processed corresponds to the position of the target object when the face is facing forward, and the third image to be processed corresponds to the position of the target object when the face is turned right by a second preset angle.

12. A training device for a three-dimensional image generation model, comprising: A target three-dimensional face image determination module is used to determine a target three-dimensional face image using a sample image; A predicted three-dimensional face image generation module, used for inputting the sample image into the three-dimensional image generation model to be trained to obtain a predicted three-dimensional face image; A training module, used for determining a difference between the target 3D face image and the predicted 3D face image, and training the 3D image generation model to be trained according to the difference until the difference is within an allowable range; Wherein, the target three-dimensional face image determination module includes: A facial key point data determination submodule, used to determine the facial key point data in the sample image based on the sample image; A face region image determination submodule is used to detect the coordinate values ​​of face key points in the sample image through a pre-trained face key point data set, determine the coordinate range of the face frame corresponding to the face region according to the coordinate values ​​of each face key point, and perform segmentation and masking processing on a part of the face region in the three-dimensional face image according to the coordinate range of the face frame to obtain a face region image; The target three-dimensional face image determination submodule is used to calculate and process the pixel value of each pixel in the face area image according to preset rules, and determine the preprocessed three-dimensional face image based on the calculation result of each pixel value to obtain the target three-dimensional face image, wherein the calculation result of each pixel value is within a preset range.

13. The device according to claim 12, wherein: The predicted three-dimensional face image generation module is also used for: Inputting the sample image into a feature extraction layer in a three-dimensional image generation model to be trained, and receiving a predicted three-dimensional face image from an image generation layer in the three-dimensional image generation model to be trained; Among them, the feature extraction layer is configured to extract parameter information of the sample image and input the parameter information of the sample image into the image generation layer; the image generation layer is configured to receive the parameter information of the sample image and output the predicted three-dimensional face image.

14. The device according to claim 13, wherein: The training module includes: A first loss value calculation submodule, used to calculate a first loss value based on the pixel value of the predicted three-dimensional face image and the pixel value of the target three-dimensional face image; A second loss value calculation submodule, used to calculate a second loss value based on the key point coordinate values ​​of the predicted three-dimensional face image and the key point coordinate values ​​of the sample image; The difference calculation submodule is used to sum the first loss value and the second loss value to obtain the difference between the target three-dimensional face image and the predicted three-dimensional face image.

15. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 7.

16. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause a computer to execute the method according to any one of claims 1 to 7.