A liveness detection model training method and liveness detection method

Abstract

The present application relates to the field of biometric identification technology, and in particular to a method for training a liveness detection model and a liveness detection method. The liveness detection model training method includes: constructing an initial model for liveness detection, the initial model for liveness detection includes at least three convolution modules, a feature fusion module and a pooling module; obtaining a face image, and obtaining at least three feature images when the face image passes through each convolution module in turn; using the feature fusion module to fuse the at least three feature images to obtain a fourth feature image; using the pooling module to pool the fourth feature image, and using the fourth feature image and the fourth feature image after pooling to supervise the initial model for liveness detection and iteratively update the initial model for liveness detection to obtain a trained liveness detection model. The embodiment of the present application has a faster inference speed; a lightweight model is used, which can be successfully deployed on a chip with low computing power; and a more accurate liveness detection model can be obtained.

Description

Technical Field

[0001] The present application relates to the field of biometric identification technology, and in particular to a liveness detection model training method and a liveness detection method. Background Art

[0002] As face recognition technology matures, its commercial applications are becoming more and more extensive, for example, it is widely used in financial transactions, access control systems, mobile terminals and other fields. However, faces are very easy to copy using photos, videos, models or masks, so the counterfeiting of legitimate users' faces is a major threat to the security of face recognition and authentication systems. In order to prevent malicious people from forging and stealing other people's biometric features for identity authentication, the biometric recognition system needs to have a liveness detection function.

[0003] At present, the liveness detection technology commonly used in face recognition technology has begun to adopt neural network models. However, it is usually unable to capture the fine-grained information of face data, so the effect on high-precision resin masks and 3D head models is not satisfactory, and they are usually misjudged as real people; or it can effectively prevent prosthetic attacks, but some real people cannot pass the biopsy, resulting in a bad user experience; or even if the model has a good anti-attack effect and a high pass rate for real people, it is usually too large and slow to be deployed on the terminal, and cannot be deployed on some chips with low computing power. Summary of the invention

[0004] In view of this, an embodiment of the present application provides a liveness detection model training method and a liveness detection method, which can solve at least one technical problem in the related art.

[0005] In a first aspect, an embodiment of the present application provides a method for training a liveness detection model, comprising:

[0006] Constructing an initial model for liveness detection, wherein the initial model for liveness detection includes at least three convolution modules, a feature fusion module, and a pooling module;

[0007] Acquire a face image, and obtain at least three feature images when the face image passes through each convolution module in sequence;

[0008] Using the feature fusion module to fuse the at least three feature images to obtain a fourth feature image;

[0009] The fourth feature image is pooled by using the pooling module, and the fourth feature image and the pooled fourth feature image are used to perform supervised training on the initial model for liveness detection and iteratively update the initial model for liveness detection to obtain a trained liveness detection model.

[0010] As an implementation of the first aspect, the acquiring of a facial image and obtaining at least three feature images when the facial image passes through each convolution module in sequence include:

[0011] Acquire a face image, and upsample or downsample the face image to obtain a target image;

[0012] The at least three convolution modules of the initial model for liveness detection are used to perform feature learning on the target image to obtain at least three feature images.

[0013] As an implementation of the first aspect, the fourth feature image is pooled by the pooling module, and the fourth feature image and the pooled fourth feature image are used to supervise the initial model for liveness detection and iteratively update the initial model for liveness detection to obtain a trained liveness detection model, including:

[0014] Using the pooling module to perform average pooling on the fourth feature image to obtain a fifth feature image and a sixth feature image;

[0015] The losses between the fourth feature image, the fifth feature image and the sixth feature image and their corresponding preset supervisory images are calculated according to the loss function, and the weight parameters of the initial model of liveness detection are iteratively updated according to the losses to obtain a trained liveness detection model.

[0016] As an implementation manner of the first aspect, the loss function is a mean square error loss function; the fourth feature image, the fifth feature image, and the sixth feature image have different sizes.

[0017] In a second aspect, an embodiment of the present application provides a living body detection method, including:

[0018] Obtain the image to be detected of the target object's face;

[0019] Inputting the image to be detected into a trained liveness detection model obtained by the liveness detection model training method as described in the first aspect or any implementation manner of the first aspect, and outputting a feature image;

[0020] The feature image is binarized, and the mean of the binarized feature image is solved, and whether the target object is a living body is determined according to the mean.

[0021] As an implementation of the second aspect, the step of inputting the image to be detected into a trained liveness detection model obtained by the liveness detection model training method described in the first aspect or any implementation of the first aspect, and outputting a feature image includes:

[0022] If it is determined that the image to be detected meets the preset quality condition, the image to be detected is input into a trained liveness detection model obtained by the liveness detection model training method as described in the first aspect or any implementation of the first aspect, and a feature image is output.

[0023] As an implementation of the second aspect, the image to be detected includes at least one of an infrared image, a color image, and a depth image of a face of the target object;

[0024] The step of inputting the image to be detected into a trained liveness detection model obtained by the liveness detection model training method as described in the first aspect or any implementation of the first aspect, and outputting a feature image comprises:

[0025] The image to be detected is input into a liveness detection model corresponding to the image to be detected, and a trained liveness detection model obtained by using the liveness detection model training method as described in the first aspect or any implementation of the first aspect is used to output a feature image.

[0026] In a third aspect, an embodiment of the present application provides a living body detection model training device, including:

[0027] A construction module, used to construct an initial model for liveness detection, wherein the initial model for liveness detection includes at least three convolution modules, a feature fusion module and a pooling module;

[0028] An acquisition module, used for acquiring a face image;

[0029] A convolution execution module, used to obtain at least three feature images when the face image passes through each convolution module in sequence;

[0030] a feature fusion execution module, configured to use the feature fusion module to fuse the at least three feature images to obtain a fourth feature image;

[0031] A pooling execution module is used to use the pooling module to pool the fourth feature image, and use the fourth feature image and the pooled fourth feature image to supervise the training of the initial model of liveness detection and iteratively update the initial model of liveness detection to obtain a trained liveness detection model.

[0032] In a fourth aspect, an embodiment of the present application provides a door lock system based on liveness detection, including:

[0033] A camera module, used for collecting an image of a target object to be detected;

[0034] A module for acquiring an image to be detected, used to acquire the image to be detected of the target object's face;

[0035] a feature image acquisition module, configured to input the image to be detected into a trained liveness detection model obtained by the liveness detection model training method as described in the first aspect or any implementation of the first aspect, and output a feature image;

[0036] The judgment module is used to binarize the feature image and calculate the mean of the binarized feature image, and judge whether the target object is a living body according to the mean to perform identity authentication.

[0037] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, it implements the liveness detection model training method as described in the first aspect or any implementation manner of the first aspect; or it implements the liveness detection method as described in the second aspect or any implementation manner of the second aspect.

[0038] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the liveness detection model training method as described in the first aspect or any implementation of the first aspect is implemented; or the liveness detection method as described in the second aspect or any implementation of the second aspect is implemented.

[0039] In the seventh aspect, an embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device executes the liveness detection model training method as described in the first aspect or any implementation of the first aspect, or executes the liveness detection method as described in the second aspect or any implementation of the second aspect.

[0040] The embodiment of the present application uses a complete face image instead of an image block, which makes the inference faster; it uses a lightweight model and can be successfully deployed on a chip with low computing power; it uses a customized backbone network structure and multi-supervisory images to more effectively extract the features of face data and capture the difference between a high-precision prosthesis and a real person to obtain a more accurate liveness detection model. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0042] Figure 1 This is a schematic diagram of an implementation flow of a liveness detection model training method provided in one embodiment of the present application;

[0043] Figure 2 It is a structural schematic diagram of an initial model of liveness detection provided by an embodiment of the present application;

[0044] Figure 3 This is a schematic diagram of a specific implementation flow of step S120 in a living body detection model training method provided in an embodiment of the present application;

[0045] Figure 4 is a schematic diagram of a sample image for supervised training used in a liveness detection model training method provided in an embodiment of the present application;

[0046] Figure 5 is a schematic diagram of samples and corresponding labels used in a liveness detection model training method provided in an embodiment of the present application;

[0047] Figure 6 This is a schematic diagram of an implementation flow of a liveness detection method provided in an embodiment of the present application;

[0048] Figure 7 This is a schematic diagram of an implementation flow of a liveness detection method provided by another embodiment of the present application;

[0049] Figure 8 This is a schematic diagram of an implementation flow of a liveness detection method provided by another embodiment of the present application;

[0050] Fig. 9 This is a schematic diagram of an implementation flow of a liveness detection method provided by another embodiment of the present application;

[0051] Fig.10 It is a structural schematic diagram of a living body detection model training device provided in one embodiment of the present application;

[0052] Fig.11 It is a structural schematic diagram of a door lock system based on liveness detection provided in one embodiment of the present application;

[0053] Fig.12 It is a structural schematic diagram of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0054] In the following description, specific details such as specific system structures, technologies, etc. are provided for the purpose of illustration rather than limitation, so as to provide a thorough understanding of the embodiments of the present application. However, it should be clear to those skilled in the art that the present application may also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to prevent unnecessary details from obstructing the description of the present application.

[0055] The term "and / or" as used in this specification and the appended claims refers to and includes any and all possible combinations of one or more of the associated listed items.

[0056] "One embodiment" or "some embodiments" described in the specification of this application means that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Therefore, the sentences "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0057] In addition, in the description of the present application, "plurality" means two or more. The terms "first" and "second" are only used to distinguish the description and cannot be understood as indicating or implying relative importance.

[0058] In order to illustrate the technical solution described in this application, a specific embodiment is provided below for illustration.

[0059] Figure 1 This is a schematic diagram of the implementation process of a liveness detection model training method provided in an embodiment of the present application. The liveness detection model training method in this embodiment can be executed by an electronic device. The electronic device includes but is not limited to a computer, a tablet computer, a server, a mobile phone, a camera or a wearable device. The server includes but is not limited to an independent server or a cloud server. Figure 1 As shown, the living body detection model training method may include steps S110 to S140.

[0060] S110, constructing an initial model for liveness detection, wherein the initial model for liveness detection includes at least three convolution modules, a feature fusion module, and a pooling module.

[0061] In step S110, an initial model for liveness detection is constructed as a model to be trained. The initial model for liveness detection includes a set of weight parameters to be learned. The initial model for liveness detection includes at least three convolution modules, a feature fusion module and a pooling module.

[0062] In some embodiments, the initial model for liveness detection is as follows: Figure 2 As shown, the initial model of liveness detection includes three convolution modules, a feature fusion module and a pooling module, wherein the three convolution modules are respectively a first convolution module, a second convolution module and a third convolution module.

[0063] In some embodiments, the convolution kernels of the convolution module, the feature fusion module and the pooling module in the initial model of liveness detection are all 3×3.

[0064] S120, acquiring a face image, and obtaining at least three feature images when the face image passes through each convolution module in sequence.

[0065] The face image is a face image sample, including a positive sample and a negative sample. The face image includes but is not limited to one or more of a face infrared image, a face color image, and a face depth image.

[0066] In some embodiments, Figure 3 As shown, step S120 includes the following steps S121 to S122.

[0067] S121, acquiring a face image, and upsampling or downsampling the face image to obtain a target image.

[0068] Upsampling, also known as image interpolation or image magnification, is mainly aimed at magnifying the face image so that the enlarged face image, i.e., the target image, meets the input image size of the initial model for liveness detection. Downsampling, also known as downsampling, is mainly aimed at reducing the face image so that the reduced face image, i.e., the target image, meets the input image size of the liveness detection model.

[0069] In some embodiments, preferably, the input image size of the initial model for liveness detection is 112×112.

[0070] S122, using at least three convolution modules of the initial model of liveness detection to perform feature learning on the target image to obtain at least three feature images.

[0071] In some embodiments, each convolution module includes at least three convolution layers with a convolution kernel of 3×3 to perform feature learning on the target image and output a feature image. The size of the feature images output by each convolution module may be the same.

[0072] As a non-limiting example, the initial model of liveness detection includes three convolution modules. Among them, the feature image output by the first convolution module is 15×15, the feature image output by the second convolution module is 15×15, and the feature image output by the third convolution module is 15×15. The structure of these three convolution modules can be regarded as a convolution layer with a convolution kernel of 3×3, two convolution layers with a convolution kernel of 3×3, and three convolution layers with a convolution kernel of 3×3 in parallel. The three convolution modules are used to obtain different feature layers of the input target image.

[0073] S130: Using a feature fusion module to fuse at least three feature images to obtain a fourth feature image.

[0074] The feature fusion module includes a fusion (concat) function, which fuses and connects the effective features in the feature image obtained by each convolution module to obtain a fourth feature image with only one channel.

[0075] As a non-limiting example, the feature image output by the first convolution module is 15×15, the feature image output by the second convolution module is 15×15, and the feature image output by the third convolution module is 15×15. These three feature images are fused using the feature fusion module to obtain a fourth feature image of 15×15.

[0076] S140, using a pooling module to average-pool the fourth feature image, and using the fourth feature image and the fourth feature image after average-pooling to supervise the initial model of liveness detection and iteratively update the initial model of liveness detection to obtain a trained liveness detection model.

[0077] The fourth feature image is average-pooled by using a pooling module to obtain a fifth feature image and a sixth feature image, which are the fourth feature image after average pooling.

[0078] In some embodiments, the pooling module includes two convolution layers with a convolution kernel of 3×3. After the fourth feature image passes through the pooling module, two fifth feature images and sixth feature images of different sizes are output. Among them, the pooling module is used to reduce the dimension of the feature image, which can increase the receptive field for each layer of the convolution structure while reducing the parameters, thereby preserving the significant features of the feature image.

[0079] As a non-limiting example, the fourth feature image is 15×15, and the pooling module is used to perform two convolution operations on the 15×15 fourth feature image, and upsampling is performed to obtain 8×8 and 5×5 feature images, namely the fifth feature image and the sixth feature image, respectively.

[0080] In some embodiments, the fourth feature image and the fourth feature image after average pooling are used to perform supervised training on the initial model of liveness detection and the initial model of liveness detection is iteratively updated to obtain a trained liveness detection model, including: calculating the loss between the fourth feature image, the fifth feature image, and the sixth feature image and their respective corresponding preset supervisory images according to a loss function, and iteratively updating the weight parameters of the initial model of liveness detection according to the loss to obtain a trained liveness detection model.

[0081] As an implementation method, the fourth feature image, the fifth feature image and the sixth feature image are binarized, and the mean square error (MSE) loss calculation is performed between each pixel of the three binarized images and their corresponding preset supervised binary images, so as to update the weight parameters of the initial model of livingness detection according to the loss calculation results to obtain a model with high recognition accuracy.

[0082] As a non-limiting example, first, the mean square error between the fourth feature image of 15×15, the fifth feature image of 8×8, and the sixth feature image of 5×5 and the preset supervisory image of the same size is calculated. Figure 4 As shown, taking the training of the liveness detection model based on the infrared face image as an example, the liveness detection initial model outputs a 15×15 fourth feature image, and the fourth feature image corresponds to a 15×15 preset supervision image; the 15×15 fourth feature image is averaged and pooled to obtain an 8×8 fifth feature image and a 5×5 sixth feature image, the 8×8 fifth feature image corresponds to an 8×8 preset supervision image, and the 5×5 sixth feature image corresponds to a 5×5 preset supervision image, and the 8×8 preset supervision image and the 5×5 preset supervision image play the role of auxiliary supervision. This embodiment uses three sizes, such as 15×15, 8×8, and 5×5 feature maps for supervision, so that the model accuracy is higher.

[0083] like Figure 5 As shown in the figure, the training of the initial model of liveness detection based on infrared face images is used as an example. The labels of the traditional initial model of liveness detection are 1 for liveness (positive sample) and 0 for pseudo-body (negative sample). In this application, the labels are changed to 15×15 tensors, for example, 15×15 binary images, pseudo depth maps or binary images after face mask, etc. After a large number of experiments, it is proved that the use of Figure 5 The binary image after the face mask is used for supervised training, and the obtained liveness detection model has the highest accuracy. Therefore, in this application, the preset supervision image preferably uses the binary image after the face mask.

[0084] Specifically, the 15×15 fourth feature image is binarized, and the first mean square error between the binarized fourth feature image and the preset 15×15 supervised binary image is calculated; the 8×8 fifth feature image is binarized, and the second mean square error between the binarized fifth feature image and the preset 8×8 supervised binary image is calculated; the 5×5 sixth feature image is binarized, and the third mean square error between the binarized sixth feature image and the preset 5×5 supervised binary image is calculated.

[0085] For example, the first mean square error, the second mean square error, and the third mean square error can be calculated using the following formulas:

[0086]

[0087] Among them, y i represents the pixel value of the i-th pixel in the fourth feature image, the fifth feature image or the sixth feature image after binarization, It represents the pixel value of the i-th pixel in the preset supervised binary image of the same size as the fourth feature image, the fifth feature image or the sixth feature image obtained by using a complex model such as Resnet152, and n represents the number of pixels.

[0088] Then, the loss is calculated according to the first mean square error, the second mean square error and the third mean square error, and the weight parameters of the initial model of liveness detection are updated according to the loss. Specifically, the three calculated mean square errors can be added together according to the weight ratio to calculate the loss, or can be directly added together to calculate the loss.

[0089] The embodiment of the present application uses a complete face image instead of an image block, which makes the inference faster. It uses a lightweight model, which is only 143kb in size after quantization and can be successfully deployed on a chip with low computing power, such as the WQ5007 IoT 3D face recognition SoC chip, which supports up to 32MB of double data rate dynamic random access memory (DDR DRAM). It uses a customized backbone network structure and multiple supervised images to more effectively extract the features of face data, and to capture the difference between a high-precision prosthesis and a real person, so as to obtain a more accurate liveness detection model.

[0090] The trained liveness detection model can be obtained by adopting the aforementioned liveness detection model training method. Specifically, the trained liveness detection model includes at least three convolution modules and a feature fusion module, and no longer includes the pooling module of the initial liveness detection model.

[0091] In some embodiments, the face image used in training the liveness detection model uses a face infrared image, a face color image, and a face depth image. At this time, the liveness detection model training method of the embodiment of the present application is used to obtain a liveness detection model for determining whether the face image is live. Since the liveness detection model is trained with multimodal images, it can be applied to more scenarios and has high robustness. It should be noted that the liveness detection method using the trained liveness detection model of this embodiment can be referred to in the subsequent Figure 6 and Figure 7 The embodiment shown.

[0092] In some other embodiments, the face image used in training the liveness detection model is a face infrared image. In this case, by using the liveness detection model training method of the embodiment of the present application, a liveness detection model for determining whether the face infrared image is live can be obtained, which can be called an infrared liveness detection model.

[0093] In some other embodiments, the face image used in training the liveness detection model adopts a color face image. In this case, by adopting the liveness detection model training method of the embodiment of the present application, a liveness detection model for determining whether the color face image is live can be obtained, which can be called a color liveness detection model.

[0094] In some other embodiments, the face image used in training the liveness detection model adopts the face depth image. In this case, by adopting the liveness detection model training method of the embodiment of the present application, a liveness detection model for judging whether the face depth image is live can be obtained, which can be called a deep liveness detection model.

[0095] It should be noted that the architectures of the infrared liveness detection model, the color liveness detection model, and the depth liveness detection model are the same, namely, the initial model, but the specific weight parameters in the architectures are obtained through training, and the values ​​of the weight parameters may be different.

[0096] When a single modality image, i.e., a face infrared image, a face color image, or a face depth image, is used to train a liveness detection model, a liveness detection result with higher accuracy can be obtained in that modality. Combining liveness detection models corresponding to different modalities can obtain more accurate liveness detection results. It should be noted that the liveness detection method using the trained infrared liveness detection model, color liveness detection model, and depth liveness detection model can be found in the subsequent Figure 8 and Fig. 9 The embodiment shown.

[0097] An embodiment of the present application also provides a liveness detection method, which can be performed by an electronic device. The electronic device is pre-deployed with a trained liveness detection model. The electronic device includes but is not limited to a computer, a tablet computer, a server, a mobile phone, a camera or a wearable device. The server includes but is not limited to an independent server or a cloud server. The trained liveness detection model obtained by the liveness detection model training method is used to perform liveness detection on the face image of the target object.

[0098] Figure 6 FIG. 1 is a schematic diagram of an implementation flow of a liveness detection method provided in an embodiment of the present application. Figure 6 As shown, the living body detection model training method may include steps S610 to S630.

[0099] S610, obtaining an image to be detected of a target object's face.

[0100] The image to be detected may be an infrared image, a color image or a depth image.

[0101] S620: Input the image to be detected into the trained living body detection model to output a feature image.

[0102] Among them, this characteristic image can be compared to the fourth characteristic image mentioned above, and will not be described in detail here.

[0103] As a non-limiting example, the feature image output by the living body detection model may be 15×15.

[0104] S630, binarizing the output feature image, calculating the mean of the binarized feature image, and judging whether the target object is a living body according to the mean.

[0105] The output feature image is binarized and the pixel value of each pixel in the feature image is normalized to 0 or 1.

[0106] In some embodiments, judging whether the target object is alive according to the mean value includes: comparing the mean value with a preset threshold value, and judging whether the target object is alive according to the comparison result.

[0107] As a non-limiting example, the preset threshold value can be any value in the numerical interval [0.5, 0.8], such as 0.5, 0.6 or 0.8, etc., preferably 0.5. For example, if the mean of the feature image after binarization is greater than or equal to 0.5, the target object is determined to be a living body; if the mean of the feature image after binarization is less than 0.5, the target object is determined not to be a living body, that is, a prosthesis. Alternatively, if the mean of the feature image after binarization is greater than 0.5, the target object is determined to be a living body; if the mean of the feature image after binarization is less than or equal to 0.5, the target object is determined to be a prosthesis.

[0108] Figure 7 is a schematic diagram of a liveness detection method according to another embodiment of the present invention. Figure 6 The embodiment shown is further optimized. Figure 7 As shown, the living body detection method may include steps S710 to S730. It should be understood that Figure 7 The embodiment shown and Figure 6 The similarities of the illustrated embodiments will not be repeated here, please refer to the above.

[0109] S710, obtaining the image to be detected of the target object's face. If it is determined that the image to be detected meets the preset quality condition, proceed to step S720, otherwise, re-obtaining the image to be detected of the target object's face until the image to be detected meets the preset quality condition.

[0110] In this embodiment, judging whether the acquired image to be detected meets the preset quality conditions includes but is not limited to: judging whether the head posture is reasonable; judging whether the face is blocked (which can be judged by face contour edge detection of depth image or infrared image); judging whether the lighting is normal (lighting can be judged by the pixel value of infrared image). If any one or more of them fails, the subsequent steps will not be entered directly, that is, the step of "inputting the image to be detected into the trained liveness detection model to output the feature image" and the subsequent steps will not be entered. The image needs to be re-captured until the newly captured image meets the preset quality conditions, then the step of "inputting the image to be detected into the trained liveness detection model to output the feature image" and the subsequent steps will be entered.

[0111] S720: Input the image to be detected into the trained living body detection model to output a feature image.

[0112] Among them, this characteristic image can be compared to the fourth characteristic image mentioned above, and will not be described in detail here.

[0113] As a non-limiting example, the feature image output by the living body detection model may be 15×15.

[0114] S730, binarizing the output feature image, calculating the mean of the binarized feature image, and judging whether the target object is a living body according to the mean.

[0115] In this embodiment, the quality of the images to be detected is screened, and only the images that meet the preset quality conditions are processed for subsequent steps, which can further improve the accuracy of the detection results.

[0116] Figure 8 FIG. 1 is a schematic diagram of an implementation flow of a liveness detection method provided by another embodiment of the present application. Figure 8 As shown, the liveness detection method may include steps S810 to S830. It should be understood that Figure 8 The embodiment shown and Figure 6 The similarities of the illustrated embodiments will not be repeated here, please refer to the above.

[0117] S810, obtaining at least one of an infrared image, a color image, and a depth image of a target object's face as an image to be detected.

[0118] S820, input the image to be detected into a living body detection model corresponding to the image to be detected and output a feature image.

[0119] S830, binarizing the output feature image, calculating the mean of the binarized feature image, and judging whether the target object is a living body according to the mean.

[0120] In this embodiment, the image to be detected may be at least one of an infrared image, a color image, and a depth image. Depending on the acquired image to be detected, a corresponding liveness detection model is used to perform liveness detection to further improve the accuracy of liveness detection.

[0121] As an implementation, specifically, the image to be detected is an infrared image, a color image or a depth image, and the corresponding liveness detection model is an infrared liveness detection model, a color liveness detection model or a depth liveness detection model. At this time, the liveness detection model corresponding to the image to be detected is used to perform liveness detection on the image to be detected.

[0122] In another implementation, specifically, the image to be detected includes an infrared image and a depth image. Corresponding to the image to be detected, an infrared liveness detection model and a depth liveness detection model are used to perform liveness detection.

[0123] at this time, Figure 8 The liveness detection method shown specifically includes: obtaining an infrared image and a depth image of the target object's face; inputting the infrared image into an infrared liveness detection model to output an infrared feature image, and inputting the depth image into a deep liveness detection model to output a depth feature image; binarizing the output infrared feature image and depth feature image respectively, and solving the first mean of the binarized infrared feature image, solving the second mean of the binarized depth feature image, and judging whether the target object is live based on the first mean and the second mean.

[0124] As a non-limiting example, judging whether the target object is a living body according to the first mean and the second mean includes: if the first mean and the second mean are both greater than or equal to a preset threshold, judging that the target object is a living body.

[0125] In another implementation, specifically, the image to be detected includes a color image and a depth image. Corresponding to the image to be detected, a color liveness detection model and a depth liveness detection model are used to perform liveness detection.

[0126] at this time, Figure 8 The liveness detection method shown specifically includes: obtaining a depth image and a color image of the target object's face; inputting the color image into a color liveness detection model to output a color feature image, and inputting the depth image into a deep liveness detection model to output a depth feature image; binarizing the output color feature image and depth feature image respectively, and solving the third mean of the binarized color feature image, solving the fourth mean of the binarized depth feature image, and judging whether the target object is alive based on the third mean and the fourth mean.

[0127] As a non-limiting example, judging whether the target object is a living body according to the third mean and the fourth mean includes: if the third mean and the fourth mean are both greater than or equal to a preset threshold, judging that the target object is a living body.

[0128] In another implementation, specifically, the image to be detected includes a color image, an infrared image, and a depth image. Corresponding to the image to be detected, a color liveness detection model, an infrared liveness detection model, and a depth liveness detection model are used to perform liveness detection.

[0129] at this time, Figure 8 The liveness detection method shown specifically includes: obtaining a color image, an infrared image and a depth image of the target object's face; inputting the color image into a color liveness detection model to output a color feature image, inputting the infrared image into an infrared liveness detection model to output an infrared feature image, and inputting the depth image into a depth liveness detection model to output a depth feature image; binarizing the output color feature image, infrared feature image and depth feature image respectively, and solving the fifth mean of the binarized color feature image, solving the sixth mean of the binarized infrared feature image, and solving the seventh mean of the binarized depth feature image, and judging whether the target object is live based on the fifth mean, the sixth mean and the seventh mean.

[0130] As a non-limiting example, judging whether the target object is alive according to the fifth mean, the sixth mean and the seventh mean includes: if the fifth mean, the sixth mean and the seventh mean are all greater than or equal to a preset threshold, judging that the target object is alive.

[0131] Fig. 9 is a schematic diagram of a liveness detection method according to another embodiment of the present invention. Figure 8 The embodiment shown is further optimized. Fig. 9 As shown, the living body detection method may include steps S910 to S930. It should be understood that Fig. 9 The embodiment shown and Figure 7 and Figure 8 The similarities of the illustrated embodiments will not be repeated here, please refer to the above.

[0132] S910, obtaining at least one of the infrared image, color image and depth image of the target object's face as the image to be detected; if the images to be detected all meet the preset quality conditions, proceed to step S920, otherwise re-obtaining the image to be detected of the target object's face until the image to be detected meets the preset quality conditions.

[0133] S920, input the image to be detected into a living body detection model corresponding to the image to be detected and output a feature image.

[0134] S930, binarizing the output feature image, calculating the mean of the binarized feature image, and judging whether the target object is a living body according to the mean.

[0135] Figure 8 and Fig. 9 In the embodiment shown, three liveness detection models based on three single-modal data, namely infrared, color and depth data, are trained based on the liveness detection model training method based on the whole face, i.e., infrared liveness detection model, color liveness detection model and deep liveness detection model, to perform liveness detection. Among them, the deep liveness detection model is used to exclude attacks such as videos, printed paper photos, and photos with holes and bends; the infrared or color liveness detection model is used to prevent attacks such as 3D head models, masks, and facial masks. Only when the liveness detection of two or three of the models is completely passed, it is considered to be a real live body, which greatly improves the accuracy of the liveness detection algorithm.

[0136] After testing, the deep liveness detection model provided by the embodiment of the present application has a real person pass rate of more than 99.9%, and can effectively prevent more than 99.5% of paper attacks or paper hole attacks. The infrared and color liveness detection model provided by the embodiment of the present application has a real person pass rate of more than 99.9%, and can prevent more than 95% of high-precision 3D head models or resin mask attacks.

[0137] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0138] An embodiment of the present application also provides a liveness detection model training device. For details not described in detail in the liveness detection model training device, please refer to the description in the above liveness detection model training method embodiment.

[0139] See also Fig.10 , Fig.10 1 is a schematic block diagram of a liveness detection model training device provided in an embodiment of the present application. The liveness detection model training device comprises: a construction module 101, an acquisition module 102, a convolution execution module 103, a feature fusion execution module 104 and a pooling execution module 105.

[0140] Among them, the construction module 101 is used to construct an initial model for liveness detection, and the initial model for liveness detection includes at least three convolution modules, a feature fusion module and a pooling module;

[0141] An acquisition module 102 is used to acquire a face image;

[0142] A convolution execution module 103, used to obtain at least three feature images when the face image passes through each convolution module in sequence;

[0143] A feature fusion execution module 104, configured to use the feature fusion module to fuse the at least three feature images to obtain a fourth feature image;

[0144] The pooling execution module 105 is used to use the pooling module to pool the fourth feature image, and use the fourth feature image and the pooled fourth feature image to supervise the training of the initial model of liveness detection and iteratively update the initial model of liveness detection to obtain a trained liveness detection model.

[0145] In some embodiments, the acquisition module 102 is specifically used to:

[0146] A face image is acquired, and up-sampled or down-sampled to obtain a target image.

[0147] The convolution execution module 103 is specifically used for:

[0148] The at least three convolution modules of the initial model for liveness detection are used to perform feature learning on the target image to obtain at least three feature images.

[0149] In some embodiments, the pooling execution module 105 is specifically used to:

[0150] Using the pooling module to perform average pooling on the fourth feature image to obtain a fifth feature image and a sixth feature image;

[0151] The losses between the fourth feature image, the fifth feature image and the sixth feature image and their corresponding preset supervisory images are calculated according to the loss function, and the weight parameters of the initial model of liveness detection are iteratively updated according to the losses to obtain a trained liveness detection model.

[0152] In some embodiments, the loss function is a mean square error loss function.

[0153] In some embodiments, the fourth characteristic image, the fifth characteristic image, and the sixth characteristic image have different sizes.

[0154] An embodiment of the present application also provides a liveness detection device. For details not described in detail in the liveness detection device, please refer to the description in the above liveness detection method embodiment.

[0155] See also Fig.11 , Fig.11 1 is a schematic block diagram of a door lock system based on liveness detection provided by an embodiment of the present application. The liveness detection device comprises: a camera module 110, a to-be-detected image acquisition module 111, a feature image acquisition module 112 and a judgment module 113, wherein:

[0156] A camera module, used for collecting an image of a target object to be detected;

[0157] The to-be-detected image acquisition module 111 is used to acquire the to-be-detected image of the target object's face;

[0158] The feature image acquisition module 112 is used to input the image to be detected into the trained liveness detection model obtained by the liveness detection model training method described above, and output a feature image;

[0159] The judgment module 113 is used to binarize the feature image and calculate the mean of the binarized feature image, and judge whether the target object is a living body according to the mean to perform identity authentication.

[0160] Furthermore, if the target object is determined to be alive, the facial information on the target object is identified and compared with the registered facial information preset on the cloud. If the facial information on the target object is consistent with the facial information on the cloud, the door lock can be opened; otherwise, the door lock will not be opened.

[0161] It should be noted that the camera module 110 can be at least one or more of a depth camera, an infrared camera, and a color camera, etc., which is not limited here.

[0162] In some embodiments, the feature image acquisition module 112 is specifically used to:

[0163] If it is determined that the image to be detected meets the preset quality condition, the image to be detected is input into a trained liveness detection model obtained by using the liveness detection model training method as in the above-mentioned embodiment, and a feature image is output.

[0164] In some embodiments, the image to be detected includes at least one of an infrared image, a color image, and a depth image of the target object's face.

[0165] In these embodiments, the feature image acquisition module 112 is specifically used for:

[0166] Input the image to be detected into a liveness detection model corresponding to the image to be detected and obtained by the liveness detection model training method of the above embodiment, and output a feature image; or

[0167] If it is determined that the image to be detected meets the preset quality condition, the image to be detected is input into the liveness detection model corresponding to the image to be detected, and the trained liveness detection model obtained by the liveness detection model training method of the above embodiment is used to output a feature image.

[0168] As an implementation method, specifically, the image to be detected is an infrared image, a color image or a depth image of the target object's face, and the corresponding liveness detection model is an infrared liveness detection model, a color liveness detection model or a depth liveness detection model. At this time, the liveness detection model corresponding to the image to be detected is used to perform liveness detection on the image to be detected.

[0169] As an implementation, specifically, the image to be detected is an infrared image and a depth image of the face of the target object. Corresponding to the image to be detected, an infrared liveness detection model and a depth liveness detection model are used to perform liveness detection.

[0170] In this implementation, the feature image acquisition module 112 is specifically used to:

[0171] Obtain an infrared image and a depth image of the target object's face; input the infrared image into an infrared liveness detection model to output an infrared feature image, and input the depth image into a deep liveness detection model to output a deep feature image.

[0172] The determination module 113 is specifically used for:

[0173] The output infrared feature image and depth feature image are binarized respectively, and the first mean of the binarized infrared feature image is solved, and the second mean of the binarized depth feature image is solved, and whether the target object is a living body is determined according to the first mean and the second mean.

[0174] In another implementation, specifically, the image to be detected includes a color image and a depth image. Corresponding to the image to be detected, a color liveness detection model and a depth liveness detection model are used to perform liveness detection.

[0175] In this implementation, the feature image acquisition module 112 is specifically used to:

[0176] The color image is input into the color living body detection model to output the color feature image, and the depth image is input into the depth living body detection model to output the depth feature image.

[0177] The determination module 113 is specifically used for:

[0178] The output color feature image and depth feature image are binarized respectively, and the third mean of the binarized color feature image is solved, and the fourth mean of the binarized depth feature image is solved, and whether the target object is alive is judged according to the third mean and the fourth mean.

[0179] In another implementation, specifically, the image to be detected includes a color image, an infrared image, and a depth image. Corresponding to the image to be detected, a color liveness detection model, an infrared liveness detection model, and a depth liveness detection model are used to perform liveness detection.

[0180] In this implementation, the feature image acquisition module 112 is specifically used to:

[0181] The color image is input into the color liveness detection model to output a color feature image, the infrared image is input into the infrared liveness detection model to output an infrared feature image, and the depth image is input into the depth liveness detection model to output a depth feature image.

[0182] The determination module 113 is specifically used for:

[0183] The output color feature image, infrared feature image and depth feature image are binarized respectively, and the fifth mean of the binarized color feature image is solved, the sixth mean of the binarized infrared feature image is solved, and the seventh mean of the binarized depth feature image is solved. Whether the target object is alive is judged based on the fifth mean, the sixth mean and the seventh mean.

[0184] An embodiment of the present application also provides an electronic device, such as Fig.12 As shown, the electronic device may include one or more processors 120 ( Fig.12 Only one is shown), a memory 121 and a computer program 122 stored in the memory 121 and executable on one or more processors 120, for example, a program for training a liveness detection model and / or a program for liveness detection. When one or more processors 120 execute the computer program 122, the various steps in the liveness detection model training method and / or the liveness detection method embodiment can be implemented. Alternatively, when one or more processors 120 execute the computer program 122, the functions of the various modules / units in the liveness detection model training device and / or the liveness detection device embodiment can be implemented, which is not limited here.

[0185] Those skilled in the art will understand that Fig.12 The electronic device is merely an example of an electronic device and does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0186] In one embodiment, the processor 120 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc.

[0187] In one embodiment, the memory 121 may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. The memory 121 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Further, the memory 121 may also include both an internal storage unit of the electronic device and an external storage device. The memory 121 is used to store computer programs and other programs and data required by the electronic device. The memory 121 may also be used to temporarily store data that has been output or is to be output.

[0188] The technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In practical applications, the above-mentioned function allocation can be completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated in a processing unit, or each unit can exist physically separately, or two or more units can be integrated in one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, which will not be repeated here.

[0189] An embodiment of the present application further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it can implement the steps in the liveness detection model training method and / or the liveness detection method embodiment.

[0190] An embodiment of the present application provides a computer program product. When the computer program product runs on an electronic device, the electronic device can implement the steps in the liveness detection model training method and / or the liveness detection method embodiment.

[0191] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0192] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0193] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0194] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0195] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0196] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and the computer program can implement the steps of the above-mentioned various method embodiments when executed by the processor. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying computer program code, recording medium, U disk, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electric carrier signals and telecommunication signals.

[0197] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application, and should all be included in the protection scope of the present application.

Claims

1. A method for training a liveness detection model, characterized in that: include: Constructing an initial model for liveness detection, wherein the initial model for liveness detection includes at least three convolution modules, a feature fusion module, and a pooling module; Acquire a face image, and obtain at least three feature images when the face image passes through each convolution module in sequence; Using the feature fusion module to fuse the at least three feature images to obtain a fourth feature image; The fourth feature image is pooled by using the pooling module to reduce its dimension, and the fourth feature image and two pooled fourth feature images of different sizes are used to perform supervised training on the initial model of liveness detection and iteratively update the initial model of liveness detection to obtain a trained liveness detection model.

2. The method for training a liveness detection model according to claim 1, wherein: The step of acquiring a facial image and obtaining at least three feature images when the facial image passes through each convolution module in sequence comprises: Acquire a face image, and upsample or downsample the face image to obtain a target image; The at least three convolution modules of the initial model for liveness detection are used to perform feature learning on the target image to obtain at least three feature images.

3. The method for training a liveness detection model according to claim 1 or 2, wherein: The method of using the pooling module to pool the fourth feature image, and using the fourth feature image and the pooled fourth feature image to perform supervised training on the initial model for liveness detection and iteratively updating the initial model for liveness detection to obtain a trained liveness detection model includes: Using the pooling module to perform average pooling on the fourth feature image to obtain a fifth feature image and a sixth feature image; The losses between the fourth feature image, the fifth feature image and the sixth feature image and their corresponding preset supervisory images are calculated according to the loss function, and the weight parameters of the initial model of liveness detection are iteratively updated according to the losses to obtain a trained liveness detection model.

4. The method for training a liveness detection model according to claim 3, wherein: The loss function is a mean square error loss function; the fourth feature image, the fifth feature image and the sixth feature image have different sizes.

5. A method for detecting a living body, characterized in that: include: Obtain the image to be detected of the target object's face; Inputting the image to be detected into a trained liveness detection model obtained by the liveness detection model training method according to any one of claims 1 to 4, and outputting a feature image; The feature image is binarized, and the mean of the binarized feature image is solved, and whether the target object is a living body is determined according to the mean.

6. The living body detection method according to claim 5, characterized in that: The step of inputting the image to be detected into a trained liveness detection model obtained by the liveness detection model training method according to any one of claims 1 to 4, and outputting a feature image, comprises: If it is determined that the image to be detected meets the preset quality condition, the image to be detected is input into a trained liveness detection model obtained by the liveness detection model training method according to any one of claims 1 to 4, and a feature image is output.

7. The living body detection method according to claim 5 or 6, characterized in that: The image to be detected includes at least one of an infrared image, a color image, and a depth image of the face of the target object; The step of inputting the image to be detected into a trained liveness detection model obtained by the liveness detection model training method according to any one of claims 1 to 4, and outputting a feature image, comprises: The image to be detected is input into a liveness detection model corresponding to the image to be detected and obtained by using the liveness detection model training method as described in any one of claims 1 to 4, and a feature image is output.

8. A liveness detection model training device, characterized in that: include: A construction module, used to construct an initial model for liveness detection, wherein the initial model for liveness detection includes at least three convolution modules, a feature fusion module and a pooling module; An acquisition module, used for acquiring a face image; A convolution execution module, used to obtain at least three feature images when the face image passes through each convolution module in sequence; a feature fusion execution module, configured to use the feature fusion module to fuse the at least three feature images to obtain a fourth feature image; A pooling execution module is used to use the pooling module to pool the fourth feature image for dimensionality reduction, and use the fourth feature image and two fourth feature images after pooling of different sizes to supervise the training of the initial model of liveness detection and iteratively update the initial model of liveness detection to obtain a trained liveness detection model.

9. A door lock system based on liveness detection, characterized in that: include: A camera module, used for collecting an image of a target object to be detected; A module for acquiring an image to be detected, used to acquire the image to be detected of the target object's face; A feature image acquisition module, used for inputting the image to be detected into a trained liveness detection model obtained by the liveness detection model training method according to any one of claims 1 to 4, and outputting a feature image; The judgment module is used to binarize the feature image and calculate the mean of the binarized feature image, and judge whether the target object is a living body according to the mean to perform identity authentication.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements the liveness detection model training method as described in any one of claims 1 to 4, or implements the liveness detection method as described in any one of claims 5 to 7.

11. A computer storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, it implements the liveness detection model training method as described in any one of claims 1 to 4, or implements the liveness detection method as described in any one of claims 5 to 7.

Images