Human face liveness detection method, system, device and storage medium

Abstract

The present invention provides a method, system, device and storage medium for face liveness detection, the method comprising: collecting training sample images; inputting the training sample images into a face liveness detection model, the face liveness detection model comprising a feature extraction network, a Fourier supervision branch, a deep supervision branch and a liveness classification branch; respectively calculating the loss functions of the Fourier supervision branch, the deep supervision branch and the liveness classification branch, and training the face liveness detection model according to the loss functions; inputting the image to be detected into the trained face liveness detection model to obtain an output liveness classification result, the liveness classification result comprising at least one detected liveness attribute category. The present invention provides a simple auxiliary supervision signal to supervise the training of the liveness detection model, thereby improving the liveness detection effect.

Description

Background Art

[0002] Face liveness detection is a method to determine the real physiological characteristics of the object in some identity verification scenarios. In face recognition applications, liveness detection can verify whether the user is the real live person through combined actions such as blinking, opening the mouth, shaking the head, and nodding, using technologies such as face key point positioning and face tracking. It can effectively resist common attack methods such as photos, face swapping, masks, occlusion, and screen reshoots, thereby helping users identify fraudulent behavior and protecting their interests.

[0003] In the prior art, there has been a method for human activity detection based on the CNN-RNN (Convolutional Neural Network-Recurrent Neural Network) model, which uses depth maps and rPPG (Remote Photoplethysmography) signals for auxiliary supervision in the spatial and temporal domains. This method is mainly reflected in the paper "Learning Deep Models for Face Anti-Spoofing: Binary or Auxiliary Supervision". However, this method also has some problems. It requires video as an input signal to establish an rPPG signal, and the rPPG signal is easily interfered in an open environment. Compared with pictures, the amount of video data is too large, which will greatly increase the processing time. In addition, this method uses real depth maps for supervision, which will result in excessively high sample costs, limit the scale of training samples, and thus affect the effect of model training. Summary of the invention

[0004] In view of the problems in the prior art, the purpose of the present invention is to provide a face liveness detection method, system, device and storage medium, provide a simple auxiliary supervision signal, supervise the training of the liveness detection model, and improve the liveness detection effect.

[0005] The embodiment of the present invention provides a method for detecting liveness of a face, comprising the following steps:

[0006] Collect training sample images;

[0007] Inputting the training sample image into a face liveness detection model, wherein the face liveness detection model includes a feature extraction network, a Fourier supervision branch, a deep supervision branch, and a liveness classification branch;

[0008] Calculating the loss functions of the Fourier supervision branch, the depth supervision branch and the living body classification branch respectively, and training the face living body detection model according to the loss functions;

[0009] The image to be detected is input into the trained face liveness detection model to obtain an output liveness classification result, wherein the liveness classification result includes at least one detected liveness attribute category.

[0010] In some embodiments, after the training sample image is input into the Fourier supervision branch, the Fourier supervision branch performs Fourier transform and normalization on the training sample image, and then adjusts the size to a preset size.

[0011] In some embodiments, after the training sample image is input into the depth supervision branch, the depth supervision branch uses PRNet to generate a pseudo depth map of a preset size for the negative sample image, and uses an image of a preset size with all pixel values ​​of 0 as a depth map for the positive sample image.

[0012] In some embodiments, the feature extraction network adopts a residual network and a pyramid network, the residual network is a resnet18 network or a resnet34 network, and the output of the pyramid network is respectively connected to the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch.

[0013] In some embodiments, respectively calculating the loss functions of the Fourier supervision branch, the deep supervision branch, and the living body classification branch comprises the following steps:

[0014] Calculate the Fourier supervision branch loss function according to the output of the first output auxiliary supervision branch and the output of the Fourier supervision branch;

[0015] Calculate the deep supervision branch loss function according to the output of the second output auxiliary supervision branch and the output of the deep supervision branch;

[0016] The output of the third output auxiliary supervision branch is input into the living body classification branch to obtain a classification result of the living body classification branch;

[0017] The living body classification branch loss function is calculated according to the classification result of the living body classification branch and the label of the training sample image.

[0018] In some embodiments, the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch adopt the same structure, and the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch respectively include convolutional layers.

[0019] In some embodiments, training the face liveness detection model according to the loss function comprises the following steps:

[0020] The loss function of the face liveness detection model is obtained by weighted summing the Fourier supervision branch loss function, the depth supervision branch loss function and the liveness classification branch loss function;

[0021] The face liveness detection model is iteratively trained by adopting a stochastic gradient descent method.

[0022] In some embodiments, the Fourier supervision branch loss function adopts the MES loss function, and the deep supervision loss function adopts the CLD loss function.

[0023] In some embodiments, the classification branch loss function includes a face attribute loss function, a spoofing type loss function, an illumination loss function, an environment loss function, and a positive and negative classification loss function.

[0024] In some embodiments, the facial attribute loss function adopts the BCE loss function, the deception type loss function, the illumination loss function and the environment loss function respectively adopt the CrossEntropy loss function, and the positive and negative classification loss function adopts the arcface loss function.

[0025] The face liveness detection method of the present invention generates a Fourier spectrum and a depth map respectively by adding a Fourier supervision branch and a depth supervision branch, thereby supervising the liveness classification branch, providing a simple auxiliary supervision signal, supervising the training of the liveness detection model, and improving the liveness detection effect. The present invention does not need to use video as an input signal to establish an rPPG signal, is not easily affected by the environment, and improves the stability and accuracy of detection.

[0026] The embodiment of the present invention further provides a face liveness detection system for implementing the face liveness detection method, characterized in that the system comprises:

[0027] A sample collection module, used for collecting training sample images;

[0028] A model input module, used to input a training sample image into a face liveness detection model, wherein the face liveness detection model includes a feature extraction network, a Fourier supervision branch, a deep supervision branch, and a liveness classification branch;

[0029] A model training module, used to calculate the loss functions of the Fourier supervision branch, the deep supervision branch and the living body classification branch respectively, and train the face liveness detection model according to the loss functions;

[0030] The liveness detection module is used to input the image to be detected into the trained face liveness detection model to obtain an output liveness classification result, wherein the liveness classification result includes at least one detected liveness attribute category.

[0031] The face liveness detection system of the present invention generates a Fourier spectrum and a depth map respectively by adding a Fourier supervision branch and a depth supervision branch, thereby supervising the liveness classification branch, providing a simple auxiliary supervision signal to supervise the training of the liveness detection model and improving the liveness detection effect. The present invention does not need to use video as an input signal to establish an rPPG signal, is not easily affected by the environment, and improves the stability and accuracy of detection.

[0032] The embodiment of the present invention further provides a face liveness detection device, comprising:

[0033] processor;

[0034] a memory storing executable instructions of the processor;

[0035] Wherein, the processor is configured to perform the steps of the face liveness detection method by executing the executable instructions.

[0036] When the processor of the face liveness detection device of the present invention executes the executable instructions in the memory, the steps of the face liveness detection method are implemented. Therefore, the face liveness detection device can also achieve the technical effects of the above-mentioned face liveness detection method.

[0037] An embodiment of the present invention further provides a computer-readable storage medium for storing a program, wherein the program implements the steps of the method for live face detection when executed by a processor.

[0038] When the executable instructions in the computer-readable storage medium of the present invention are executed by the processor, the steps of the face liveness detection method are implemented. Therefore, the computer-readable storage medium can also achieve the technical effect of the above-mentioned face liveness detection method.

[0039] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Other features, objectives and advantages of the present invention will become more apparent from a reading of the detailed description of non-limiting embodiments made with reference to the following accompanying drawings.

[0041] Figure 1 is a flow chart of a method for detecting live faces according to an embodiment of the present invention;

[0042] Figure 2 is a schematic diagram of a face liveness detection model according to an embodiment of the present invention;

[0043] Figure 3 2 is a schematic diagram of a pyramid layer output auxiliary supervision branch structure according to an embodiment of the present invention;

[0044] Figure 4 is a schematic diagram of a living body classification branch according to an embodiment of the present invention;

[0045] Figure 5 is a structural schematic diagram of a face liveness detection system according to an embodiment of the present invention;

[0046] Figure 6 is a structural schematic diagram of a face liveness detection device according to an embodiment of the present invention;

[0047] Figure 7 It is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. DETAILED DESCRIPTION

[0048] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0049] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0050] like Figure 1 As shown, an embodiment of the present invention provides a method for detecting liveness of a face, comprising the following steps:

[0051] S100: Acquire a training sample image, wherein the training sample image has a mark of whether it is a living body;

[0052] S200: Inputting a training sample image into a face liveness detection model, wherein the face liveness detection model includes a feature extraction network, a Fourier supervision branch, a deep supervision branch, and a liveness classification branch;

[0053] S300: Calculating the loss functions of the Fourier supervision branch, the depth supervision branch and the living body classification branch respectively, and training the face living body detection model according to the loss functions;

[0054] S400: Input the image to be detected into the trained face liveness detection model to obtain an output liveness classification result, wherein the liveness classification result includes at least one detected liveness attribute category, and the liveness attribute category may include, for example, a classification category of whether the liveness is alive or not, and may further increase the classification category of the liveness gender, the classification category of the liveness age segment, etc. as needed. If the liveness is not alive, it may be classified according to the attack type, etc.

[0055] The face liveness detection method of the present invention trains a face liveness detection model through steps S100 to S300, and adds a Fourier supervision branch and a depth supervision branch to the face liveness detection model to generate a Fourier spectrum and a depth map respectively, thereby supervising the liveness classification branch, providing a simple auxiliary supervision signal to supervise the training of the liveness detection model, and improving the effect of liveness detection in the image to be detected through step S400. The present invention does not need to use video as an input signal to establish an rPPG signal, is not easily affected by the environment, and improves the stability and accuracy of detection.

[0056] In this embodiment, the training sample images are collected in step S100, and the CelebA-Spoof dataset is used. There are mainly two types of images in the dataset, the spoof image represents a non-living image, and the live image represents a living image. CelebA-Spoof has a wide range of attack types, which are divided into four major attack types, namely Print (photos, posters, A4 paper), PaperCut (paper faces only, paper faces with upper bodies, and paper faces with different facial features removed), Replay (faces on screens such as computers, tablets, and mobile phones), and 3DMask (mask type), and data collection is performed in accordance with the above three dimensions under a variety of lighting conditions (normal light, strong light, backlight, and dark light) and environments (indoors and outdoors). Therefore, CelebA-Spoof has many advantages such as large data volume, rich and diverse data, and rich annotation information, and can be well applied to the training of face liveness detection models.

[0057] like Figure 2As shown, the face liveness detection model includes a feature extraction network, a Fourier supervision branch, a deep supervision branch and a liveness classification branch. The feature maps extracted by the feature extraction network are used to calculate the loss functions of the corresponding branches with the Fourier supervision branch and the deep supervision branch, respectively, and the feature maps extracted by the feature extraction network can be used for liveness classification branch to perform liveness detection classification. The Fourier supervision branch includes Fourier transform, normalization and size scaling. Specifically, after the training sample image is input into the Fourier supervision branch, the Fourier supervision branch performs Fourier transform on the training sample image to generate a Fourier spectrum, and then adjusts the size to a preset size, such as 224*224, after centering and normalization. When calculating the loss function of the Fourier supervision branch, the loss function is calculated based on the Fourier spectrum and the feature map extracted by the feature extraction network.

[0058] In this embodiment, after the training sample image is input into the deep supervision branch, the deep supervision branch uses PRNet to generate a pseudo depth map of a preset size for the negative sample image, and the pseudo depth map has the same size as the input training sample image, for example, scaled to 224*224, and uses an image of a preset size with all pixel values ​​of 0 as the depth map for the positive sample image, and the depth map has the same size as the input training sample image, for example, scaled to 224*224. When calculating the loss function of the deep supervision branch, the loss function is calculated based on the pseudo depth map or the depth map and the feature map extracted by the feature extraction network. When the training sample image uses the CelebA-Spoof dataset, the negative sample image is the spoof image, and the positive sample image is the live image.

[0059] The output of the feature extraction network is connected to the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch respectively. In this embodiment, the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch adopt the same structure, and the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch respectively include a convolution layer. Figure 3 The pyramid layer output auxiliary supervision branch network connected to the feature extraction network is shown. The Fourier supervision branch, the deep supervision branch, and the living body classification branch each use a Figure 3 The network structure of the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch are respectively Figure 3The size of the pyramid output feature map is shown in Table 1. For the auxiliary supervision branch (Fourier supervision branch and deep supervision branch), the feature map is finally cat and reshaped to B*224*224. For the living body classification branch, it is finally cat and passed through a k=7*7 convolution layer, a BN (Batch Normalization) layer and an activation layer (activation function layer). The output channel dimension is the number of categories 2. Figure 3 In the figure, A represents the feature map, B represents the image with stride=2, k=3*3, and C represents the image with stride=1, k=3*3.

[0060] Table 1 Feature pyramid output feature map size

[0061] P3 B*256*56*56 B*256*7*7 P4 B*256*28*28 B*256*7*7 P5 B*256*14*14 B*256*7*7 P6 B*256*7*7 B*256*7*7

[0062] like Figure 4 As shown, the feature extraction network adopts resnet34 network and four-layer pyramid network BiFPN, and the output of the pyramid network BiFPN is respectively connected to the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch. In other embodiments, the resnet34 network can also be changed to resnet18 network. Resnet network is a residual network. The residual network is mainly composed of residual blocks. After the residual blocks are introduced into the deep learning network, the network can reach a very deep, the effect of the network also becomes better, and the image recognition accuracy is very high. The Resnet18 network includes 17 convolutional layers and a fully connected layer. In the single-layer object detection algorithm, it is impossible to use a single layer of feature maps to effectively characterize objects of various scales at the same time. Therefore, the object detection algorithm later gradually developed into using feature maps of different layers to form a feature pyramid network (FPN, Feature Parymid Network), characterize objects of different layers, and then perform object detection based on the feature pyramid. BiFPN is a feature pyramid network with good bidirectional fusion effect.

[0063] In this embodiment, in step S300, respectively calculating the loss functions of the Fourier supervision branch, the deep supervision branch and the living body classification branch includes the following steps:

[0064] Calculate the Fourier supervision branch loss function according to the output feature map of the first output auxiliary supervision branch and the output Fourier spectrum map of the Fourier supervision branch;

[0065] Calculate the depth supervision branch loss function according to the output feature map of the second output auxiliary supervision branch and the output depth map of the depth supervision branch;

[0066] The output feature map of the third output auxiliary supervision branch is input into the living body classification branch to obtain a classification result of the living body classification branch;

[0067] The living body classification branch loss function is calculated according to the classification result of the living body classification branch and the classification label pre-marked on the training sample image.

[0068] In this embodiment, in step S300, training the face liveness detection model according to the loss function includes the following steps:

[0069] The loss function of the face liveness detection model is obtained by weighted summing the Fourier supervision branch loss function, the depth supervision branch loss function and the liveness classification branch loss function;

[0070] The face liveness detection model is iteratively trained by adopting a stochastic gradient descent method.

[0071] In this embodiment, the Fourier supervision branch loss function adopts the MES loss function, and the weight is, for example, 0.8, and the deep supervision loss function adopts the CLD loss function, and the weight is, for example, 0.8. The classification branch loss function includes a face attribute loss function, a spoof type loss function, an illumination loss function, an environment loss function, and a positive and negative classification loss function. Specifically, the face attribute loss function adopts the BCE loss function, and the weight is, for example, 0.5, the spoof type loss function, the illumination loss function, and the environment loss function respectively adopt the CrossEntropy loss function, and the weights are, for example, 0.5, 0.1, and 0.01, respectively, and the positive and negative classification loss function adopts the arcface loss function, and the weight is, for example, 1.0. The weight values ​​here are for example only. In other alternative implementations, the weights may also be other values ​​and may be adjusted as needed.

[0072] The loss function is a cross entropy function, the optimizer uses stochastic gradient descent (SGD), lr is 1e-1, weight_decay is 5e-4, momentum is 0.9, training uses P100, batchsize is 128, and epoch is 100. The types of loss functions selected, the classification of classification branch loss functions, the types of optimizers, etc. listed here are all examples. In other alternative implementations, they can be adjusted and adopted in different ways from those listed above, all of which belong to the protection scope of the present invention.

[0073] like Figure 5As shown, the embodiment of the present invention further provides a face liveness detection system for implementing the face liveness detection method, characterized in that the system comprises:

[0074] The sample acquisition module M100 is used to acquire training sample images, where the training sample images have a mark of whether they are alive or not;

[0075] A model input module M200 is used to input a training sample image into a face liveness detection model, wherein the face liveness detection model includes a feature extraction network, a Fourier supervision branch, a deep supervision branch, and a liveness classification branch;

[0076] The model training module M300 is used to calculate the loss functions of the Fourier supervision branch, the deep supervision branch and the living body classification branch respectively, and train the face liveness detection model according to the loss functions;

[0077] The liveness detection module M400 is used to input the image to be detected into the trained face liveness detection model to obtain an output liveness classification result, wherein the liveness classification result includes at least one detected liveness attribute category, wherein the liveness attribute category may include, for example, a classification category of whether the liveness is alive or not, and may further increase the classification category of liveness gender, the classification category of liveness age segment, etc. as needed.

[0078] The face liveness detection system of the present invention trains a face liveness detection model through a sample acquisition module M100, a model input module M200 and a model training module M300, and adds a Fourier supervision branch and a depth supervision branch to the face liveness detection model to generate a Fourier spectrum and a depth map respectively, thereby supervising the liveness classification branch, providing a simple auxiliary supervision signal to supervise the training of the liveness detection model, and improving the effect of liveness detection in the image to be detected by the liveness detection module M400. The present invention does not need to use video as an input signal to establish an rPPG signal, is not easily affected by the environment, and improves the stability and accuracy of detection.

[0079] In the face liveness detection system of the present invention, the functions of each module can be implemented by the specific implementation of the face liveness detection method as described above. For example, the sample acquisition module M100 can use images in the CelebA-Spoof data set as training sample images. There are mainly two types of images in this data set, the spoof image represents a non-live image, and the live image represents a live image. In addition, CelebA-Spoof has many advantages such as large data volume, rich and diverse data, and rich annotation information, and can be well applied to the training of face liveness detection models.

[0080] The face liveness detection model can be used as follows Figure 2 to Figure 4The network structure shown. Among them, the face liveness detection model includes a feature extraction network, a Fourier supervision branch, a deep supervision branch and a liveness classification branch. The feature map extracted by the feature extraction network is used to calculate the loss function of the corresponding branch with the Fourier supervision branch and the deep supervision branch, and the feature map extracted by the feature extraction network can be used for the liveness classification branch to perform liveness detection classification. The Fourier supervision branch includes Fourier transform, normalization and size scaling. Specifically, after the training sample image is input into the Fourier supervision branch, the Fourier supervision branch performs Fourier transform on the training sample image to generate a Fourier spectrum, and then adjusts the size to a preset size after centering and normalization. After the training sample image is input into the deep supervision branch, the deep supervision branch uses PRNet to generate a pseudo depth map of a preset size for the negative sample image, and the pseudo depth map is the same size as the input training sample image, and uses a preset size of the image with all pixel values ​​of 0 as the depth map for the positive sample image, and the depth map is the same size as the input training sample image.

[0081] The output of the feature extraction network is respectively connected to the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch. In this embodiment, the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch adopt the same structure, and the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch respectively include convolution layers. The first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch can respectively adopt the following structure: Figure 3 The network structure shown in the figure. The feature extraction network adopts a resnet18 network or a resnet34 network and a four-layer pyramid network BiFPN, and the output of the pyramid network BiFPN is respectively connected to the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch.

[0082] The model training module M300 can use the specific implementation of the above step S300 to calculate the loss function of the face liveness detection model, and iteratively optimize and train the face liveness detection model. Specifically, the Fourier supervision branch loss function is calculated based on the output feature map of the first output auxiliary supervision branch and the output Fourier spectrum map of the Fourier supervision branch; the deep supervision branch loss function is calculated based on the output feature map of the second output auxiliary supervision branch and the output depth map of the deep supervision branch; the output feature map of the third output auxiliary supervision branch is input into the liveness classification branch to obtain the classification result of the liveness classification branch; the liveness classification branch loss function is calculated based on the classification result of the liveness classification branch and the classification label pre-marked by the training sample image; the Fourier supervision branch loss function, the deep supervision branch loss function and the liveness classification branch loss function are weighted and summed to obtain the loss function of the face liveness detection model; the face liveness detection model is iteratively trained using stochastic gradient descent.

[0083] An embodiment of the present invention also provides a face liveness detection device, comprising a processor; a memory, in which executable instructions of the processor are stored; wherein the processor is configured to execute the steps of the face liveness detection method by executing the executable instructions.

[0084] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as systems, methods or program products. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits", "modules" or "platforms".

[0085] Refer to the following Figure 6 The electronic device 600 according to this embodiment of the present invention is described. Figure 6 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0086] like Figure 6 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0087] The storage unit stores a program code, which can be executed by the processing unit 610, so that the processing unit 610 performs the steps of various exemplary embodiments of the present invention described in the above-mentioned face liveness detection method section of this specification. For example, the processing unit 610 can perform the following steps: Figure 1 Follow the steps shown in .

[0088] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0089] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0090] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0091] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 650. Furthermore, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) via a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 via a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0092] When the processor of the face liveness detection device of the present invention executes the executable instructions in the memory, the steps of the face liveness detection method are implemented, so the face liveness detection device can also achieve the technical effect of the above-mentioned face liveness detection method. That is, the face liveness detection model is trained through steps S100 to S300, and Fourier supervision branches and depth supervision branches are added to the face liveness detection model to generate Fourier spectra and depth maps respectively, thereby supervising the liveness classification branch, providing a simple auxiliary supervision signal to supervise the training of the liveness detection model, and improving the effect of liveness detection in the image to be detected through step S400. The present invention does not need to use video as an input signal to establish an rPPG signal, and is not easily affected by the environment, thereby improving the stability and accuracy of the detection.

[0093] The embodiment of the present invention also provides a computer-readable storage medium for storing a program, which implements the steps of the face liveness detection method when executed by a processor. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is executed on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the above face liveness detection method section of this specification.

[0094] refer to Figure 7 As shown, a program product 800 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be executed on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0095] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0096] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0097] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0098] When the executable instructions in the computer-readable storage medium of the present invention are executed by the processor, the steps of the face liveness detection method are implemented, and therefore, the computer-readable storage medium can also achieve the technical effect of the above-mentioned face liveness detection method. That is, the face liveness detection model is trained through steps S100 to S300, and Fourier supervision branches and depth supervision branches are added to the face liveness detection model to generate Fourier spectra and depth maps respectively, thereby supervising the liveness classification branch, providing a simple auxiliary supervision signal to supervise the training of the liveness detection model, and improving the effect of liveness detection in the image to be detected through step S400. The present invention does not need to use video as an input signal to establish an rPPG signal, and is not easily affected by the environment, thereby improving the stability and accuracy of the detection.

[0099] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A method for detecting liveness of a face, characterized in that: The steps include: Collect training sample images; Inputting the training sample image into a face liveness detection model, wherein the face liveness detection model includes a feature extraction network, a Fourier supervision branch, a deep supervision branch, and a liveness classification branch; Calculating the loss functions of the Fourier supervision branch, the depth supervision branch and the living body classification branch respectively, and training the face living body detection model according to the loss functions; Inputting the image to be detected into the trained face liveness detection model to obtain an output liveness classification result, wherein the liveness classification result includes at least one detected liveness attribute category; After the training sample image is input into the Fourier supervision branch, the Fourier supervision branch performs Fourier transform on the training sample image to generate a Fourier spectrum, and when calculating the loss function of the Fourier supervision branch, the loss function is calculated based on the Fourier spectrum and the feature map extracted by the feature extraction network; After the training sample image is input into the depth supervision branch, the depth supervision branch uses PRNet to generate a pseudo depth map of a preset size for the negative sample image, and the pseudo depth map has the same size as the input training sample image, and uses an image of a preset size with all pixel values ​​of 0 as a depth map for the positive sample image, and the depth map has the same size as the input training sample image. When calculating the loss function of the depth supervision branch, the loss function is calculated based on the pseudo depth map or the depth map and the feature map extracted by the feature extraction network; The feature map extracted by the feature extraction network is used for the living body classification branch to perform living body detection classification, and the living body classification branch loss function is calculated according to the classification result of the living body classification branch and the classification label pre-marked by the training sample image; Among them, the loss function of the Fourier supervision branch loss function, the depth supervision branch loss function and the living body classification branch loss function are weighted and summed to obtain the loss function of the face living body detection model.

2. The method for face liveness detection according to claim 1, characterized in that: After the Fourier supervision branch performs Fourier transform on the training sample image to generate the Fourier spectrum, the following steps are also included: the Fourier supervision branch normalizes the training sample image and adjusts the size to a preset size.

3. The method for face liveness detection according to claim 1, characterized in that: The feature extraction network adopts a residual network and a pyramid network, the residual network is a resnet18 network or a resnet34 network, and the output of the pyramid network is respectively connected to a first output auxiliary supervision branch, a second output auxiliary supervision branch and a third output auxiliary supervision branch.

4. The method for face liveness detection according to claim 3, characterized in that: The loss functions of the Fourier supervision branch, the deep supervision branch and the living body classification branch are calculated respectively, including the following steps: Calculate the Fourier supervision branch loss function according to the output of the first output auxiliary supervision branch and the output of the Fourier supervision branch; Calculate the deep supervision branch loss function according to the output of the second output auxiliary supervision branch and the output of the deep supervision branch; The output of the third output auxiliary supervision branch is input into the living body classification branch to obtain a classification result of the living body classification branch; The living body classification branch loss function is calculated according to the classification result of the living body classification branch and the label of the training sample image.

5. The method for face liveness detection according to claim 3, characterized in that: The first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch adopt the same structure, and the first output auxiliary supervision branch, the second output auxiliary supervision branch and the third output auxiliary supervision branch respectively include convolutional layers.

6. The method for face liveness detection according to claim 1, characterized in that: After weighted summing of the Fourier supervision branch loss function, the depth supervision branch loss function and the liveness classification branch loss function to obtain the loss function of the face liveness detection model, the following steps are also included: The face liveness detection model is iteratively trained by adopting a stochastic gradient descent method.

7. The method for face liveness detection according to claim 1, characterized in that: The Fourier supervision branch loss function uses the MES loss function, and the deep supervision loss function uses the CLD loss function.

8. The method for face liveness detection according to claim 1, characterized in that: The classification branch loss functions include face attribute loss function, deception type loss function, illumination loss function, environment loss function and positive and negative classification loss function.

9. The method for face liveness detection according to claim 8, characterized in that: The face attribute loss function adopts the BCE loss function, the deception type loss function, the illumination loss function and the environment loss function respectively adopt the CrossEntropy loss function, and the positive and negative classification loss function adopts the arcface loss function.

10. A face liveness detection system, used to implement the face liveness detection method according to any one of claims 1 to 9, characterized in that: The system comprises: A sample collection module, used for collecting training sample images; A model input module, used to input a training sample image into a face liveness detection model, wherein the face liveness detection model includes a feature extraction network, a Fourier supervision branch, a deep supervision branch, and a liveness classification branch; A model training module, used to calculate the loss functions of the Fourier supervision branch, the deep supervision branch and the living body classification branch respectively, and train the face liveness detection model according to the loss functions; The liveness detection module is used to input the image to be detected into the trained face liveness detection model to obtain an output liveness classification result, wherein the liveness classification result includes at least one detected liveness attribute category.

11. A face liveness detection device, characterized in that: include: processor; a memory storing executable instructions of the processor; The processor is configured to execute the steps of the face liveness detection method according to any one of claims 1 to 9 by executing the executable instructions.

12. A computer-readable storage medium for storing a program, characterized in that: When the program is executed by a processor, the steps of the face liveness detection method described in any one of claims 1 to 9 are implemented.

Images