Human face liveness detection method, device, electronic device and readable storage medium

By obtaining multi-frame images from face videos and mapping them to depth maps, and combining RGB and depth information for face live detection, the problem of low accuracy of face recognition in the prior art is solved, and higher accuracy is achieved Sex and resistance to attack.

CN115082992BActive Publication Date: 2024-12-03PING AN BANK CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202210744097.1
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2022-06-27
Publication Date
2024-12-03
Estimated Expiration
2042-06-27

AI Technical Summary

Technical Problem

Existing facial recognition technology only uses two-dimensional information, which makes it easy for attackers to use fake faces to pass face detection, affecting the accuracy of live detection.

Method used

By obtaining multi-frame face images from face videos, mapping them into depth maps, combining RGB images and depth map information, inputting the trained face recognition model for recognition, and using dual defense to improve the accuracy of live detection and attack resistance.

Benefits of technology

It effectively improves the accuracy and anti-attack ability of face live detection, and uses the dual features of face images and depth maps to enhance the characteristics that distinguish live or inactive, and improves the accuracy of detection.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN115082992B_ABST
    Figure CN115082992B_ABST
Patent Text Reader

Abstract

The present application relates to the field of image recognition technology, and provides a method, device, electronic device and readable storage medium for face liveness detection. The method comprises: obtaining multiple frames of face images from a face video that executes a liveness verification instruction; mapping each face image one by one to each corresponding depth map to obtain each face depth map; selecting multiple target face images from each face image to input a trained first face recognition model to obtain a first recognition result, and selecting multiple target face depth maps from each face depth map to input a trained second face recognition model to obtain a second recognition result; determining a liveness detection result based on the first recognition result and the second recognition result. The face liveness detection method provided in the embodiment of the present application can improve the accuracy of face liveness detection.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of image recognition technology, and in particular to a method, device, electronic device and readable storage medium for detecting liveness of a face. Background Art

[0002] With the development of mobile Internet, the use of face authentication is becoming more and more common. At the same time, the security problem of attackers using fake faces to complete face recognition is becoming more and more common. In order to prevent attackers from using fake faces to complete face recognition, existing technologies usually use motion biopsy, silent liveness detection, facial glare liveness detection and other methods to resist the use of fake faces by cybercriminals to complete online face recognition.

[0003] However, the above-mentioned face recognition method only utilizes the two-dimensional information of the face, and there are fewer features available when recognizing the face, which makes it easy for attackers to use fake faces to pass face detection, affecting the accuracy of liveness detection of the face. Summary of the invention

[0004] The present application aims to solve at least one of the technical problems existing in the related art. To this end, the present application proposes a method for detecting liveness of a face, which can improve the accuracy of detecting liveness of a face.

[0005] The present application also proposes a face liveness detection device.

[0006] The application also provides an electronic device.

[0007] The present application also proposes a readable storage medium.

[0008] A method for detecting live faces according to an embodiment of the first aspect of the present application includes:

[0009] Acquire multiple frames of face images from a face video executing a liveness verification instruction;

[0010] Mapping each of the face images to a corresponding depth map one by one to obtain each face depth map;

[0011] From each of the face images, select multiple target face images and input them into a trained first face recognition model to obtain a first recognition result, and from each of the face depth maps, select multiple target face depth maps and input them into a trained second face recognition model to obtain a second recognition result;

[0012] Determine a liveness detection result according to the first recognition result and the second recognition result;

[0013] The first face recognition model is trained by training samples of RGB face images, and the second face recognition model is trained by training samples of depth maps of faces.

[0014] The face liveness detection method provided in the embodiment of the present application obtains multiple frames of face images from a face video, and maps each face image to a corresponding depth map. After obtaining the face depth map, each face image and each face depth map are respectively input into different face recognition models for recognition, and the recognition result of each face image and the recognition result of each face depth map are used to determine the liveness detection result, thereby utilizing the dual defense of RGB image information and depth map information of the face image to perform liveness detection, increasing the available features in the face liveness recognition process, and thereby more effectively improving the accuracy of biopsy and the anti-attack capability.

[0015] According to an embodiment of the present application, obtaining multiple frames of face images from a face video executing a liveness verification instruction includes:

[0016] Performing face extraction on each frame image of the face video to obtain multiple frames of initial images;

[0017] Perform a face region enlargement operation on each of the initial images to obtain each of the face images.

[0018] According to an embodiment of the present application, the liveness verification instruction includes a head shaking instruction, the first face recognition model includes a first timing model, and the second face recognition model includes a second timing model;

[0019] The method of selecting a plurality of target face images from each of the face images and inputting them into a trained first face recognition model to obtain a first recognition result, and selecting a plurality of target face depth images from each of the face depth images and inputting them into a trained second face recognition model to obtain a second recognition result includes:

[0020] From each of the facial images, multiple target facial images are selected and input into a trained first time series model to obtain first time series features, and from each of the facial depth maps, multiple target facial depth maps are selected and input into a trained second time series model to obtain second time series features.

[0021] According to an embodiment of the present application, each of the target face images is extracted from each of the face images according to a preset extraction interval;

[0022] Each of the target face depth maps is extracted from each of the face depth maps according to the preset extraction interval.

[0023] According to one embodiment of the present application, at least one of the first timing model and the second timing model includes a residual network.

[0024] According to an embodiment of the present application, determining a liveness detection result according to the first recognition result and the second recognition result includes:

[0025] According to a first preset weight of the first recognition result and a second preset weight of the second recognition result, weighting the first recognition result and the second recognition result to determine a liveness detection probability;

[0026] The liveness detection probability is greater than a preset probability, and the liveness detection result is determined to be liveness.

[0027] According to one embodiment of the present application, the first preset weight is smaller than the second preset weight.

[0028] A face liveness detection device according to an embodiment of the second aspect of the present application includes:

[0029] A face image acquisition module is used to acquire multiple frames of face images from a face video executing a liveness verification instruction;

[0030] A depth map acquisition module, used to map each of the face images to a corresponding depth map, and acquire each face depth map;

[0031] A face image detection module is used to select a plurality of target face images from each of the face images and input them into a trained first face recognition model to obtain a first recognition result, and to select a plurality of target face depth images from each of the face depth images and input them into a trained second face recognition model to obtain a second recognition result;

[0032] A face liveness detection module, used to determine a liveness detection result according to the first recognition result and the second recognition result;

[0033] The first face recognition model is trained by training samples of RGB face images, and the second face recognition model is trained by training samples of depth maps of faces.

[0034] An electronic device according to an embodiment of the third aspect of the present application includes a processor and a memory storing a computer program, and when the processor executes the computer program, the face liveness detection method described in any of the above embodiments is implemented.

[0035] According to the readable storage medium of the fourth aspect of the present application, a computer program is stored thereon, and when the computer program is executed by a processor, the method for face liveness detection described in any of the above embodiments is implemented.

[0036] According to the computer program product of the fifth aspect of the present application, the computer program product includes: when the computer program is executed by a processor, the face liveness detection method as described in any of the above embodiments is implemented.

[0037] The above one or more technical solutions in the embodiments of the present application have at least one of the following technical effects:

[0038] By obtaining multiple frames of face images from a face video and mapping each face image to a corresponding depth map, after obtaining the face depth map, each face image and each face depth map are respectively input into different face recognition models for recognition, and the recognition result of each face image and the recognition result of each face depth map are used to determine the liveness detection result, thereby utilizing the dual defense of the RGB image information and the depth map information of the face image for liveness detection, increasing the available features in the face liveness recognition process, and thereby more effectively improving the accuracy of biopsy and the anti-attack capability.

[0039] Furthermore, by performing face extraction on face videos and obtaining multiple frames of initial images, the face area is enlarged on each initial image to obtain the final face image, so that there are more features in the face image for distinguishing between living and non-living objects, thereby improving the accuracy of subsequent face liveness detection.

[0040] Furthermore, by combining the temporal features of facial images and the temporal features of facial depth maps, face liveness detection is performed, thereby utilizing the obvious differences in the temporal features of facial images and depth maps between synthesized or reshot videos and face videos captured in vivo to more accurately perform face liveness detection.

[0041] Furthermore, by extracting the target face image and the target face depth map according to a preset extraction interval, the interval time between two adjacent target face images or two adjacent target face depth maps is made the same, thereby improving the accuracy of the acquired timing features. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] In order to more clearly illustrate the technical solutions in the present application or the prior art, a brief introduction will be given below to the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0043] Figure 1 It is a flowchart of a method for detecting live faces provided in an embodiment of the present application;

[0044] Figure 2 is a schematic diagram of the network structure of the timing model in an embodiment of the present application;

[0045] Figure 3 is a structural schematic diagram of a face liveness detection device provided in an embodiment of the present application;

[0046] Figure 4It is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solutions and advantages of this application clearer, the technical solutions in this application will be clearly and completely described below in conjunction with the drawings in the embodiments of this application. Obviously, the described embodiments are part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0048] The following will introduce and explain in detail the face liveness detection method, device, electronic device and readable storage medium provided in the embodiments of the present application through several specific embodiments.

[0049] In one embodiment, a face liveness detection method is provided, which is applied to a server for performing face liveness detection. The server may be an independent server or a server cluster composed of multiple servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence sampling point devices.

[0050] like Figure 1 As shown, a method for detecting live faces in this embodiment includes:

[0051] Step 101, obtaining multiple frames of face images from a face video executing a liveness verification instruction;

[0052] Step 102, mapping each of the face images to a corresponding depth map one by one to obtain each face depth map;

[0053] Step 103, selecting a plurality of target face images from each of the face images and inputting them into a trained first face recognition model to obtain a first recognition result, and selecting a plurality of target face depth images from each of the face depth images and inputting them into a trained second face recognition model to obtain a second recognition result;

[0054] Step 104, determining a liveness detection result according to the first recognition result and the second recognition result;

[0055] The first face recognition model is trained by training samples of RGB face images, and the second face recognition model is trained by training samples of depth maps of faces.

[0056] By obtaining multiple frames of face images from a face video and mapping each face image to a corresponding depth map, after obtaining the face depth map, each face image and each face depth map are respectively input into different face recognition models for recognition, and the recognition result of each face image and the recognition result of each face depth map are used to determine the liveness detection result, thereby utilizing the dual defense of the RGB image information and the depth map information of the face image for liveness detection, increasing the available features in the face liveness recognition process, and thereby more effectively improving the accuracy of biopsy and the anti-attack capability.

[0057] In one embodiment, the collection of face video can be performed by a terminal device with a laser radar. After the face video executing the liveness verification instruction is collected by the terminal device, multiple frames of RGB face images are extracted from the face video through a face detector, such as mediapipe. Among them, the face video can be a face reflection video. Specifically, a color sequence composed of multiple colors can be pre-selected, such as randomly selecting four colors from nine colors of red, orange, yellow, green, cyan, blue, purple, black and white to generate a color sequence, and then the color sequence is sequentially passed through an irradiation device, such as the screen of the terminal device to generate light to reflect the face, to obtain a face reflection video, and randomly intercept the face image in the face reflection video through the face detector, thereby reducing the possibility of the face image being stolen.

[0058] Among them, the liveness verification instruction is an instruction used to instruct a certain part of the face to perform a corresponding operation during the liveness verification process, such as instructing the face to perform an eye blinking operation, instructing the face to perform a mouth opening operation, or instructing the face to perform a head shaking operation, etc. After the liveness verification instruction is issued, multiple frames of face images are captured from the face video, so that the captured multiple frames of face images have different features, and then the different features of the multiple frames of face images can be used for face liveness detection in the subsequent process, thereby improving the accuracy of the subsequent face liveness detection.

[0059] Considering that the features for distinguishing liveness from non-liveness are not only present in the central area of ​​the face, but also concentrated at the edge and outside the face, in order to further improve the accuracy of subsequent face liveness verification, in one embodiment, multiple frames of face images are obtained from the face video executing the liveness verification instruction, including:

[0060] Performing face extraction on each frame image of the face video to obtain multiple frames of initial images;

[0061] Perform a face region enlargement operation on each of the initial images to obtain each of the face images.

[0062] In one embodiment, after the liveness verification instruction is issued, the face region of each frame image of the face video is extracted in advance by a face detector such as mediapipe, and the extracted face image is the initial image. After the initial image is extracted, at least two vertex coordinates of any initial image are obtained, such as the upper left vertex coordinates (face_box.x1, face_box.y1) of the initial image and the upper right vertex coordinates (face_box.x2, face_box.y2) of the initial image. After the upper left vertex coordinates and the upper right vertex coordinates are obtained, the face region is enlarged according to a preset multiple, such as enlarging the face region by 1.45 times, so that the upper left vertex coordinates of the initial image become: (face_box.x1-1.45*face_box.x1,ace_box.y1-1.45*face_box.y1).

[0063] The coordinates of the upper right vertex become: (face_box.x2+1.45*face_box.x2,ace_box.y2+1.45*face_box.y2).

[0064] Similarly, the coordinates of the vertex at the lower left corner and the vertex at the lower right corner may also be changed according to the above method, so that the initial image after the face area is enlarged is determined as a face image.

[0065] It is understandable that, in addition to changing the coordinates of the four vertices simultaneously, it is also possible to change the coordinates of only any two vertices on the same side, such as changing the coordinates of only the upper left and upper right vertices, or only the upper left and lower left vertices.

[0066] By extracting faces from face videos and obtaining multiple frames of initial images, the face area of ​​each initial image is expanded to obtain the final face image, so that there are more features in the face image for distinguishing between living and non-living objects, thereby improving the accuracy of subsequent face liveness detection.

[0067] In one embodiment, after acquiring the face image, depth map mapping is performed on each face image, and each frame of the face image is mapped to the depth map of each frame of the D channel, thereby obtaining each face depth map. It can be understood that since the face area of ​​the face image is enlarged by a preset multiple compared to the initial image directly captured from the face video, the face area of ​​the face depth map is also enlarged by a preset multiple.

[0068] In one embodiment, after obtaining each face depth map, each face image is combined with each face depth map for liveness detection. Specifically, from each face image, multiple target face images are selected and input into a trained first face recognition model to obtain a first detection probability output by the first face recognition model; at the same time, multiple target face depth images are selected from each face depth map and input into a trained second face recognition model to obtain a second detection probability output by the second face recognition model. Among them, the selection of target face images can be to use all face images as target face images; or, randomly extract N target face images from each face image; or, it can be multiple target face images extracted according to a preset extraction interval, such as extracting one face image as the target face image every two face images. Similarly, the extraction of target face depth maps can be the same as the extraction of face images.

[0069] To further improve the accuracy of face liveness detection, in one embodiment, the liveness verification instruction includes a head shaking instruction, the first face recognition model includes a first timing model, and the second face recognition model includes a second timing model;

[0070] The method of selecting a plurality of target face images from each of the face images and inputting them into a trained first face recognition model to obtain a first recognition result, and selecting a plurality of target face depth images from each of the face depth images and inputting them into a trained second face recognition model to obtain a second recognition result includes:

[0071] From each of the facial images, multiple target facial images are selected and input into a trained first time series model to obtain first time series features, and from each of the facial depth maps, multiple target facial depth maps are selected and input into a trained second time series model to obtain second time series features.

[0072] In one embodiment, in addition to using the single image features of the target face image and the target face depth map for liveness detection, due to the fact that when the synthesized or reshot video is shaking its head, there will be a "weird" afterimage or blurred edge information in the RGB image, which is quite different from the normal shaking of the head, and when the normal person is shaking his head, the portrait of the person shaking his head can be clearly seen, while the depth map of the common attack methods such as synthesis and reshooting will only see a piece without any portrait. Therefore, the face image and face depth map obtained when the user is instructed to shake his head can be used to more accurately perform liveness detection of the face. Specifically, after selecting multiple target face images from each face image, the multiple target face images are used as a set of image sets and input into the trained first time series model to obtain the first time series feature. Similarly, the target face depth map selected from each face depth map is also used as a set of image sets and input into the trained second time series model to obtain the first time series feature.

[0073] Among them, the first time series model and the second time series model are improved on the basis of the C3D time series model to improve the robustness of the model and improve the computational efficiency of the model. C3D is actually the 3D version of 2D convolution, which converts 2D convolution into 3D convolution and converts the input B*H*W*C into B*H*W*C*D. Using 3D convolution, you can treat several consecutive images as one D, which makes it easier to extract time series features.

[0074] In order to make the time series features obtained by the first time series model and the second time series model more accurate, in one embodiment, the target face image is extracted from each of the face images according to a preset extraction interval. For example, if there are N images in each interval, one face image is extracted from each of the face images as the target face image. Similarly, each of the target face depth images is extracted from each of the face depth images according to the preset extraction interval.

[0075] By extracting the target face image and the target face depth map according to a preset extraction interval, the interval time between two adjacent target face images or two adjacent target face depth maps is made the same, thereby improving the accuracy of the acquired temporal features.

[0076] In one embodiment, if Figure 2 As shown, the first time series model and / or the second time series model uses a residual network to enhance the robustness and learning ability of the model, thereby greatly improving the accuracy of liveness detection. After completing the extraction of each target face image and each target face depth map, each target face image is taken as a group and input into the trained first time series model to obtain the first time series feature cls1, and the first time series feature is used as the first detection probability; similarly, each target face depth map can be taken as a group and input into the trained second time series model to obtain the second time series feature cls2, and the second time series feature is used as the second detection probability.

[0077] By combining the temporal features of facial images and the temporal features of facial depth maps, face liveness detection is performed. In this way, the face liveness detection can be performed more accurately by utilizing the obvious differences in the temporal features of facial images and depth maps between synthetic or reshot videos and face videos captured in vivo.

[0078] After obtaining the first detection probability and the second detection probability, that is, the first recognition result and the second recognition result, the liveness detection result can be determined according to the first recognition result and the second recognition result. For example, when both the first recognition result and the second recognition result are less than the preset probability, the liveness detection result is judged to be non-liveness; or when the first recognition result is less than the first preset probability, and the second recognition result is less than the second preset probability, the liveness detection is judged to be non-liveness; or, the first recognition result and the second recognition result can also be input into the classifier, and the class confidence of the positive or negative class output by the classifier is determined according to the comparison result of the class confidence and the preset confidence. Among them, the positive class represents a live body, and the negative class represents a non-live body. When the class confidence is greater than the preset confidence, the class corresponding to the class confidence is determined to be the liveness detection result; otherwise, the class opposite to the class confidence is determined to be the liveness detection result.

[0079] Exemplarily, the category confidence of the live body output by the classifier is 0.6, and the preset confidence is 0.55. At this time, the category confidence is greater than the preset confidence, and the category confidence is judged to be credible, so the liveness detection result can be determined to be live; otherwise, the category confidence is judged to be unreliable, and the liveness detection result is determined to be non-live.

[0080] In one embodiment, the two results can also be effectively fused by a weighted method. Figure 3 As shown, determining the liveness detection result according to the first recognition result and the second recognition result includes:

[0081] According to a first preset weight of the first recognition result and a second preset weight of the second recognition result, weighting the first recognition result and the second recognition result to determine a liveness detection probability;

[0082] The liveness detection probability is greater than a preset probability, and the liveness detection result is determined to be liveness.

[0083] In one embodiment, after obtaining the first recognition result and the second recognition result, the first recognition result is adjusted by the first preset weight, and the second recognition result is adjusted by the second preset weight, and the adjusted first recognition result is added to the adjusted second recognition result to obtain the probability of liveness detection.

[0084] Taking into account that when a normal person shakes his head, the portrait of the person shaking his head can be clearly seen, while the depth map of common attack methods such as synthesis and reshooting will only show a piece without any portrait appearing, that is, the timing characteristics of the depth map can better confirm whether it is a living body. Therefore, in one embodiment, the second preset weight is greater than the first preset weight to further improve the accuracy of liveness detection.

[0085] For example, the first preset weight may be 0.3, and the second preset weight may be 0.7. In this case, the probability of liveness detection is:

[0086] P(cls)=0.3*cls1+0.7*cls2

[0087] Among them, cls1 is the first recognition result, cls2 is the second recognition result, and P is the liveness detection probability.

[0088] After determining the liveness detection probability, the liveness detection probability is compared with the preset probability. If the liveness detection probability is greater than the preset probability, the liveness detection result is determined to be live; otherwise, the liveness detection result is determined to be non-live. The preset probability can be set according to the actual situation, such as 0.65.

[0089] The following is a description of the face liveness detection device provided by the present application. The face liveness detection device described below and the face liveness detection method described above can be referenced to each other.

[0090] In one embodiment, if Figure 3 As shown, a human face liveness detection device is provided, comprising:

[0091] The face image acquisition module 210 is used to acquire multiple frames of face images from the face video executing the liveness verification instruction;

[0092] A depth map acquisition module 220, used to map each of the face images to a corresponding depth map, to acquire each face depth map;

[0093] The face image detection module 230 is used to select a plurality of target face images from each of the face images and input them into a trained first face recognition model to obtain a first recognition result, and select a plurality of target face depth images from each of the face depth images and input them into a trained second face recognition model to obtain a second recognition result;

[0094] A face liveness detection module 240, configured to determine a liveness detection result according to the first recognition result and the second recognition result;

[0095] The first face recognition model is trained by training samples of RGB face images, and the second face recognition model is trained by training samples of depth maps of faces.

[0096] By obtaining multiple frames of face images from a face video and mapping each face image to a corresponding depth map, after obtaining the face depth map, each face image and each face depth map are respectively input into different face recognition models for recognition, and the recognition result of each face image and the recognition result of each face depth map are used to determine the liveness detection result, thereby utilizing the dual defense of the RGB image information and the depth map information of the face image for liveness detection, increasing the available features in the face liveness recognition process, and thereby more effectively improving the accuracy of biopsy and the anti-attack capability.

[0097] In one embodiment, the face image acquisition module 210 is specifically used for:

[0098] Performing face extraction on each frame image of the face video to obtain multiple frames of initial images;

[0099] Perform a face region enlargement operation on each of the initial images to obtain each of the face images.

[0100] In one embodiment, the liveness verification instruction includes a head shaking instruction, the first face recognition model includes a first timing model, and the second face recognition model includes a second timing model; the face image detection module 230 is specifically used to:

[0101] From each of the facial images, multiple target facial images are selected and input into a trained first time series model to obtain first time series features, and from each of the facial depth maps, multiple target facial depth maps are selected and input into a trained second time series model to obtain second time series features.

[0102] In one embodiment, each of the target facial images is extracted from each of the facial images according to a preset extraction interval;

[0103] Each of the target face depth maps is extracted from each of the face depth maps according to the preset extraction interval.

[0104] In one embodiment, at least one of the first timing model and the second timing model includes a residual network.

[0105] In one embodiment, the face liveness detection module 240 is specifically used for:

[0106] According to a first preset weight of the first recognition result and a second preset weight of the second recognition result, weighting the first recognition result and the second recognition result to determine a liveness detection probability;

[0107] The liveness detection probability is greater than a preset probability, and the liveness detection result is determined to be liveness.

[0108] In one embodiment, the first preset weight is smaller than the second preset weight.

[0109] Figure 4 An example of a physical structure diagram of an electronic device is shown in FIG. Figure 4 As shown, the electronic device may include: a processor 810, a communication interface 820, a memory 830 and a communication bus 840, wherein the processor 810, the communication interface 820, and the memory 830 communicate with each other through the communication bus 840. The processor 810 may call a computer program in the memory 830 to execute a method for detecting liveness of a face, for example, including:

[0110] Acquire multiple frames of face images from a face video executing a liveness verification instruction;

[0111] Mapping each of the face images to a corresponding depth map one by one to obtain each face depth map;

[0112] From each of the face images, select multiple target face images and input them into a trained first face recognition model to obtain a first recognition result, and from each of the face depth maps, select multiple target face depth maps and input them into a trained second face recognition model to obtain a second recognition result;

[0113] Determine a liveness detection result according to the first recognition result and the second recognition result;

[0114] The first face recognition model is trained by training samples of RGB face images, and the second face recognition model is trained by training samples of depth maps of faces.

[0115] In addition, the logic instructions in the above-mentioned memory 830 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0116] On the other hand, an embodiment of the present application further provides a storage medium, the storage medium includes a computer program, the computer program can be stored on a non-transitory readable storage medium, when the computer program is executed by a processor, the computer can execute the face liveness detection method provided in the above embodiments, for example, including:

[0117] Acquire multiple frames of face images from a face video executing a liveness verification instruction;

[0118] Mapping each of the face images to a corresponding depth map one by one to obtain each face depth map;

[0119] From each of the face images, select multiple target face images and input them into a trained first face recognition model to obtain a first recognition result, and from each of the face depth maps, select multiple target face depth maps and input them into a trained second face recognition model to obtain a second recognition result;

[0120] Determine a liveness detection result according to the first recognition result and the second recognition result;

[0121] The first face recognition model is trained by training samples of RGB face images, and the second face recognition model is trained by training samples of depth maps of faces.

[0122] On the other hand, an embodiment of the present application further provides a processor-readable storage medium, the processor-readable storage medium storing a computer program, the computer program is used to enable the processor to execute the methods provided in the above embodiments, for example, including:

[0123] Acquire multiple frames of face images from a face video executing a liveness verification instruction;

[0124] Mapping each of the face images to a corresponding depth map one by one to obtain each face depth map;

[0125] From each of the face images, select multiple target face images and input them into a trained first face recognition model to obtain a first recognition result, and from each of the face depth maps, select multiple target face depth maps and input them into a trained second face recognition model to obtain a second recognition result;

[0126] Determine a liveness detection result according to the first recognition result and the second recognition result;

[0127] The first face recognition model is trained by training samples of RGB face images, and the second face recognition model is trained by training samples of depth maps of faces.

[0128] The processor-readable storage medium can be any available medium or data storage device that can be accessed by the processor, including but not limited to magnetic storage (such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO)), optical storage (such as CDs, DVDs, BDs, HVDs, etc.), and semiconductor storage (such as ROM, EPROM, EEPROM, non-volatile memory (NAND FLASH), solid-state drives (SSDs)), etc.

[0129] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, i.e., they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative effort.

[0130] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0131] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for detecting liveness of a face, characterized in that: include: Acquire multiple frames of face images from a face video executing a liveness verification instruction; Mapping each of the face images to a corresponding depth map one by one to obtain each face depth map; From each of the face images, select multiple target face images and input them into a trained first face recognition model to obtain a first recognition result, and from each of the face depth maps, select multiple target face depth maps and input them into a trained second face recognition model to obtain a second recognition result; Determine a liveness detection result according to the first recognition result and the second recognition result; The first face recognition model is trained by training samples of RGB face images, and the second face recognition model is trained by training samples of depth maps of faces. The first face recognition model includes a first time series model, and the second face recognition model includes a second time series model; The method of selecting a plurality of target face images from each of the face images and inputting them into a trained first face recognition model to obtain a first recognition result, and selecting a plurality of target face depth images from each of the face depth images and inputting them into a trained second face recognition model to obtain a second recognition result includes: From the facial images obtained when the user is instructed to shake his head, multiple target facial images are selected and input into a trained first timing model to obtain first timing features, and from the facial depth maps obtained when the user is instructed to shake his head, multiple target facial depth maps are selected and input into a trained second timing model to obtain second timing features.

2. The method for face liveness detection according to claim 1, characterized in that: From the face video executing the liveness verification instruction, multiple frames of face images are obtained, including: Performing face extraction on each frame image of the face video to obtain multiple frames of initial images; Perform a face region enlargement operation on each of the initial images to obtain each of the face images.

3. The method for face liveness detection according to claim 1, characterized in that: Each of the target face images is extracted from each of the face images according to a preset extraction interval; Each of the target face depth maps is extracted from each of the face depth maps according to the preset extraction interval.

4. The method for face liveness detection according to claim 1, characterized in that: At least one of the first timing model and the second timing model includes a residual network.

5. The method for face liveness detection according to claim 1, characterized in that: The determining the liveness detection result according to the first recognition result and the second recognition result includes: Weighting the first recognition result and the second recognition result according to a first preset weight of the first recognition result and a second preset weight of the second recognition result to determine a liveness detection probability; The liveness detection probability is greater than a preset probability, and the liveness detection result is determined to be liveness.

6. The method for face liveness detection according to claim 5, characterized in that: The first preset weight is smaller than the second preset weight.

7. A face liveness detection device, characterized in that: include: A face image acquisition module is used to acquire multiple frames of face images from a face video executing a liveness verification instruction; A depth map acquisition module, used to map each of the face images to a corresponding depth map, and acquire each face depth map; A face image detection module is used to select a plurality of target face images from each of the face images and input them into a trained first face recognition model to obtain a first recognition result, and to select a plurality of target face depth images from each of the face depth images and input them into a trained second face recognition model to obtain a second recognition result; A face liveness detection module, used to determine a liveness detection result according to the first recognition result and the second recognition result; The first face recognition model is trained by training samples of RGB face images, and the second face recognition model is trained by training samples of depth maps of faces. The first face recognition model includes a first time series model, and the second face recognition model includes a second time series model; The facial image detection module is specifically used to: select multiple target facial images from each facial image obtained when the user is instructed to shake his head, input them into a trained first time series model, and obtain first time series features; and select multiple target facial depth images from each facial depth image obtained when the user is instructed to shake his head, input them into a trained second time series model, and obtain second time series features.

8. An electronic device comprising a processor and a memory storing a computer program, characterized in that: When the processor executes the computer program, the method for face liveness detection according to any one of claims 1 to 6 is implemented.

9. A readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for face liveness detection according to any one of claims 1 to 6 is implemented.

Citation Information

Patent Citations

  • Face-in-vivo detection method, device, equipment and storage medium

    CN109034102A

  • Living body fusion feature-based authentication method and apparatus, and computer device

    CN113190819A