Human face liveness detection method and system

Abstract

The present invention provides a method for detecting live faces, comprising: obtaining an infrared image of a face to be detected and image data containing depth information; performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image; in response to the size of the face region being within a preset range, performing face plane fitting on the face region in the image data containing depth information; in response to the result of the face plane fitting meeting a preset requirement, performing face side fitting on the face region in the image data containing depth information; and in response to the result of the face side fitting meeting a preset requirement, determining that the face to be detected is a live face. Compared with traditional face liveness detection methods, the face liveness detection method provided by the present invention has high detection accuracy.

Description

Technical Field

[0001] The present invention relates to the field of face detection, and further to a face liveness detection method and a system thereof, and in particular to a face liveness detection method and a system thereof based on a TOF camera device. Background Art

[0002] With the development of science and technology, face recognition technology has made great progress and has been widely used in the fields of payment, security and embedded mobile terminals, such as face payment, face scanning for security checks at stations or airports, and face scanning to start up embedded mobile terminals, bringing great convenience to people's lives.

[0003] However, it should be pointed out that traditional face recognition systems often face the challenge of non-living intrusion. Traditional face recognition systems are often deceived by printed papers, photos, videos and face molds with human faces, which greatly affects the security performance of face recognition systems. For example, without the consent of the owner, someone can open the owner's mobile terminal with a photo of the owner, which will cause great losses to the users of mobile terminals with face recognition functions. Therefore, ensuring that face recognition systems are protected from non-living intrusions has become a focus of attention. In face recognition systems, liveness detection has become a crucial step and the main means to prevent the system from being attacked by non-living attacks.

[0004] There are roughly three traditional liveness detection methods: liveness detection methods based on traditional image knowledge, liveness detection methods based on deep learning neural networks, and liveness detection methods based on human-computer interaction.

[0005] In the detection process, the liveness detection method based on traditional image knowledge uses the color image of the face to extract texture information, and then distinguishes live faces from non-live faces. However, this detection method is easily affected by environmental changes, especially when the lighting changes. The imaging effect of the color camera is easily affected by environmental factors such as lighting, which reduces the accuracy of liveness judgment and leads to poor robustness of the liveness detection algorithm.

[0006] In the detection process, the liveness detection method based on deep learning neural network must first design a reasonable and effective network structure, use a large amount of pre-collected live and non-live data to train the network to extract features, and finally achieve liveness detection. The liveness detection method based on deep learning neural network will have a greatly reduced liveness recognition effect when facing liveness with a large background difference from the training sample.

[0007] Human-computer interaction liveness detection methods usually require the person being identified to perform a series of prescribed actions in front of the camera during the detection process, such as nodding, opening the mouth, closing the eyes, etc. Adding human-computer interaction during the detection process can enhance the reliability of the liveness detection algorithm to a certain extent, but adding an interactive process does not conform to the concept of intelligent development and will inevitably reduce the user experience of the face recognition system.

[0008] In summary, in view of the defects and shortcomings of traditional liveness detection methods, it is necessary to improve traditional liveness detection methods. Summary of the invention

[0009] An object of the present invention is to provide a face liveness detection method and system thereof, which, compared with the traditional face liveness detection method, can perform face liveness detection in a complex environment.

[0010] Another object of the present invention is to provide a face liveness detection method and a detection system thereof, which, compared with traditional face liveness detection methods, can perform face liveness detection in complex lighting environments.

[0011] Another object of the present invention is to provide a face liveness detection method and a detection system thereof, which, compared with the traditional face liveness detection method, can overcome the interference of external light during the face liveness detection process and has good face liveness detection stability.

[0012] Another object of the present invention is to provide a face liveness detection method and system thereof, which adopts multiple liveness attribute judgment mechanisms and can effectively perform liveness detection.

[0013] Another object of the present invention is to provide a face liveness detection method and system thereof, which combines infrared images and image data containing depth information to perform multi-mechanism face liveness detection with high face liveness detection accuracy.

[0014] Accordingly, in order to achieve at least one of the above invention objectives, the present invention provides a method for detecting a live face, comprising:

[0015] Acquire an infrared image of the face to be tested and image data containing depth information collected by a TOF camera device;

[0016] Performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image;

[0017] Based on the face area, the face feature points and the image data containing the depth information, liveness detection is performed, wherein the liveness detection process includes:

[0018] In response to the size of the face region being within a preset range, performing face plane fitting on the face region in the image data containing the depth information;

[0019] In response to the result of the face plane fitting meeting a preset requirement, performing face side fitting on the face area in the image data containing the depth information; and

[0020] In response to the result of the face profile fitting meeting a preset requirement, it is determined that the face to be tested is a living face.

[0021] In some preferred embodiments of the present invention, after acquiring an infrared image of a human face to be tested and image data containing depth information collected by a TOF camera device, and before performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image, the method further comprises:

[0022] performing normalization processing on the infrared image; and / or

[0023] Perform filtering processing on the image data containing the depth information.

[0024] In some preferred embodiments of the present invention, performing face region detection on the infrared image to obtain the face region in the infrared image includes:

[0025] Performing face region detection on the infrared image to obtain a series of face frames; and

[0026] The face frame with the largest size among the series of face frames is selected as the face frame, wherein the area included in the face frame is the face area in the infrared image.

[0027] In some preferred embodiments of the present invention, in the process of performing face region detection and face feature point detection on the infrared image to obtain the position information of the face region and face feature points in the infrared image, the method further comprises:

[0028] The face area is tracked.

[0029] In some preferred embodiments of the present invention, the method for detecting liveness of a face further comprises:

[0030] In response to not detecting a face region and / or a face feature point in the infrared image, it is determined that the face to be detected is a non-living face.

[0031] In some preferred embodiments of the present invention, performing facial feature point detection on the infrared image to obtain position information of facial feature points in the infrared image includes:

[0032] Performing face region detection on the infrared image to obtain a set of face feature points; and

[0033] The group of facial feature points is transformed to obtain five feature points, namely, a left eye, a right eye, a nose, a left corner of the mouth, and a right corner of the mouth, and their position information.

[0034] In some preferred embodiments of the present invention, performing face profile fitting on the face region in the image data containing depth information includes:

[0035] Performing horizontal fitting on the face area in the image data containing the depth information; and

[0036] Perform vertical fitting on the face area in the image data containing the depth information.

[0037] In some preferred embodiments of the present invention, after determining that the human face to be detected is a living human face, the method further includes:

[0038] Matching the human face to be tested in a human face database; and

[0039] In response to matching the human face to be tested in the human face database, a detection result related to the human face to be tested is generated, wherein the detection result includes personnel information corresponding to the human face to be tested in the human face database.

[0040] In some preferred embodiments of the present invention, the image data containing depth information is any one of point cloud data, depth image or RGB-D fusion image.

[0041] According to another aspect of the present invention, the present invention further provides a face liveness detection system, which comprises:

[0042] An image data acquisition unit, used to acquire an infrared image of the face to be tested and image data containing depth information collected by the TOF camera device;

[0043] A face detection unit, used to perform face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image; and

[0044] A liveness detection unit is used to perform liveness detection based on the face area, the face feature points and the image data containing depth information, wherein the liveness detection process includes:

[0045] In response to the size of the face region being within a preset range, performing face plane fitting on the face region in the image data containing the depth information;

[0046] In response to the result of the face plane fitting meeting a preset requirement, performing face side fitting on the face area in the image data containing the depth information; and

[0047] In response to the result of the face profile fitting meeting a preset requirement, it is determined that the face to be tested is a living face.

[0048] According to another aspect of the present invention, the present invention further provides an electronic device, comprising:

[0049] Processor; and

[0050] A memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes a method for detecting live faces, wherein the method for detecting live faces comprises:

[0051] Acquire an infrared image of the face to be tested and image data containing depth information collected by a TOF camera device;

[0052] Performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image;

[0053] Based on the face area, the face feature points and the image data containing the depth information, liveness detection is performed, wherein the liveness detection process includes:

[0054] In response to the size of the face region being within a preset range, performing face plane fitting on the face region in the image data containing the depth information;

[0055] In response to the result of the face plane fitting meeting a preset requirement, performing face side fitting on the face area in the image data containing the depth information; and

[0056] In response to the result of the face profile fitting meeting a preset requirement, it is determined that the face to be tested is a living face.

[0057] According to another aspect of the present invention, the present invention further provides a computer-readable storage medium, characterized in that the computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a computer device, it can be operated to perform a live face detection method, wherein the live face detection method includes:

[0058] Acquire an infrared image of the face to be tested and image data containing depth information collected by a TOF camera device;

[0059] Performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image;

[0060] Based on the face area, the face feature points and the image data containing the depth information, liveness detection is performed, wherein the liveness detection process includes:

[0061] In response to the size of the face region being within a preset range, performing face plane fitting on the face region in the image data containing the depth information;

[0062] In response to the result of the face plane fitting meeting a preset requirement, performing face side fitting on the face area in the image data containing the depth information; and

[0063] In response to the result of the face profile fitting meeting a preset requirement, it is determined that the face to be tested is a living face.

[0064] Other purposes and advantages of the present invention will be further reflected through the specific implementation manner and the contents of the claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0065] Figure 1 It is a flowchart of a method for face liveness detection according to a preferred embodiment of the present invention.

[0066] Figure 2 It is a flowchart of a method for face liveness detection according to a preferred embodiment of the present invention.

[0067] Figure 3 The figure is a schematic diagram of infrared image acquisition of a method for face liveness detection according to a preferred embodiment of the present invention.

[0068] Figure 4 1 is a schematic diagram of face tracking of a method for face liveness detection according to a preferred embodiment of the present invention.

[0069] Figure 5 It is a schematic diagram of side result fitting of a face liveness detection method according to a preferred embodiment of the present invention.

[0070] Figure 6 It is a structural schematic diagram of a face liveness detection system according to a preferred embodiment of the present invention. DETAILED DESCRIPTION

[0071] The following description is used to disclose the present invention so that those skilled in the art can implement the present invention. The preferred embodiments described below are only examples, and those skilled in the art can think of other obvious variations. The basic principles of the present invention defined in the following description can be applied to other embodiments, variations, improvements, equivalents, and other technical solutions that do not deviate from the spirit and scope of the present invention.

[0072] Those skilled in the art should understand that, in the disclosure of the present invention, the terms "longitudinal", "lateral", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside" and the like indicating the orientation or position relationship are based on the orientation or position relationship shown in the drawings, which are only for the convenience of describing the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operate in a specific orientation. Therefore, the above terms should not be understood as limiting the present invention.

[0073] It is to be understood that the term "one" should be understood as "at least one" or "one or more", that is, in one embodiment, the number of an element may be one, while in another embodiment, the number of the element may be multiple, and the term "one" should not be understood as a limitation on the quantity.

[0074] Reference Manual Attached Figures 1 to 6 , the face liveness detection method 100 provided by the present invention is described, which includes:

[0075] 101: Acquire an infrared image of a face to be tested and image data containing depth information collected by a TOF camera device;

[0076] 102: Performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image;

[0077] 103: Performing liveness detection based on the face area, the face feature points, and the image data containing the depth information, wherein the liveness detection process includes:

[0078] 1031: In response to a size of the face region being within a preset range, performing face plane fitting on the face region in the image data containing the depth information;

[0079] 1032: In response to the result of the face plane fitting meeting a preset requirement, performing face side fitting on the face region in the image data containing the depth information; and

[0080] 1033: In response to the result of the face profile fitting meeting a preset requirement, determining that the measured face is a live face.

[0081] In the step 101, in the step of acquiring an infrared image of the face to be tested and image data containing depth information captured by a TOF camera device, the infrared image of the face to be tested and the image data containing depth information are captured by the TOF camera device, wherein the image data containing depth information includes any one of point cloud data, depth image or RGB-D fusion image.

[0082] Furthermore, between the step of acquiring an infrared image of the face to be tested and image data containing depth information collected by the TOF camera device and the step of performing face area detection and face feature point detection on the infrared image to obtain the position information of the face area and face feature points in the infrared image, the following is further included:

[0083] 104: performing normalization processing on the infrared image; and / or

[0084] 105: Perform filtering processing on the image data containing the depth information.

[0085] In step 102, performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image further includes:

[0086] 106: Performing face region detection on the infrared image to obtain a series of face frames; and

[0087] 107: Selecting a face frame with the largest size among the series of face frames as the face frame, wherein the area included in the face frame is the face area in the infrared image.

[0088] In the step 106, face region detection is performed on the infrared image to obtain a series of face frames, and face frame detection is performed on the infrared image using the trained deep learning model to obtain a series of face frames. The sizes of all face frames in the series of face frames are calculated respectively, and the face frame with the largest size in the series of face frames is selected as the face frame, wherein the area included in the face frame is the face region in the infrared image.

[0089] Furthermore, in step 102, performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image further includes:

[0090] 108: Track the face area.

[0091] Reference Manual Attached Figure 4, it can be understood by those skilled in the art that, in the process of liveness detection, the position of the face to be tested relative to the TOF camera device is not fixed, that is, the distance, angle and other positional relationships of the face to be tested relative to the TOF camera device may change, thereby causing the size and position of the face area of ​​the face to be tested in the infrared image to change accordingly. For example, when the distance between the face to be tested and the TOF camera device increases, the face area of ​​the face to be tested becomes smaller in the infrared image, and when the distance between the face to be tested and the TOF camera device decreases, the face area of ​​the face to be tested becomes larger in the infrared image. Therefore, in the present invention, after determining the face area in the infrared image, a tracking algorithm is used to track the face area in real time so that the face area can truly reflect the face area of ​​the face to be tested.

[0092] Specifically, since the area contained in the largest face frame in the series of face frames is the face area in the infrared image, when tracking the face area, it is only necessary to track the largest face frame, that is, to use the tracking algorithm to track the largest face frame in real time.

[0093] Furthermore, in step 102, performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image further includes:

[0094] 109: Performing face region detection on the infrared image to obtain a set of face feature points; and

[0095] 1010: Convert the set of facial feature points to obtain five feature points of a left eye, a right eye, a nose, a left corner of the mouth, and a right corner of the mouth and their position information.

[0096] Preferably, in this preferred embodiment, face region detection is performed on the infrared image, and 68 face feature points are detected and obtained, and then the 68 face feature points are converted to obtain five feature points of left eye, right eye, nose, left corner of mouth and right corner of mouth and their position information. It should be understood by those skilled in the art that in other preferred embodiments of the present invention, the number of face feature points obtained by performing face region detection on the infrared image can also be other numbers. The five face feature points of left eye, right eye, nose, left corner of mouth and right corner of mouth obtained after performing face feature point conversion can also be other positions of the face region, such as eyebrows, cheeks, etc. As long as the purpose of the present invention can be achieved, the number of face feature points obtained by performing face region detection on the infrared image and the specific positions of the face feature points after performing face feature point conversion should not constitute limitations on the present invention.

[0097] Specifically, after acquiring the facial feature points and converting the facial feature points, Rodrigues is used to perform three-dimensional posture estimation, and the facial posture angle can be output, and Yaw, Pitch, and Roll are used for facial angle judgment.

[0098] In step 103, liveness detection is performed based on the face area, the face feature points and the image data containing depth information to determine whether the face to be detected is alive.

[0099] Specifically, in step 1031, in response to the size of the face area being within a preset range, in performing plane fitting on the face area in the image data containing depth information, the size of the face area is first calculated, and then a determination is made as to whether the size of the face area is within the preset range; when the size of the face area is within the preset range, plane fitting is performed on the face area in the image data containing depth information.

[0100] Preferably, in response to the area of ​​the face region being within the preset range, plane fitting is performed on the face region in the image data containing depth information. It is understandable that, since the face frame of the largest size surrounds the face region, the area of ​​the face region can be obtained by calculating the length and width of the face frame of the largest size and then calculating the area of ​​the face frame of the largest size. When the area of ​​the face region is within the preset area range, plane fitting is performed on the face region in the image data containing depth information.

[0101] In some other preferred embodiments of the present invention, it is also possible to determine whether the size of the face region is within the preset range by determining whether the length of the diagonal line of the face frame with the largest size is within a preset length range. When the length of the diagonal line of the face frame with the largest size is within the preset length range, it is determined that the size of the face region is within the preset range; when the length of the diagonal line of the face frame with the largest size is not within the preset length range, it is determined that the size of the face region is not within the preset range.

[0102] Reference Manual Attached Figure 3 It should be pointed out that by judging whether the size of the face area in the infrared image is within the preset range, the interference of photos and videos on face liveness detection can be quickly eliminated. It can be understood that the infrared light emitted by TOF is near-infrared through emission modulation, and no imaging is performed on electronic video screens and photos. Therefore, the interference of photos and videos can be quickly eliminated by judging whether the size of the face area in the infrared image is within the preset range.

[0103] Furthermore, between step 102 and step 103, the following steps are further included:

[0104] 1011: In response to not detecting a face region and / or a face feature point in the infrared image, determining that the face to be detected is a non-living face.

[0105] It is understandable that when the human face to be tested is a non-living object, such as a photo, video, or mold, the infrared image acquired by the TOF camera device cannot acquire the image of the non-living object, so the human face area and / or the human face feature points cannot be detected in the infrared image. Therefore, when the human face area and / or the human face feature points are not detected in the infrared image, it can be determined that the human face to be tested is a non-living object.

[0106] In some other preferred embodiments of the present invention, the determination that the human face to be tested is non-living can also be in response to the size of the human face area not being within the preset range, and the human face to be tested is determined to be non-living. It should be understood by those skilled in the art that in the actual detection process, it is often interfered by the outside world. Even when the TOF camera device acquires infrared images of non-living objects, such as photos, videos, and molds, it often collects some living information in the external environment, but usually has a large side size difference from the real living body. Therefore, when the size of the human face area is not within the preset range, it can be determined that the human face to be tested is non-living.

[0107] Furthermore, those skilled in the art will appreciate that a real living human face is three-dimensional, has projections and depressions, and is non-planar, whereas facial images on ordinary paper and coated paper are often planar. Therefore, by performing plane fitting on the facial region in the image data containing depth information and analyzing the plane fitting effect, it is possible to quickly distinguish between living and non-living objects.

[0108] Specifically, in step 1032, in response to the face plane fitting result meeting the preset requirement, the face area in the image data containing the depth information is fitted sideways. It can be understood that when the face plane fitting result meets the preset requirement, it can be explained to a certain extent that the human body to be tested is alive, and then the face area in the image data containing the depth information is fitted sideways. When the face plane fitting result does not meet the preset requirement, it can be shown that the human body to be tested is not alive, and the output is that the human body to be tested is not alive.

[0109] It should be pointed out that after the result of the face plane fitting meets the preset requirements, performing side fitting on the face area in the image data containing depth information again can improve the accuracy of face liveness detection to a certain extent and improve the reliability of face liveness detection.

[0110] In step 1033, in response to the result of the face side fitting meeting the preset requirements, the face to be tested is determined to be a live face, that is, when the result of the face side fitting meets the preset requirements, the face to be tested is determined to be a live face. It can be understood by those skilled in the art that the side of a real live face has obvious contour undulations, while non-live face images such as face images on ordinary paper and coated paper do not have the same undulation characteristics as real live faces when bent, so it is possible to judge whether the face to be tested is live based on the result of the face side fitting.

[0111] Furthermore, performing face profile fitting on the face area in the image data containing the depth information further includes:

[0112] 1012: Perform horizontal fitting on the face region in the image data containing the depth information; and

[0113] 1013: Perform vertical fitting on the face region in the image data containing the depth information.

[0114] That is, when the side fitting is performed on the face area in the image data containing the depth information, it is performed simultaneously in the horizontal direction and the vertical direction to increase the effect of the side fitting.

[0115] Preferably, when judging whether the result of the side face fitting meets the preset requirements, it is based on whether the contour fluctuation of the side face is within a preset fluctuation range. When the contour fluctuation of the side face is within the preset fluctuation range, it is determined that the result of the side face fitting meets the preset requirements, and the face to be tested is alive; when the contour fluctuation of the side face is not within the preset fluctuation range, it is determined that the result of the side face fitting does not meet the preset requirements, and the face to be tested is not alive. It should be understood by those skilled in the art that in other preferred embodiments of the present invention, when judging whether the result of the side face fitting meets the preset requirements, other judgment methods can also be used as the basis for judgment. As long as the purpose of the present invention can be achieved, the specific method of judging whether the result of the side face fitting meets the preset requirements should not constitute a limitation to the present invention.

[0116] Reference Manual Attached Figure 5 When the face to be tested is a real live face, the side profile fitting result has obvious contour fluctuations in both the top view and the side profile, while when the face to be tested is a face image on ordinary paper and coated paper, the side profile fitting result has no obvious contour fluctuations in both the top view and the side profile. Therefore, the contour fluctuations of the face side profile fitting can be used to distinguish whether the face to be tested is live or not.

[0117] It is worth mentioning that the face liveness detection method provided by the present invention integrates multiple ways of comprehensive judgment, including face area size judgment, face plane fitting and face side fitting, making full use of the differences between live and non-live objects, and can effectively exclude non-live objects such as printed paper, photos, videos, coated paper and molds. In addition, the accuracy of liveness judgment can be greatly improved by combining the properties of the infrared camera itself.

[0118] Furthermore, after determining that the face to be detected is a live face, the face liveness detection method provided by the present invention further comprises:

[0119] 1014: Matching the face to be tested in a face database; and

[0120] 1015: In response to matching the human face to be tested in the face database, generating a detection result related to the human face to be tested, wherein the detection result includes personnel information corresponding to the human face to be tested in the face database.

[0121] Specifically, in step 1014, matching the face to be tested in the face database includes:

[0122] Extracting a feature vector of the face to be tested; and

[0123] Matching is performed within the face database based on the face feature vector.

[0124] Specifically, extracting the feature vector of the face to be tested includes:

[0125] The facial feature vector of the face to be tested is extracted by a feature extraction model trained by deep learning.

[0126] Matching in the face database based on the face feature vector includes:

[0127] The extracted facial feature vector of the face to be tested is compared with the facial feature vector stored in the face database to match the face to be tested.

[0128] The face database pre-stores preset face information, which includes a face feature vector and corresponding personnel information. The face feature vector is used to match the face to be tested, and the personnel information is used to indicate the information of the preset face, such as one or more combinations of name, gender, photo, ID number, work number, etc.

[0129] In step 1014, in matching the face to be tested in the face database, the acquired feature vector of the face to be tested is used as a matching condition, and a preset face having a corresponding facial feature vector as the face to be tested is matched in the face database.

[0130] In the step 1015, in response to matching the face to be tested in the face database, a detection result related to the face to be tested is generated, wherein the detection result includes personnel information corresponding to the face to be tested in the face database, wherein the personnel information includes one or more combinations of name, gender, photo, ID number, work number, etc.

[0131] In this preferred embodiment, when the face to be tested is matched in the face database, a detection result related to the face to be tested is generated. Preferably, the detection result includes the personnel information corresponding to the face to be tested in the face database. In other preferred embodiments of the present invention, the detection result related to the face to be tested can also include a control instruction, and the control instruction is used to control the opening of the door to facilitate the passage of the person with the identified face to be tested. It should be understood by those skilled in the art that the control instruction can also control the execution of other actions. As long as the purpose of the invention of the present invention can be achieved, the specific action controlled by the control instruction should not constitute a limitation on the present invention. It should be understood by those skilled in the art that in other preferred embodiments of the present invention, the detection result related to the face to be tested can also be other information. As long as the purpose of the invention of the present invention can be achieved, the specific form of the detection result related to the face to be tested generated should not constitute a limitation on the present invention.

[0132] Reference Manual Attached Figure 6 According to another aspect of the present invention, the present invention further provides a face liveness detection system, which includes an image data acquisition unit 10, a face detection unit 20 and a liveness detection unit 30, wherein the image data acquisition unit 10 is used to acquire an infrared image of a face to be detected and image data containing depth information collected by a TOF camera device, the face detection unit 20 is used to perform face area detection and face feature point detection on the infrared image to obtain position information of the face area and face feature points in the infrared image, and the liveness detection unit 30 is used to perform liveness detection based on the face area, the face feature points and the image data containing depth information, wherein the liveness detection process includes:

[0133] In response to the size of the face region being within a preset range, performing face plane fitting on the face region in the image data containing the depth information;

[0134] In response to the result of the face plane fitting meeting a preset requirement, performing face side fitting on the face area in the image data containing the depth information; and

[0135] In response to the result of the face profile fitting meeting a preset requirement, it is determined that the face to be tested is a living face.

[0136] Furthermore, the living body detection unit 30 is further configured to determine that the face to be detected is a non-living face in response to failure to detect a face region and / or face feature points in the infrared image.

[0137] The image data containing depth information is any one of point cloud data, depth image or RGB-D fusion image.

[0138] Furthermore, the face liveness detection system further includes a data preprocessing unit 40, and the data preprocessing unit 40 is used to normalize the infrared image and / or filter the image data containing depth information. It should also be pointed out that after the data preprocessing unit 40 normalizes the infrared image and / or filters the image data containing depth information, the face detection unit 20 performs face area detection and face feature point detection on the infrared image.

[0139] The face detection unit 20 further includes a face frame detection module 21 and a size comparison module 22, wherein the face frame detection module 21 is used to perform face area detection on the infrared image to obtain a series of face frames; the size comparison module 22 is used to select the face frame with the largest size in the series of face frames as the face frame, wherein the area included in the face frame with the largest size is the face area in the infrared image.

[0140] The face detection unit 20 further includes a face tracking module 23, and the face tracking module 23 is used to track the face area in the infrared image. It can be understood that since the area included in the face frame with the largest size is the face area, in other words, the face tracking module 23 tracks the face frame with the largest size in real time for tracking the face area in the infrared image.

[0141] Preferably, the face tracking module 23 uses a tracking algorithm to track the face area in real time.

[0142] Furthermore, the face detection unit 20 further includes a feature point acquisition module 24 and a feature point conversion module 25, wherein the feature point acquisition module 24 is used to acquire a group of face feature points corresponding to the face area in the infrared image, and the feature point conversion module 25 is used to convert the group of face feature points acquired by the feature point acquisition module 24 to obtain five feature points of the left eye, right eye, nose, left corner of the mouth and right corner of the mouth and their position information.

[0143] Specifically, the liveness detection unit 30 further includes a face plane fitting module 31 and a face side fitting module 32, wherein the face plane fitting module 31 is used to perform face plane fitting on the face area in the image data containing depth information in response to the size of the face area being within a preset range, and the face side fitting module 32 is used to perform side fitting on the face area in the image data containing depth information in response to the result of the face plane fitting meeting the preset requirements, and the face side fitting module 32 is also used to determine that the face to be tested is a live face in response to the result of the face side fitting meeting the preset requirements.

[0144] Furthermore, the face liveness detection system provided by the present invention further includes a face data storage unit 50 and a face matching unit 60, the face data storage unit 50 is used to store preset face data, the face matching unit 60 is used to match the face to be tested based on the face data storage unit 50, and when the face to be tested is matched in the face data storage unit 50, a detection result related to the face to be tested is generated, wherein the detection result includes the personnel information corresponding to the face to be tested in the face database.

[0145] Preferably, the preset face data stored in the face data storage unit 50 includes preset face feature points and information of the person corresponding to the preset face, such as name, gender, photo, ID number, and work number. The face data storage unit 50 can also be called a face database. Accordingly, when the face matching unit 60 matches the face to be tested in the face data storage unit 50, a detection result related to the face to be tested is generated, wherein the detection result includes the person information corresponding to the face to be tested in the face database, wherein the person information includes a combination of one or more of name, gender, photo, ID number, or work number.

[0146] In some other variant embodiments of the present invention, when the face to be tested is matched in the face data storage unit 50, the detection result generated related to the face to be tested can also include a control instruction or is itself a control instruction, and the control instruction is used to control the opening of a control device, such as a door access system, to allow a person with the face to be tested to pass. It should be understood by those skilled in the art that as long as the purpose of the present invention can be achieved, the specific type of the detection result generated related to the face to be tested should not constitute a limitation to the present invention.

[0147] According to another aspect of the present invention, the present invention further provides an electronic device, including a processor and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the above-mentioned face liveness detection method provided by the present invention.

[0148] According to another aspect of the present invention, the present invention further provides a computer-readable storage medium, on which computer program instructions are stored. When the computer program instructions are executed by a computer device, they can be operated to execute the above-mentioned face liveness detection method provided by the present invention.

[0149] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0150] It should be understood by those skilled in the art that the embodiments of the present invention described above and shown in the accompanying drawings are only examples and do not limit the present invention. The purpose of the present invention has been fully and effectively achieved. The functional and structural principles of the present invention have been demonstrated and explained in the embodiments, and the embodiments of the present invention may be deformed or modified in any way without departing from the principles.

Claims

1. A method for detecting liveness of a face, characterized in that: include: Acquire an infrared image of the face to be tested and image data containing depth information collected by a TOF camera device; Performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image; Based on the face area, the face feature points and the image data containing the depth information, liveness detection is performed, wherein the liveness detection process includes: In response to the size of the face region being within a preset range, performing face plane fitting on the face region in the image data containing the depth information; In response to the result of the face plane fitting meeting the preset requirement, performing face side fitting on the face area in the image data containing the depth information, so as to distinguish the face to be tested as a living body or a non-living body according to the contour fluctuation of the face side fitting; and In response to the result of the face profile fitting meeting a preset requirement, determining that the face to be tested is a live face; Performing face side fitting on the face area in the image data containing the depth information includes: performing horizontal fitting on the face area in the image data containing the depth information and performing vertical fitting at the same time.

2. The method for face liveness detection according to claim 1, wherein: After acquiring an infrared image of a human face to be tested and image data containing depth information collected by a TOF camera device, and before performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image, the method further includes: performing normalization processing on the infrared image; and / or Perform filtering processing on the image data containing the depth information.

3. The method for face liveness detection according to claim 1, wherein: Performing face region detection on the infrared image to obtain the face region in the infrared image includes: Performing face region detection on the infrared image to obtain a series of face frames; and The face frame with the largest size among the series of face frames is selected as the face frame, wherein the area included in the face frame is the face area in the infrared image.

4. The method for face liveness detection according to claim 3, wherein: In the process of performing face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image, the method further comprises: The face area is tracked.

5. The method for face liveness detection according to claim 1, further comprising: In response to not detecting a face region and / or a face feature point in the infrared image, it is determined that the face to be detected is a non-living face.

6. The method for face liveness detection according to claim 4, wherein: Performing facial feature point detection on the infrared image to obtain position information of facial feature points in the infrared image includes: Performing face region detection on the infrared image to obtain a set of face feature points; and The group of facial feature points is transformed to obtain five feature points, namely, a left eye, a right eye, a nose, a left corner of the mouth, and a right corner of the mouth, and their position information.

7. The method for face liveness detection according to claim 1, after determining that the face to be detected is a live face, further comprising: Matching the face to be tested in a face database; and In response to matching the human face to be tested in the human face database, a detection result related to the human face to be tested is generated, wherein the detection result includes personnel information corresponding to the human face to be tested in the human face database.

8. The method for face liveness detection according to any one of claims 1 to 7, wherein: The image data containing depth information is any one of point cloud data, depth image or RGB-D fusion image.

9. A face liveness detection system, characterized in that: include: An image data acquisition unit, used to acquire an infrared image of the face to be measured and image data containing depth information collected by the TOF camera device; A face detection unit, used to perform face region detection and face feature point detection on the infrared image to obtain position information of the face region and face feature points in the infrared image; as well as A liveness detection unit is used to perform liveness detection based on the face area, the face feature points and the image data containing depth information, wherein the liveness detection process includes: In response to the size of the face region being within a preset range, performing face plane fitting on the face region in the image data containing the depth information; In response to the result of the face plane fitting meeting the preset requirements, the face side fitting is performed on the face area in the image data containing the depth information, so as to distinguish the face to be tested as a living body or a non-living body through the contour fluctuation of the face side fitting; wherein, the face side fitting is performed on the face area in the image data containing the depth information, including: performing horizontal fitting on the face area in the image data containing the depth information and performing vertical fitting at the same time; and In response to the result of the face profile fitting meeting a preset requirement, it is determined that the face to be tested is a living face.

10. An electronic device, characterized in that: include: processor; as well as A memory, in which computer program instructions are stored, and when the computer program instructions are executed by the processor, the processor executes the face liveness detection method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer program instructions, and when the computer program instructions are executed by a computer device, they can be operated to execute the face liveness detection method according to any one of claims 1 to 8.

Images