Living face recognition method based on color image and near infrared image

Abstract

The present invention discloses a method for live face recognition based on color images and near-infrared images, comprising the following steps: performing face recognition on the color image and the near-infrared image respectively; detecting the positions of the facial features on the face recognition result, and projecting the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix, and judging whether it is a live face according to the projection result. The method for live face recognition based on color images and near-infrared images of the present invention first obtains the rotation matrix and the translation matrix between the near-infrared camera and the color camera, and then performs face recognition on the color image and the near-infrared image respectively, and finally projects the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix, thereby judging whether it is a live face according to the projection result, reducing the influence of the surrounding environment during face recognition, and improving the reliability of face recognition.

Description

Technical Field

[0001] The invention belongs to the technical field of face recognition, and in particular relates to a living face recognition method based on color images and near-infrared images. Background Art

[0002] At present, face recognition technology has become increasingly mature and is widely used in unattended occasions. Common face recognition systems include live face detection and detection based on three-dimensional facial information.

[0003] However, in actual use, face recognition technology still has many insurmountable difficulties. Among them, most of the difficulties come from the complexity of the face as an identification feature, that is, it is more sensitive to the influence of the environment. At the same time, the surrounding environment, such as light intensity, direction of light source and ambient color, will cause the image of the face to change greatly. These factors make it difficult to detect and recognize faces. The complexity and variability of facial expressions, as well as factors such as the expressions of faces in different images are very different, so the facial posture presented has great uncertainty, as well as problems such as occlusion, which also poses great difficulties for face detection. Summary of the invention

[0004] In order to solve the above problems, the present invention provides a living face recognition method based on color images and near-infrared images, which reduces the influence of the surrounding environment during face recognition and improves the reliability of face recognition.

[0005] The technical solution adopted by the present invention is:

[0006] A method for live face recognition based on color images and near infrared images comprises the following steps:

[0007] S1, performing stereo matching between a near infrared camera and a color camera, and obtaining a rotation matrix and a translation matrix between the two cameras according to the relative posture between the near infrared camera and the color camera;

[0008] S2, reading in a color image and a near-infrared image and performing face recognition on the color image and the near-infrared image respectively to obtain a face recognition result;

[0009] S3. Detect the positions of the facial features on the face recognition result, and project the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix, and determine whether it is a living face based on the projection result.

[0010] Preferably, S2 specifically includes the following steps:

[0011] S21, read in color images and near infrared images;

[0012] S22, performing a first face recognition on the color image and the near-infrared image by using a recursive convolutional neural network algorithm, if the recognition is successful, obtaining a first face recognition result, otherwise, returning to S21;

[0013] S23, performing a second face recognition on the first face recognition result through a support vector machine, if the recognition is successful, obtaining the second face recognition result, otherwise, returning to S21.

[0014] Preferably, in S23, the second face recognition is specifically a binary face recognition of face and non-face.

[0015] Preferably, S3 specifically includes the following steps:

[0016] S31, detecting the positions of the facial features on the face recognition result;

[0017] S32, projecting the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix;

[0018] S33, determine whether the error of the facial features coordinates detected by the near-infrared image projected on the facial features coordinates detected by the color image is less than a preset threshold. If so, determine it as a living face. Otherwise, an alarm is issued and return to S2.

[0019] Preferably, in S32, the specific calculation formula for projecting the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix is:

[0020] AR+T=A'

[0021] Wherein, A is the coordinate on the near-infrared image, A' is the coordinate projected onto the color image, R is the rotation matrix of the near-infrared camera and the color camera, and T is the translation matrix of the near-infrared camera and the color camera.

[0022] Preferably, in S33, the calculation formula for the error of the facial features coordinates detected by the near-infrared image projected on the facial features coordinates detected by the color image is:

[0023]

[0024] Where Δx Li =|x′ Li -x Li | is the difference between the horizontal coordinates of the facial features in the near-infrared image and the horizontal coordinates of the facial features in the color image, Δy Li =|y′ Li -y′ Li| is the difference between the vertical coordinates of the facial features in the near-infrared image and the vertical coordinates of the facial features in the color image, i is the label of the facial features, and N is a natural number.

[0025] Preferably, in S33, the preset threshold is 30 pixel values.

[0026] Preferably, the five senses include two eyes, a nose and a mouth.

[0027] Preferably, in S3, an open source computer vision function library is used to detect the positions of facial features based on the face recognition result.

[0028] Preferably, in S1, an open source computer vision function library is used to perform stereo matching between the near-infrared camera and the color camera.

[0029] Compared with the prior art, the live face recognition method based on color images and near-infrared images of the present invention first obtains the rotation matrix and translation matrix between the near-infrared camera and the color camera, then performs face recognition on the color image and the near-infrared image respectively, and finally projects the coordinates of the facial features detected in the near-infrared image onto the coordinates of the facial features detected in the color image through the rotation matrix and the translation matrix, thereby judging whether it is a live face based on the projection result, reducing the influence of the surrounding environment during face recognition, and improving the reliability of face recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a flow chart of a method for live face recognition based on color images and near infrared images provided by an embodiment of the present invention;

[0031] Figure 2 It is a specific flow chart of a living face recognition method based on color images and near-infrared images provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0033] The embodiment of the present invention provides a method for live face recognition based on color images and near infrared images, such as Figure 1-2 As shown, the following steps are included:

[0034] S1, performing stereo matching between a near infrared camera and a color camera, and obtaining a rotation matrix and a translation matrix between the two cameras according to the relative posture between the near infrared camera and the color camera;

[0035] S2, reading in a color image and a near-infrared image and performing face recognition on the color image and the near-infrared image respectively to obtain a face recognition result;

[0036] S3. Detect the positions of the facial features on the face recognition result, and project the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix, and determine whether it is a living face based on the projection result.

[0037] In this way, by first obtaining the rotation matrix and translation matrix between the near-infrared camera and the color camera, and then performing face recognition on the color image and the near-infrared image respectively, and finally projecting the coordinates of the facial features detected in the near-infrared image onto the coordinates of the facial features detected in the color image through the rotation matrix and translation matrix, it is possible to determine whether it is a living face based on the projection result, reduce the influence of the surrounding environment during face recognition, and improve the reliability of face recognition.

[0038] Wherein, in said S3, an open source computer vision library (ie, Opencv library) is used to detect the positions of the facial features on the face recognition result. In said S1, an open source computer vision library (ie, Opencv library) is used to perform stereo matching between the near infrared camera and the color camera.

[0039] Among them, the OpenCV function library is a cross-platform computer vision library released under the BSD license (open source), which can run on Linux, Windows, Android and Mac OS operating systems. It is lightweight and efficient - consisting of a series of C functions and a small number of C++ classes, while providing interfaces for languages ​​such as Python, Ruby, and MATLAB, and implementing many common algorithms in image processing and computer vision.

[0040] The S2 specifically includes the following steps:

[0041] S21, read in color images and near infrared images;

[0042] S22, performing a first face recognition on the color image and the near-infrared image by using a recursive convolutional neural network algorithm, if the recognition is successful, obtaining a first face recognition result, otherwise, returning to S21;

[0043] S23, performing a second face recognition on the first face recognition result through a support vector machine, if the recognition is successful, obtaining the second face recognition result, otherwise, returning to S21.

[0044] In this way, a color image and a near-infrared image with a face are first read in, and then a first face recognition is performed on the color image and the near-infrared image through a recursive convolutional neural network algorithm (i.e., the Faster-RCNN algorithm) to obtain a first face recognition result. If no face is recognized, the process returns to S21; then a second face recognition is performed on the first face recognition result through a support vector machine to obtain a second face recognition result. If no face is recognized, the process returns to S21; wherein, the second face recognition is specifically a two-category face recognition of face and non-face, and this recognition is mainly used to improve the accuracy of face recognition.

[0045] Among them, the Faster-RCNN algorithm is a selective search algorithm that completes the extraction of target candidate areas, reduces time and space overhead, achieves fast and accurate detection, and realizes end-to-end target (face) detection.

[0046] The S3 specifically includes the following steps:

[0047] S31, detecting the positions of the facial features on the face recognition result;

[0048] S32, projecting the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix;

[0049] S33, determine whether the error of the facial features coordinates detected by the near-infrared image projected on the facial features coordinates detected by the color image is less than a preset threshold. If so, determine it as a living face. Otherwise, an alarm is issued and return to S2.

[0050] In this way, the positions of the facial features are first detected on the face recognition results (including near-infrared face recognition results and color face recognition results), and then the coordinates of the facial features detected by the near-infrared image are projected onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix. Finally, it is determined whether the error of the coordinates of the facial features detected by the near-infrared image projected onto the coordinates of the facial features detected by the color image is less than a preset threshold. If so, it is determined to be a living face. Otherwise, an alarm is issued and the process returns to S2.

[0051] Among them, in S32, the specific calculation formula for projecting the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through the rotation matrix and the translation matrix is:

[0052] AR+T=A'

[0053] Wherein, A is the coordinate on the near-infrared image, A' is the coordinate projected onto the color image, R is the rotation matrix of the near-infrared camera and the color camera, and T is the translation matrix of the near-infrared camera and the color camera.

[0054] In this way, by projecting the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix, a specific calculation formula can be used to quantify the projection result, and it can be more accurately determined whether the coordinates of the facial features detected by the near-infrared image are projected onto the coordinates of the facial features detected by the color image.

[0055] Ideally, the coordinates of points projected from the near-infrared camera to the color camera, such as the eyes, nose, and mouth, coincide with the coordinates calculated on the color image. However, there is generally a certain error in actual calculations. Therefore, in S33, the calculation formula for the error of the coordinates of the facial features detected by the near-infrared image projected on the coordinates of the facial features detected by the color image is:

[0056]

[0057] Where Δx Li =|x′ Li -x Li | is the difference between the horizontal coordinates of the facial features in the near-infrared image and the horizontal coordinates of the facial features in the color image, Δy Li =|y′ Li -y′ Li | is the difference between the vertical coordinates of the facial features in the near-infrared image and the vertical coordinates of the facial features in the color image, i is the label of the facial features, and N is a natural number.

[0058] When ε is less than the threshold, it is considered that the face detected on the color image and the face detected on the near-infrared image are the same face, and the face with temperature is a living face, otherwise it is not. In this empirical value: the threshold of ε is 30 pixels.

[0059] The five senses include two eyes, a nose and a mouth.

[0060] In this way, the corresponding face is detected through the five senses including two eyes, a nose and a mouth. At the same time, other facial features such as eyebrows and ears can also be detected in the later more accurate face recognition process.

[0061] The live face recognition method based on color images and near-infrared images of the present invention first obtains the rotation matrix and translation matrix between the near-infrared camera and the color camera, then performs face recognition on the color image and the near-infrared image respectively, and finally projects the coordinates of the facial features detected by the near-infrared image onto the coordinates of the facial features detected by the color image through the rotation matrix and the translation matrix, thereby judging whether it is a live face according to the projection result, reducing the influence of the surrounding environment during face recognition, and improving the reliability of face recognition.

[0062] The above is only a preferred specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention should be based on the protection scope of the claims.

Claims

1. A method for live face recognition based on color images and near infrared images, characterized in that: The steps include: S1, performing stereo matching between a near infrared camera and a color camera, and obtaining a rotation matrix and a translation matrix between the two cameras according to the relative posture between the near infrared camera and the color camera; S21, read in color images and near infrared images; S22, performing a first face recognition on the color image and the near-infrared image by using a recursive convolutional neural network algorithm, if the recognition is successful, obtaining a first face recognition result, otherwise, returning to S21; S23, performing a second face recognition on the first face recognition result by using a support vector machine, if the recognition is successful, obtaining the second face recognition result, otherwise, returning to S21; S31, detecting the positions of the facial features on the face recognition result; S32, projecting the coordinates of the facial features detected by the near infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix; the specific calculation formula for projecting the coordinates of the facial features detected by the near infrared image onto the coordinates of the facial features detected by the color image through a rotation matrix and a translation matrix is: AR+T=A' Wherein, A is the coordinate on the near-infrared image, A' is the coordinate projected onto the color image, R is the rotation matrix of the near-infrared camera and the color camera, and T is the translation matrix of the near-infrared camera and the color camera; S33, determine whether the error of the facial features coordinates detected by the near-infrared image projected on the facial features coordinates detected by the color image is less than a preset threshold. If so, determine it as a living face. Otherwise, an alarm is issued and return to S2.

2. The method for live face recognition based on color images and near infrared images according to claim 1, characterized in that: In S23, the second face recognition is specifically a two-category face recognition of face and non-face.

3. The method for live face recognition based on color images and near infrared images according to claim 2, characterized in that: In S33, the calculation formula of the error of the coordinates of the facial features detected by the near-infrared image projected on the coordinates of the facial features detected by the color image is: Where Δx Li =|x′ Li -x Li | is the difference between the horizontal coordinates of the facial features in the near-infrared image and the horizontal coordinates of the facial features in the color image, Δy Li =|y′ Li -y′ Li | is the difference between the vertical coordinates of the facial features in the near-infrared image and the vertical coordinates of the facial features in the color image, i is the label of the facial features, and N is a natural number.

4. The method for live face recognition based on color images and near infrared images according to claim 3 is characterized in that: In S33, the preset threshold is 30 pixel values.

5. The method for live face recognition based on color images and near infrared images according to any one of claims 1 to 4, characterized in that: The five senses include two eyes, a nose and a mouth.

6. The method for live face recognition based on color images and near infrared images according to any one of claims 1 to 4, characterized in that: In S3, an open source computer vision function library is used to detect the positions of the facial features based on the face recognition result.

7. The method for live face recognition based on color images and near infrared images according to any one of claims 1 to 4, characterized in that: In S1, an open source computer vision function library is used to perform stereo matching between the near-infrared camera and the color camera.

Images