Living body detection method, device, detector, medium and intelligent device

Abstract

The present invention discloses a method, device, detector, medium and intelligent device for detecting live bodies. The method comprises: obtaining a first two-dimensional face image and a second two-dimensional face image of a photographed object; extracting feature points of the first two-dimensional face image and the second two-dimensional face image; performing pose estimation according to the feature points to obtain three-dimensional information of the feature points; analyzing whether preset conditions are met based on the three-dimensional information; if the preset conditions are met, obtaining a result that the photographed object is non-live; if the preset conditions are not met, obtaining a result that the photographed object is live. The present invention can reduce the cost of implementing live body detection.

Description

Technical Field

[0001] The present invention relates to the field of machine vision technology, and in particular to a living body detection method, device, detector, medium and intelligent equipment. Background Art

[0002] In the field of face recognition, in order to prevent forged face attacks, liveness detection based on face recognition is usually required, that is, to determine whether the current face is real.

[0003] Liveness detection based on face recognition is divided into cooperative liveness detection and silent liveness detection. Cooperative liveness detection requires users to complete specified actions according to instructions, which has poor user experience and slow speed. Silent liveness detection is a non-sensing liveness detection that is completed without user cooperation.

[0004] At present, there are generally two ways to detect silent liveness. One is to use binocular cameras, stereo cameras, RGB-D (depth) cameras, etc. to collect three-dimensional information of the target, which can complete fake face attacks on 2D media (printed photos, display screens, etc.), but the hardware cost is relatively high. The other is to use a monocular camera to collect images for liveness detection. Usually, a deep network is used to estimate the depth of the collected images and identify 2D fake faces, but it has a large amount of calculations and requires a large amount of data for model training, which is difficult to implement. Summary of the invention

[0005] The technical problem to be solved by the present invention is that the hardware cost of silent liveness detection in the prior art is high and the implementation is difficult, resulting in high implementation cost.

[0006] In order to solve the above technical problems, the present invention provides a living body detection method, apparatus, detector, medium and intelligent device.

[0007] A liveness detection method, comprising:

[0008] Acquire a first two-dimensional face image and a second two-dimensional face image of the photographed object;

[0009] Extracting feature points of the first two-dimensional face image and the second two-dimensional face image;

[0010] Performing pose estimation based on the feature points to obtain three-dimensional information of the feature points;

[0011] Analyzing whether a preset condition is met based on the three-dimensional information;

[0012] If the preset condition is met, a result is obtained that the photographed object is a non-living object;

[0013] If the preset condition is not met, a result is obtained that the photographed object is a living body.

[0014] In one embodiment, obtaining a first two-dimensional face image and a second two-dimensional face image of a photographed object includes:

[0015] Selecting a frame of two-dimensional face image from the two-dimensional face image of the photographed object to obtain the first two-dimensional face image;

[0016] A two-dimensional face image whose face center relative displacement from the first two-dimensional face image is within a preset range is selected as the second two-dimensional face image.

[0017] In one embodiment, performing pose estimation based on the feature points to obtain three-dimensional information of the feature points includes:

[0018] Matching feature points in the first two-dimensional face image and the second two-dimensional face image to find a matching point pair;

[0019] Performing a reprojection error analysis based on the matching point pairs to obtain an error evaluation value;

[0020] Calculating a confidence level according to the error evaluation value;

[0021] If the confidence level is greater than or equal to a preset value, a result is obtained that the photographed object is a non-living object;

[0022] If the confidence level is less than the preset value, the three-dimensional information of the feature point is obtained.

[0023] In one embodiment, the error evaluation value includes a first error evaluation value and a second error evaluation value; and performing reprojection error analysis based on the matching point pair to obtain the error evaluation value includes:

[0024] Adopting an iterative algorithm, taking a preset number of matching point pairs as the minimum data set for multiple times, solving the homography matrix and basic matrix of each minimum data set;

[0025] In each iteration, the reprojection error of the matching point pair under the homography matrix transformation is calculated and scored to obtain the first error evaluation value;

[0026] In each iteration, the reprojection error of the matching point pair under the basic matrix transformation is calculated and scored to obtain a second error evaluation value;

[0027] A maximum first error evaluation value is selected from the first error evaluation values ​​of the multiple iterations, and a maximum second error evaluation value is selected from the second error evaluation values ​​of the multiple iterations.

[0028] In one embodiment, the step of calculating the reprojection error of the matching point pair under the homography matrix transformation and scoring to obtain a first error evaluation value includes:

[0029] Based on the homography matrix corresponding to the matching point pair, calculating the square error of the displacement of the matching point pair after projecting from the first two-dimensional face image to the second two-dimensional face image, to obtain a first error term of the matching point pair from the first two-dimensional face image to the second two-dimensional face image;

[0030] Based on the homography matrix corresponding to the matching point pair, calculating the square error of the displacement of the matching point pair after projecting from the second two-dimensional face image to the first two-dimensional face image, to obtain a first error term of the matching point pair from the second two-dimensional face image to the first two-dimensional face image;

[0031] Compare the first error term of each matching point pair with a first preset threshold, and select the first error term that is smaller than the first preset threshold;

[0032] Calculating the difference between the first preset threshold and the selected first error term to obtain a first error score;

[0033] The first error scores are accumulated to obtain the first error evaluation value.

[0034] In one embodiment, the step of calculating the reprojection error of the matching point pair under the basic matrix transformation and scoring to obtain a second error evaluation value includes:

[0035] The matching point pair is transformed from the first two-dimensional face image to the second two-dimensional face image into a first epipolar line through the basic matrix, and the distance from the feature point in the second two-dimensional face image in the matching point pair to the first epipolar line is calculated to obtain a second error term of the matching point pair from the first two-dimensional face image to the second two-dimensional face image;

[0036] The matching point pair is transformed from the second two-dimensional face image to the first two-dimensional face image into a second epipolar line through the basic matrix, and the distance from the feature point in the first two-dimensional face image in the matching point pair to the second epipolar line is calculated to obtain a second error term of the matching point pair from the second two-dimensional face image to the first two-dimensional face image;

[0037] Compare the second error term of each matching point pair with a second preset threshold, and select a second error term that is smaller than the second preset threshold;

[0038] Calculate the difference between the first preset threshold and the selected second error term to obtain a second error score;

[0039] The second error scores are accumulated to obtain the second error evaluation value.

[0040] In one embodiment, the matching of feature points in the first two-dimensional face image and the second two-dimensional face image, after finding a matching point pair, and performing reprojection error analysis based on the matching point pair before obtaining an error evaluation value, further comprises:

[0041] Determining the number of matching point pairs;

[0042] If the number is less than the preset number, the process returns to the step of acquiring the first two-dimensional face image and the second two-dimensional face image of the photographed object.

[0043] In one embodiment, obtaining the three-dimensional information of the feature point includes:

[0044] Obtaining a rotation matrix and a translation matrix based on the feature points;

[0045] Based on the rotation matrix and the translation matrix, the three-dimensional information of the feature points is obtained by using triangulation method.

[0046] In one embodiment, analyzing whether a preset condition is met based on the three-dimensional information includes:

[0047] Based on the three-dimensional information of the feature points, a best fitting plane and a sum of squares of distances from each feature point to the best fitting plane are obtained;

[0048] Compare the distance sum squared with a first preset threshold, and count the number of distance sum squared values ​​that are less than or equal to the first preset threshold;

[0049] Calculate the ratio of the number of statistics to the total number of feature points;

[0050] Calculate the normalized variance based on the three-dimensional information of each feature point;

[0051] If the ratio is greater than a preset ratio and the normalized variance is greater than a preset variance, the preset condition is met.

[0052] A living body detection device, comprising:

[0053] An image acquisition module, used to acquire a first two-dimensional face image and a second two-dimensional face image of a photographed object;

[0054] A feature point extraction module, used to extract feature points of the first two-dimensional face image and the second two-dimensional face image;

[0055] A three-dimensional information obtaining module, used to perform pose estimation based on the feature points and obtain the three-dimensional information of the feature points;

[0056] The condition analysis module is used to analyze whether a preset condition is met based on the three-dimensional information; if the preset condition is met, a result is obtained that the photographed object is non-living; if the preset condition is not met, a result is obtained that the photographed object is living.

[0057] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0058] Acquire a first two-dimensional face image and a second two-dimensional face image of the photographed object;

[0059] Extracting feature points of the first two-dimensional face image and the second two-dimensional face image;

[0060] Performing pose estimation based on the feature points to obtain three-dimensional information of the feature points;

[0061] Analyzing whether a preset condition is met based on the three-dimensional information;

[0062] If the preset condition is met, a result is obtained that the photographed object is a non-living object;

[0063] If the preset condition is not met, a result is obtained that the photographed object is a living body.

[0064] A detector comprises a memory and a processor, wherein the memory stores a computer program, and the processor implements the following steps when executing the computer program:

[0065] Acquire a first two-dimensional face image and a second two-dimensional face image of the photographed object;

[0066] Extracting feature points of the first two-dimensional face image and the second two-dimensional face image;

[0067] Performing pose estimation based on the feature points to obtain three-dimensional information of the feature points;

[0068] Analyzing whether a preset condition is met based on the three-dimensional information;

[0069] If the preset condition is met, a result is obtained that the photographed object is a non-living object;

[0070] If the preset condition is not met, a result is obtained that the photographed object is a living body.

[0071] A smart device comprises a device body, a camera and the above-mentioned detector, wherein the detector is connected to the device body.

[0072] In one embodiment, the camera is a monocular camera.

[0073] Compared with the prior art, one or more embodiments of the above scheme may have the following advantages or beneficial effects:

[0074] Two two-dimensional face images of the photographed object are used to estimate the posture and obtain three-dimensional information. Based on the three-dimensional information, it is analyzed whether the photographed object is alive, so as to realize silent liveness detection. On the one hand, two-dimensional face images can be collected and photographed with a low-cost camera. Compared with the solutions using binocular cameras, stereo cameras, and RGB-D (depth) cameras in the prior art, the hardware cost is lower. On the other hand, by estimating the posture, obtaining three-dimensional information, and completing liveness detection based on the three-dimensional information, there is no need to use a deep network or a large amount of data to train the model. Compared with the solutions using deep networks in the prior art, the difficulty of implementation is greatly reduced. Therefore, the implementation cost of liveness detection can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] The scope of the present disclosure may be better understood by reading the following detailed description of exemplary embodiments in conjunction with the accompanying drawings. The drawings included are:

[0076] Figure 1 is a schematic diagram of a flow chart of a liveness detection method in an embodiment;

[0077] Figure 2 is a schematic flow chart of a liveness detection method in another embodiment;

[0078] Figure 3 FIG. 4 is a structural block diagram of a living body detection device in an embodiment. DETAILED DESCRIPTION

[0079] In order to make the objectives, technical solutions and advantages of the present invention clearer, the implementation method of the present invention will be described in detail below in conjunction with the accompanying drawings and embodiments, so that the implementation process of how the present invention applies technical means to solve technical problems and achieve technical effects can be fully understood and implemented accordingly.

[0080] In one embodiment, a method for detecting a living body is provided. Figure 1 As shown, the method comprises the following steps:

[0081] S110: Acquire a first two-dimensional face image and a second two-dimensional face image of the photographed object.

[0082] The photographed object refers to an object for which liveness detection is required, such as a human face. The first two-dimensional human face image and the second two-dimensional human face image are two-dimensional images obtained by photographing the photographed object; specifically, there is a certain displacement between the first two-dimensional human face image and the second two-dimensional human face image.

[0083] A two-dimensional face image can be obtained by photographing the subject with a camera / camera that shoots two-dimensional images, such as a monocular camera; when the face is close to the monocular camera, image acquisition is performed continuously, and two frames of two-dimensional face images with a certain displacement are selected from the acquired images to obtain a first two-dimensional face image and a second two-dimensional face image.

[0084] S120: Extracting feature points of the first two-dimensional face image and the second two-dimensional face image.

[0085] The feature points are relatively prominent key points in the image; corresponding to the two-dimensional face image, the extracted feature points are face feature points. Specifically, a feature point extraction method is used to extract the feature points of the first two-dimensional face image and the feature points of the second two-dimensional face image.

[0086] S130: Estimating the position and posture of the feature points to obtain the three-dimensional information of the feature points.

[0087] The three-dimensional information includes three-dimensional space coordinates.

[0088] S140: Analyze whether preset conditions are met based on the three-dimensional information.

[0089] The preset condition is a condition that the feature points located in the same plane meet, and can be set according to actual needs. If the preset condition is met based on the three-dimensional information analysis, step S150 is executed; if the preset condition is not met based on the three-dimensional information analysis, step S160 is executed.

[0090] S150: Obtaining a result that the photographed object is a non-living object.

[0091] S160: Obtaining a result that the photographed object is a living body.

[0092] After step S150 and step S160, the result of the liveness detection may be outputted through the information output device.

[0093] The above-mentioned liveness detection method uses two two-dimensional face images of the photographed object to estimate the posture and obtain three-dimensional information, and analyzes whether the photographed object is alive based on the three-dimensional information to achieve silent liveness detection. On the one hand, two-dimensional face images can be collected and photographed using a low-cost camera. Compared with the solutions using binocular cameras, stereo cameras, and RGB-D (depth) cameras in the prior art, the hardware cost is lower. On the other hand, by estimating the posture, obtaining three-dimensional information, and completing liveness detection based on the three-dimensional information, there is no need to use a deep network or a large amount of data to train the model. Compared with the solutions using deep networks in the prior art, the difficulty of implementation is greatly reduced, and therefore the implementation cost of liveness detection can be reduced.

[0094] In one embodiment, step S110 includes: selecting a frame of two-dimensional face image from the two-dimensional face image of the photographed object to obtain a first two-dimensional face image; selecting a two-dimensional face image whose relative displacement from the face center of the first two-dimensional face image is within a preset range as a second two-dimensional face image.

[0095] Among them, the preset range can be preset according to actual needs. For example, the preset range can be [0.05, 0.1], that is, an interval greater than or equal to 0.05 and less than 0.1. After collecting the two-dimensional face image, the coordinates of the face center of the two-dimensional face image can be obtained, and the relative displacement with the face center of the first two-dimensional face image can be calculated, and a frame of two-dimensional face image with a relative displacement within the preset range is selected as the second two-dimensional face image. By selecting two two-dimensional face images with a relative displacement of the face center within the preset range, it is ensured that there is a certain displacement between the first two-dimensional face image and the second two-dimensional face image, thereby improving the accuracy of detection.

[0096] For example, after a monocular camera detects a face for the first time, it starts collecting images for liveness detection, continuously records image frames containing the face, and records the coordinates of the center of the face; selects the two-dimensional face image in which the face is detected for the first time as the first two-dimensional face image I1, selects the two-dimensional face image with a relative displacement of [0.05, 0.1] with respect to the face center of I1 as the second two-dimensional face image I2, and performs grayscale conversion on the two frames of images to obtain grayscale images.

[0097] In one of the embodiments, a two-dimensional facial image whose relative displacement from the facial center of the first two-dimensional facial image is within a preset range is selected as the second two-dimensional facial image, including: calculating the relative displacement between the facial center of the first two-dimensional facial image and the facial center of the two-dimensional facial image collected later, and if the relative displacement is within the preset range, selecting the corresponding two-dimensional facial image as the second two-dimensional facial image.

[0098] The step of calculating the relative displacement between the face center of the first two-dimensional face image and the face center of the two-dimensional face image collected later includes:

[0099]

[0100] The face center coordinates of the first two-dimensional face image I1 are (x1, y1), the face center coordinates of the second two-dimensional face image I2 are (x2, y2), w is the width of the face boundary, and h is the height of the face boundary.

[0101] In one embodiment, step S120 includes: performing ORB (Oriented Fast and Rotated Brief) feature extraction on the face areas of the first two-dimensional face image and the second two-dimensional face image respectively to obtain feature points of the first two-dimensional face image and the second two-dimensional face image.

[0102] Through ORB feature extraction, the processing speed is fast. It can be understood that in other embodiments, other methods can be used to implement feature extraction, and the extracted feature points can meet the scale transformation invariance and rotation transformation invariance, such as SURF feature extraction.

[0103] In one embodiment, S130 includes steps (a1) to (a5).

[0104] Step (a1): Match feature points in the first two-dimensional face image and the second two-dimensional face image to find matching point pairs.

[0105] When quickly acquiring the feature points of the face area, its feature description is also acquired; according to the feature point descriptions of the two two-dimensional face images, the feature points are matched to find the matching point pairs [keypoints1, keypoints2] in the two two-dimensional face images. For example, the ORB detection and key point matching modules in OpenCV can be used for implementation.

[0106] Step (a2): Perform reprojection error analysis based on matching point pairs to obtain an error evaluation value.

[0107] Step (a3): Calculate the confidence level based on the error evaluation value.

[0108] If the confidence level is greater than or equal to the preset value, execute step (a4); otherwise, if the confidence level is less than the preset value, execute step (a5).

[0109] Step (a4): Obtain a result that the photographed object is a non-living object.

[0110] Step (a5): Obtain the three-dimensional information of the feature points.

[0111] The error evaluation value obtained by reprojection error analysis is used to calculate the confidence level, and a preliminary judgment is made on the liveness detection based on the confidence level to improve the accuracy of liveness detection. If the confidence level is greater than or equal to the preset value, it indicates that the feature points are in the same plane and can be judged as non-live. If the confidence level is less than the preset value, the next step is continued to obtain the three-dimensional information. The preset value can be preset according to the actual situation. For example, the preset value can be 0.7.

[0112] In one embodiment, the error evaluation value includes a first error evaluation value and a second error evaluation value. Step (a2) includes steps (a21) to (a24).

[0113] Step (a21): using an iterative algorithm, taking a preset number of matching point pairs as the minimum data set for multiple times, and solving the homography matrix and the basic matrix of each minimum data set.

[0114] Specifically, in each iteration, a preset number of matching point pairs is used as the minimum data set, and the homography matrix and basic matrix of each minimum data set are solved. The preset number can be set specifically according to actual needs. For example, the preset number can be 8; eight points are randomly selected from all matching point pairs as the minimum data set, and the eight-point method is used to solve the homography matrix H and basic matrix F of the face posture change.

[0115] Step (a22): In each iteration, the reprojection error of the matching point pairs under the homography matrix transformation is calculated and scored to obtain a first error evaluation value.

[0116] Specifically, according to the homography matrix corresponding to the matching point pair and the data set to which the matching point pair belongs, the reprojection error is calculated and scored to obtain the first error evaluation value SH of the current iteration. For example, the chi-square test can be used to score the reprojection error.

[0117] Step (a23): In each iteration, the reprojection error of the matching point pairs under the basic matrix transformation is calculated and scored to obtain a second error evaluation value.

[0118] Specifically, according to the basic matrix corresponding to the matching point pairs and the data set to which the matching point pairs belong, the reprojection error is calculated and scored to obtain the second error evaluation value SF of the current iteration.

[0119] Step (a24): Select the largest first error evaluation value from the first error evaluation values ​​of multiple iterations, and select the largest second error evaluation value from the second error evaluation values ​​of multiple iterations.

[0120] By adopting an iterative algorithm, a preset number of matching point pairs are taken as a data set each time, and each data set can calculate a set of homography matrix H and basic matrix F, and then calculate the first error evaluation value SH and the second error evaluation value SF for all matching point pairs until the maximum number of iterations (for example, 200 times by default), and finally take a maximum value from all SHs obtained in the iterations as the final SH; take a maximum value from all SFs obtained in the iterations as the final SF; in this way, the best homography matrix and basic matrix, as well as the corresponding best first error evaluation value SH and second error evaluation value SF can be solved. For example, the RANSAC (Random sample consensus) iterative algorithm can be used for iterative solution.

[0121] In one embodiment, step (a22) calculates the reprojection error of the matching point pair under the homography matrix transformation and scores it to obtain a first error evaluation value, including steps (a221) to (a225).

[0122] Step (a221): Based on the homography matrix corresponding to the matching point pair, calculate the square error of the displacement of the matching point pair after projecting from the first two-dimensional face image to the second two-dimensional face image, and obtain the first error term of the matching point pair from the first two-dimensional face image to the second two-dimensional face image.

[0123] The homography matrix corresponding to the matching point pair is the homography matrix corresponding to the data set to which the matching point pair belongs. The squared displacement error reflects the projection error of the matching point pair. For example, suppose the transformation matrix from the first two-dimensional face image I1 to the second two-dimensional face image I2 is H 12 (Then the transformation matrix from I2 to I1 is inv(H 12 ), namely H 12 Inverse of H, for each pair of matching points of I1 and I2 (p1(i), p2(i)), i represents the i-th matching point pair, first calculate the displacement square error of the matching point pair after projection from I1 to I2, that is, ||p2(i)-p1(i)*H 12||2 , and obtain the first error term of the matching point pair from I1 to I2.

[0124] Step (a222): Based on the homography matrix corresponding to the matching point pair, calculate the square error of the displacement of the matching point pair after projecting from the second two-dimensional face image to the first two-dimensional face image, and obtain the first error term of the matching point pair from the second two-dimensional face image to the first two-dimensional face image.

[0125] Step (a223): compare the first error term of each matching point pair with a first preset threshold, and select the first error term that is smaller than the first preset threshold.

[0126] Specifically, the first error term of the matching point pair from I1 to I2 is compared with the first preset threshold, and the first error term of the matching point pair from I2 to I1 is compared with the first preset threshold, and the first error term that is smaller than the first preset threshold is selected. The first preset threshold can be preset. For example, the first preset threshold is a critical threshold corresponding to a chi-square distribution with a degree of freedom of 2 and a significance level of 0.05, specifically 5.991.

[0127] Step (a224): Calculate the difference between the first preset threshold and the selected first error term to obtain a first error score.

[0128] Specifically, the first error score is obtained by calculating the difference between the first preset threshold and the selected first error term.

[0129] Step (a225): Accumulate the first error scores to obtain a first error evaluation value.

[0130] For example, for the homography matrix H, from I1 to I2: all matching point pairs are distinguished by the chi-square test. The first error term (error1) is greater than the first preset threshold th (the critical threshold corresponding to the chi-square distribution with 2 degrees of freedom and a significance level of 0.05), indicating that it is an external point (i.e., it does not conform to the transformation model). The first error term is less than the first preset threshold, which is an internal point (conforms to the transformation model). The internal point is used to calculate the first error score (th-error1); similarly, the first error score from I2 to I1 is calculated. Traverse all matching point pairs, accumulate all first error scores, and finally output the first error evaluation value SH. Through the homography matrix H, the two-frame image bidirectional projection error analysis and scoring are performed with high accuracy.

[0131] In one embodiment, step (a23) calculates the reprojection error of the matching point pair under the basic matrix transformation and scores it to obtain a second error evaluation value, including steps (a231) to (a235).

[0132] Step (a231): The matching point pairs from the first two-dimensional face image to the second two-dimensional face image are transformed into the first epipolar line via the basic matrix, and the distance from the feature point in the second two-dimensional face image in the matching point pair to the first epipolar line is calculated to obtain the second error term of the matching point pair from the first two-dimensional face image to the second two-dimensional face image.

[0133] Step (a232): The matching point pairs from the second two-dimensional face image to the first two-dimensional face image are transformed into second epipolar lines via the basic matrix, and the distance from the feature points in the first two-dimensional face image in the matching point pairs to the second epipolar lines is calculated to obtain the second error term of the matching point pairs from the second two-dimensional face image to the first two-dimensional face image.

[0134] Step (a233): compare the second error term of each matching point pair with a second preset threshold, and select the second error term that is smaller than the second preset threshold.

[0135] Specifically, the second error term of the matching point pair from I1 to I2 is compared with the second preset threshold, and the second error term of the matching point pair from I2 to I1 is compared with the second preset threshold, and the second error term smaller than the second preset threshold is selected. For example, the second preset threshold is a critical threshold corresponding to a chi-square distribution with a degree of freedom of 1 and a significance level of 0.05, specifically 3.841.

[0136] Step (a234): Calculate the difference between the first preset threshold and the selected second error term to obtain a second error score.

[0137] Specifically, the second error score is obtained by calculating the difference between the first preset threshold and the selected second error term. The first preset threshold is 5.991.

[0138] Step (a235): Accumulate the second error scores to obtain a second error evaluation value.

[0139] For example, for the basic matrix F, calculate the distance from the feature point to the epipolar line: Assume that the transformation matrix from I1 to I2 is F 12 (Then the transformation matrix from I2 to I1 is inv(F 12 ), that is, F 12 For each pair of matching points of I1 and I2 (p1(i), p2(i)), i represents the i-th matching point pair. The matching point pair from I1 to I2 is transformed into the epipolar line l2 by F, and the distance from p2(i) to the epipolar line is calculated:

[0140]

[0141] All matching points are distinguished by the chi-square test. If the second error term (error2) is greater than the second preset threshold, it indicates that it is an external point (i.e., it does not conform to the transformation model), and if it is less than the second preset threshold, it is an internal point (conform to the transformation model). The second error score is calculated using the internal point. Similarly, the second error score from I2 to I1 is calculated. All matching point pairs are traversed, all second error scores are accumulated, and finally the second error evaluation value SF is output. Through the basic matrix F, the two-frame image bidirectional projection error analysis and scoring are performed with high accuracy.

[0142] In one embodiment, after step (a1) and before step (a2), the method further includes: determining the number of matching point pairs; if the number is less than a preset number, returning to step S110.

[0143] By performing statistical analysis on the number of matching point pairs, when the number is less than a preset number, returning to step S110 to reselect the first two-dimensional face image and the second two-dimensional face image, it is ensured that the matching point pairs are not less than the preset number to ensure subsequent reprojection error analysis.

[0144] In one embodiment, step (a3) ​​comprises:

[0145]

[0146] conf represents confidence, SH is the first error evaluation value, and SF is the second error evaluation value; conf ≥ 0.7 indicates that the feature points are in the same plane, and it can be judged as non-living.

[0147] In one embodiment, the step of obtaining the three-dimensional information of the feature points includes: obtaining a rotation matrix and a translation matrix based on the feature points; and obtaining the three-dimensional information of the feature points using a triangulation method based on the rotation matrix and the translation matrix.

[0148] Specifically, the rotation matrix R and the translation matrix T can be obtained based on the basic matrix F of the matching point pair, and the three-dimensional information of the feature points on I1 at the same scale can be calculated by combining the matching point pair and triangulation. For example, projectionsFromFundamental() in Opencv can be used to complete the acquisition of three-dimensional information.

[0149] In one embodiment, step S140 includes steps (b1) to (b5).

[0150] Step (b1): Based on the three-dimensional information of the feature points, the best fitting plane and the sum of the squares of the distances from each feature point to the best fitting plane are obtained.

[0151] Specifically, the best fitting plane is obtained based on the three-dimensional information of the feature points of the first two-dimensional face image, or the best fitting plane is obtained based on the three-dimensional information of the feature points of the second two-dimensional face image. The best fitting plane is obtained to obtain a plane equation. Specifically, the RANSAC iteration method can be used to obtain the best plane fit. Three non-collinear points are randomly selected each time to determine a plane equation. The sum of the squares of the distances from all feature points on a two-dimensional face image to the plane equation is calculated; the plane equation with the smallest sum of squares of distances in multiple iterations is selected, which corresponds to the best fitting plane.

[0152] Step (b2): compare the distance sum squared with a first preset threshold, and count the number of distance sum squared values ​​that are less than or equal to the first preset threshold.

[0153] The square sum of the distances from each feature point to the best fitting plane is taken as the error value, and the critical threshold corresponding to the chi-square distribution with 2 degrees of freedom and a significance level of 0.05 is used as the first preset threshold to compare with the error value, so as to judge the internal point. If the error value is greater than the first preset threshold, it is an outlier, and if it is less than or equal to the first preset threshold, it is an internal point.

[0154] Step (b3): ​​Calculate the ratio of the statistical number to the total number of feature points.

[0155] The total number of feature points is the total number of feature points on a two-dimensional face image.

[0156] Step (b4): Calculate the normalized variance based on the three-dimensional information of each feature point.

[0157] Step (b5): If the ratio is greater than the preset ratio and the normalized variance is greater than the preset variance, the preset condition is met.

[0158] The preset ratio can be 0.7, and the preset variance can be 1. If the ratio of the number of inliers to the total number R>0.7 and the normalized variance>1, it indicates that the feature points are on the same plane, that is, the photographed object is not alive, otherwise it is alive. By performing inliers ratio analysis and normalized variance calculation analysis, liveness judgment is performed with high accuracy.

[0159] In one embodiment, step (b4) includes: normalizing the three-dimensional information of each feature point, and calculating a normalized variance based on the normalized value.

[0160] Specifically, normalize to [-1, 1], for the point (x i ,y i , z i ) to get the normalized point (x′ i , y' i , z′ i ), the normalized calculation of xi' is as follows:

[0161] x max =max(x1, x2, ..., x n )

[0162] x min =min(x1, x2, ..., x n )

[0163]

[0164]

[0165] Similarly, we can get y' i , z' i .

[0166] Compute the normalized variance using the normalized inliers:

[0167]

[0168]

[0169]

[0170]

[0171] The calculation is simplified by normalization.

[0172] like Figure 2 The figure shows a flowchart of a detailed implementation example.

[0173] It should be understood that although Figure 1-Figure 2The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1-Figure 2 At least part of the steps may include multiple steps or multiple stages. These steps or stages are not necessarily performed at the same time, but can be performed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed in turn or alternately with other steps or at least part of the steps or stages in other steps.

[0174] In one embodiment, Figure 3 As shown, a living body detection device is provided, comprising:

[0175] The image acquisition module 310 is used to acquire a first two-dimensional face image and a second two-dimensional face image of the photographed object; the feature point extraction module 320 is used to extract feature points of the first two-dimensional face image and the second two-dimensional face image; the three-dimensional information acquisition module 330 is used to perform pose estimation based on the feature points and obtain three-dimensional information of the feature points; the condition analysis module 340 is used to analyze whether the preset conditions are met based on the three-dimensional information; if the preset conditions are met, the result is obtained that the photographed object is non-living; if the preset conditions are not met, the result is obtained that the photographed object is living.

[0176] The above-mentioned liveness detection device uses two two-dimensional face images of the photographed object to estimate the posture and obtain three-dimensional information, and analyzes whether the photographed object is alive based on the three-dimensional information to achieve silent liveness detection. On the one hand, to collect two-dimensional face images, a low-cost camera can be used for shooting. Compared with the solutions using binocular cameras, stereo cameras, and RGB-D (depth) cameras in the prior art, the hardware cost is lower. On the other hand, by estimating the posture, obtaining three-dimensional information, and completing liveness detection based on the three-dimensional information, there is no need to use a deep network or a large amount of data to train the model. Compared with the solutions using deep networks in the prior art, the difficulty of implementation is greatly reduced, and therefore the implementation cost of liveness detection can be reduced.

[0177] For the specific definition of the liveness detection device, please refer to the definition of the liveness detection method above, which will not be repeated here. Each module in the above-mentioned liveness detection device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the detector in the form of hardware, or can be stored in the memory in the detector in the form of software, so that the processor can call and execute the operations corresponding to the above modules. It should be noted that the division of modules in the embodiments of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0178] In one embodiment, a detector is provided, including a memory and a processor. The memory stores a computer program, and the processor implements the steps of the method in the above embodiments when executing the computer program.

[0179] The above detector can implement the steps of the methods in the above embodiments, and similarly, the cost of implementing liveness detection can be reduced.

[0180] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method in the above embodiments are implemented.

[0181] The computer-readable storage medium can implement the steps of the methods in the above embodiments, and similarly, can reduce the cost of implementing liveness detection.

[0182] A smart device comprises a device body, a camera and the detector in the above embodiment, wherein the detector is connected to the device body.

[0183] In one embodiment, the camera is a monocular camera.

[0184] The above-mentioned smart device adopts the detector in the above-mentioned embodiment, and similarly, the cost of implementing living body detection is low.

[0185] Those of ordinary skill in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory or optical memory, etc. Volatile memory can include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0186] The technical features of the above embodiments can be combined arbitrarily.

[0187] All possible combinations of various technical features in the embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0188] Although the embodiments disclosed in the present invention are as above, the contents described are only embodiments adopted for facilitating the understanding of the present invention and are not intended to limit the present invention. Any technician in the technical field to which the present invention belongs can make any modifications and changes in the form and details of the implementation without departing from the spirit and scope disclosed in the present invention, but the protection scope of the present invention shall still be subject to the scope defined in the attached claims.

Claims

1. A method for detecting a living body, characterized in that: include: Acquire a first two-dimensional face image and a second two-dimensional face image of the photographed object; Extracting feature points of the first two-dimensional face image and the second two-dimensional face image; Matching feature points in the first two-dimensional face image and the second two-dimensional face image to find a matching point pair; Performing a reprojection error analysis based on the matching point pairs to obtain an error evaluation value; Calculating a confidence level according to the error evaluation value; if the confidence level is greater than or equal to a preset value, obtaining a result that the photographed object is a non-living object; If the confidence level is less than the preset value, the three-dimensional information of the feature point is obtained; wherein the error evaluation value includes a first error evaluation value and a second error evaluation value; and the reprojection error analysis based on the matching point pair is performed to obtain the error evaluation value, including: Adopting an iterative algorithm, taking a preset number of matching point pairs as the minimum data set for multiple times, solving the homography matrix and basic matrix of each minimum data set; In each iteration, the reprojection error of the matching point pair under the homography matrix transformation is calculated and scored to obtain the first error evaluation value; In each iteration, the reprojection error of the matching point pair under the basic matrix transformation is calculated and scored to obtain a second error evaluation value; Selecting a maximum first error evaluation value from the first error evaluation values ​​of the multiple iterations, and selecting a maximum second error evaluation value from the second error evaluation values ​​of the multiple iterations; Analyzing whether a preset condition is met based on the three-dimensional information; If the preset condition is met, a result is obtained that the photographed object is a non-living object; If the preset condition is not met, a result is obtained that the photographed object is a living body.

2. The method according to claim 1, characterized in that The step of acquiring the first two-dimensional face image and the second two-dimensional face image of the photographed object comprises: Selecting a frame of two-dimensional face image from the two-dimensional face image of the photographed object to obtain the first two-dimensional face image; A two-dimensional face image whose face center relative displacement from the first two-dimensional face image is within a preset range is selected as the second two-dimensional face image.

3. The method according to claim 1, characterized in that The step of calculating the reprojection error of the matching point pair under the homography matrix transformation and scoring to obtain a first error evaluation value includes: Based on the homography matrix corresponding to the matching point pair, calculating the square error of the displacement of the matching point pair after projecting from the first two-dimensional face image to the second two-dimensional face image, to obtain a first error term of the matching point pair from the first two-dimensional face image to the second two-dimensional face image; Based on the homography matrix corresponding to the matching point pair, calculating the square error of the displacement of the matching point pair after projecting from the second two-dimensional face image to the first two-dimensional face image, to obtain a first error term of the matching point pair from the second two-dimensional face image to the first two-dimensional face image; Compare the first error term of each matching point pair with a first preset threshold, and select the first error term that is smaller than the first preset threshold; Calculating the difference between the first preset threshold and the selected first error term to obtain a first error score; The first error scores are accumulated to obtain the first error evaluation value.

4. The method according to claim 1, characterized in that: The step of calculating the reprojection error of the matching point pair under the basic matrix transformation and scoring to obtain a second error evaluation value includes: The matching point pair is transformed from the first two-dimensional face image to the second two-dimensional face image into a first epipolar line through the basic matrix, and the distance from the feature point in the second two-dimensional face image in the matching point pair to the first epipolar line is calculated to obtain a second error term of the matching point pair from the first two-dimensional face image to the second two-dimensional face image; The matching point pair is transformed from the second two-dimensional face image to the first two-dimensional face image into a second epipolar line through the basic matrix, and the distance from the feature point in the first two-dimensional face image in the matching point pair to the second epipolar line is calculated to obtain a second error term of the matching point pair from the second two-dimensional face image to the first two-dimensional face image; Compare the second error term of each matching point pair with a second preset threshold, and select a second error term smaller than the second preset threshold; Calculate the difference between the second preset threshold and the selected second error term to obtain a second error score; The second error scores are accumulated to obtain the second error evaluation value.

5. The method according to claim 1, characterized in that After matching the feature points in the first two-dimensional face image and the second two-dimensional face image to find a matching point pair, and before performing a reprojection error analysis based on the matching point pair to obtain an error evaluation value, the method further includes: Determining the number of matching point pairs; If the number is less than the preset number, the process returns to the step of acquiring the first two-dimensional face image and the second two-dimensional face image of the photographed object.

6. The method according to claim 1, characterized in that The obtaining of the three-dimensional information of the feature point comprises: Obtaining a rotation matrix and a translation matrix based on the feature points; Based on the rotation matrix and the translation matrix, the three-dimensional information of the feature points is obtained by using triangulation method.

7. The method according to claim 1, characterized in that The analyzing whether a preset condition is met based on the three-dimensional information includes: Based on the three-dimensional information of the feature points, a best fitting plane and a sum of squares of distances from each feature point to the best fitting plane are obtained; Compare the distance sum squared with a first preset threshold, and count the number of distance sum squared values ​​that are less than or equal to the first preset threshold; Calculate the ratio of the number of statistics to the total number of feature points; Calculate the normalized variance based on the three-dimensional information of each feature point; If the ratio is greater than a preset ratio and the normalized variance is greater than a preset variance, the preset condition is met.

8. A living body detection device, characterized in that: include: An image acquisition module, used to acquire a first two-dimensional face image and a second two-dimensional face image of a photographed object; A feature point extraction module, used to extract feature points of the first two-dimensional face image and the second two-dimensional face image; A three-dimensional information obtaining module, used to match feature points in the first two-dimensional face image and the second two-dimensional face image to find a pair of matching points; Performing a reprojection error analysis based on the matching point pairs to obtain an error evaluation value; Calculating a confidence level according to the error evaluation value; if the confidence level is greater than or equal to a preset value, obtaining a result that the photographed object is a non-living object; If the confidence level is less than the preset value, the three-dimensional information of the feature point is obtained; wherein the error evaluation value includes a first error evaluation value and a second error evaluation value; and the reprojection error analysis based on the matching point pair is performed to obtain the error evaluation value, including: Adopting an iterative algorithm, taking a preset number of matching point pairs as the minimum data set for multiple times, solving the homography matrix and basic matrix of each minimum data set; In each iteration, the reprojection error of the matching point pair under the homography matrix transformation is calculated and scored to obtain the first error evaluation value; In each iteration, the reprojection error of the matching point pair under the basic matrix transformation is calculated and scored to obtain a second error evaluation value; Selecting a maximum first error evaluation value from the first error evaluation values ​​of the multiple iterations, and selecting a maximum second error evaluation value from the second error evaluation values ​​of the multiple iterations; The condition analysis module is used to analyze whether a preset condition is met based on the three-dimensional information; if the preset condition is met, a result is obtained that the photographed object is non-living; if the preset condition is not met, a result is obtained that the photographed object is living.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A detector comprising a memory and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

11. A smart device, characterized in that: The device comprises a device body, a camera and a detector as claimed in claim 10, wherein the detector is connected to the device body.

12. The smart device according to claim 11, characterized in that: The camera is a monocular camera.

Images