A TOF-based method for face liveness detection

Abstract

The present invention discloses a TOF-based face liveness detection method, which relates to the field of image recognition. The present invention processes and stores images, and performs multiple liveness discriminations and votings, taking into account both the multi-frame joint judgment mechanism and the multi-frame sequence timing mechanism, and combining with the imaging characteristics of TOF to make the liveness detection take into account real-time performance, stability and reliability.

Description

Technical Field

[0001] The present invention relates to the field of image recognition, and more particularly to a TOF-based face liveness detection method. Background Art

[0002] Face recognition is regarded as the most typical and successful implementation case of artificial intelligence in the past decade, and has been widely applied in many scenarios such as transportation, security, finance, education, medical care, policing, and e-commerce. With the wide use of face recognition technology and the entry of more and more face recognition products into the market, its security has received more and more attention, especially in scenarios involving personal and property safety such as security and finance.

[0003] Liveness detection (anti-spoof) mainly determines whether the detected face at the front end is a real face or an attacking face, adding reliability and security to identity ID verification. In the existing liveness detection solutions, according to the imaging technology, they can be divided into monocular, binocular, structured light, and TOF. The monocular liveness detection algorithm has poor robustness and usually requires estimating the depth image with a huge amount of calculation; the binocular liveness detection is similar to the monocular detection. Similarly, due to the large amount of calculation and the passive imaging method, the algorithm robustness is easily affected by the environment; the structured light liveness detection, like the TOF liveness detection, uses the imaging method of an active light source, and the algorithm robustness is less affected by the ambient light compared with binocular and monocular, but it is still greatly affected by the ambient light in an extremely strong light environment. At the same time, the imaging quality at medium and long distances is also lower than that of TOF; the TOF-based liveness detection solution can not only achieve similar accuracy to structured light in terms of algorithm performance robustness, but also is less affected by the environment and can work all-weather. The existing TOF liveness detection solutions mainly combine the depth map and the near-infrared IR image to judge the liveness image. Due to the characteristics of TOF physical imaging (only accepting laser of a specific frequency band), it has a natural resistance to 2D plane type attack methods. The existing solutions defend against 2D plane type attacks by processing the depth image. For 3D type attacks such as headgear and masks, since they have a similar structure on the 3D imaging physical plane, the existing solutions use the method of multi-modal data fusion to perform frame-by-frame prediction and multi-frame voting judgment. The defect of such a solution is that it fails to consider the temporal and spatial changes and rigid motion of the human face. In the defense against high-fidelity head molds and masks, the single-frame output of the model only considers the information of the human face at a certain spatio-temporal node, which is just a discrete point sampling. At this time, the model is easily broken by the attack system.

[0004] In the existing TOF live detection solutions, a mechanism of single-frame plus multi-frame joint judgment or a mechanism of single-frame plus multi-frame sequence model is adopted to directly give the result on a frame sequence. Both of these two solutions cannot take into account both the performance stability and the real-time response of the system. Therefore, how to solve the above problems is urgently needed to be studied by those skilled in the art. Summary of the Invention

[0005] A TOF-based face live detection method proposed by the present invention takes into account both the multi-frame joint judgment mechanism and the multi-frame sequence mechanism, and combines with the imaging characteristics of TOF to make the live detection take into account real-time performance, stability and reliability.

[0006] In order to achieve the above object, the present invention adopts the following technical solutions:

[0007] A TOF-based face live detection method includes the following steps:

[0008] Obtain a depth image and a near-infrared image, denoted as Depth and IR respectively;

[0009] Preprocess the depth image and the near-infrared image;

[0010] Calculate the surface normal vector RGB image of the preprocessed depth image, denoted as DRGB;

[0011] Send DRGB and IR into the DMODEL and GMODEL models for a live score judgment;

[0012] Output the current frame score, and add the current frame IR and Depth to IRQ and DQ respectively;

[0013] Judge whether the lengths of IRQ and DQ are greater than or equal to 3. If it is judged that the lengths of IRQ and DQ are less than 3, then obtain the next frame of IR and Depth and repeat the above operations;

[0014] If it is judged that the lengths of IRQ and DQ are greater than or equal to 3, then at this time, perform 3 dequeue operations on IRQ and DQ respectively (the lengths of IRQ and DQ are reset to zero), obtain 3 frames of IR images and 3 frames of Depth images, calculate the LBP-TOP features of the preprocessed depth image and the preprocessed infrared image respectively, and send them into the TMODEL model for a live judgment score calculation;

[0015] Comprehensively vote and calculate the scores corresponding to the three frames (each frame of IR + Depth) dequeued from IRQ and DQ (the scores are calculated by DMODEL and GMODEL) and the scores calculated by TMODEL based on the three frames for a total of four scores to obtain the final live score. If there are three or more results of live in the voting results, it is live, otherwise it is non-live.

[0016] Optionally, the preprocessing of the depth image and the near-infrared image is specifically as follows:

[0017] Perform bilateral filtering on the depth image and dynamic range compression on the near-infrared image to obtain a filtered depth image and a near-infrared image dynamically compressed to 8 bits;

[0018] Perform dynamic range compression on the depth image and median filtering on the near-infrared image to obtain a depth image compressed to 8 bits and a filtered near-infrared image.

[0019] Optionally, the bilateral filtering of the depth image and the dynamic range compression of the near-infrared image are specifically as follows:

[0020] S11: Perform bilateral filtering on the depth image;

[0021] S12: Calculate the gray histogram of the original near-infrared image;

[0022] S13: Calculate the numerical normalization threshold according to the gray histogram calculated in S12;

[0023] S14: Perform MinMax normalization calculation on the near-infrared image according to the threshold calculated in S13.

[0024] Optionally, the dynamic range compression of the depth image and the median filtering of the near-infrared image are specifically as follows:

[0025] S21: Calculate the gray histogram of the depth map and calculate the dynamic normalization threshold;

[0026] S22: Perform MinMax normalization on the depth map according to the threshold obtained in S21 to obtain a normalized 8-bit depth image;

[0027] S23: Perform median filtering operation on the near-infrared image to obtain a filtered near-infrared image.

[0028] Optionally, calculating the surface normal vector RGB image of the preprocessed depth image is specifically as follows:

[0029] Step S31: Perform Pad operations on the image obtained in step S22 along the H direction and the W direction respectively to obtain an image with a height of H + 2 and a width of W + 2, denoted as HP;

[0030] Step S32: Calculate the gradients in the H direction and the W direction of the image obtained in step S31 respectively to obtain two gradient images with the same size as the image obtained in step S31, denoted as HT and WT respectively. At the same time, construct a gradient map in the Z direction with the same size as HT and WT, and each element value is 1, denoted as ZT;

[0031] Step S33: Calculate the adaptive coefficient Map in the grayscale space. Map has the same size as the gradient image, and each element on Map is the square root of the sum of the squares of the gradient values at the corresponding positions on HT, WT, and ZT.

[0032] Step S34: Calculate the adaptive grayscale maps in the H, W, and Z directions respectively. Taking the calculation of the adaptive grayscale in the H direction as an example, access each element on HT, and record the element grayscale value as hi. At the same time, take out the grayscale value at the corresponding position on Map and record it as mi. At this time, the grayscale value at this position on the H adaptive grayscale map is calculated as (hi / mi + 1) / 2 * 255. The calculations in the W and Z directions are the same as that in the H direction. Denote the three calculated adaptive grayscale maps as HO, WO, and ZO respectively.

[0033] Step S35: Concatenate HO, WO, and ZO into an RGB three-channel image. WO corresponds to the R channel, HO corresponds to the G channel, and ZO corresponds to the B channel.

[0034] Optionally, calculate the LBP-TOP features of the preprocessed depth image and the preprocessed infrared image respectively, as follows:

[0035] Step S41: Obtain three frames of images dequeued from the queue DQ 3 times, and set the length of DQ to 0.

[0036] Step S42: Obtain three frames of images dequeued from the queue IRQ 3 times, and set the length of IRQ to 0.

[0037] Step S43: Calculate the LBP-TOP features based on the images obtained in Step S41.

[0038] Step S44: Calculate the LBP-TOP features based on the images obtained in Step S42.

[0039] Step S45: Concatenate the LBP-TOP features obtained in Step S43 and Step S44 and send them to TMODEL to calculate the live score.

[0040] As can be seen from the above technical solutions, compared with the prior art, the present invention discloses a face liveness detection method based on TOF, which can balance real-time performance and stable reliability on end-side devices, as follows:

[0041] 1. Compared with the liveness discrimination methods based on monocular and binocular, the present invention has a small computational complexity and is minimally affected by the environment.

[0042] 2. Compared with the methods based on multi-frame sequence modeling, the present invention only needs four votes to obtain the result for one liveness judgment, and can run in real time on end-side devices and devices with limited computing power, and has good stability.

[0043] 3. Compared with the live body discrimination method based on structured light, the method of the present invention based on TOF has better stability in anti-interference with the environment. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the drawings in the following description are only the embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained according to the provided drawings without creative work.

[0045] Figure 1 It is a schematic flow chart of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0046] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative work belong to the scope of protection of the present invention.

[0047] The embodiments of the present invention disclose a face live body detection method based on TOF, as Figure 1 shown, the specific steps are as follows:

[0048] Step S1: Obtain a depth image and a near-infrared IR image from the TOF device module;

[0049] Step S2: Perform bilateral filtering on the depth image and dynamic range compression on the IR image to obtain the filtered depth image (denoted as DB) and the IR image dynamically compressed to 8 bits (denoted as IR8);

[0050] Step S3: Perform dynamic range compression on the depth image and median filtering on the IR image; obtain the depth image compressed to 8 bits (DB8) and the filtered IR image (denoted as IRM);

[0051] Step S4: Calculate the surface normal vector RGB image of DB8, denoted as DRGB;

[0052] Step S5: Send DRGB into the depth live body judgment model DMODE for a live body score judgment;

[0053] Step S6: According to the result obtained in S5, if DMODEL determines it as a non-living body, directly use the DMODEL output as the previous frame result score, store DB8 and IRM into their respective image buffer queues, and the two image buffer queues are respectively denoted as queue DQ and queue IRQ. After the judgment of the current frame is completed, enter the next frame, and repeat steps S1 to S5; if it is determined as a living body, splice DB8 and IRM and send them into the GMODEL model for living body judgment. Multiply the GMODEL living body score and the DMODEL living body score proportionally and then add them to obtain the final score of the current frame. The proportionality coefficient is [0.2, 0.8]. At the same time, store DB8 and IRM into queue DQ and queue IRQ respectively;

[0054] Step S7: Determine whether the number of queues in queue DQ and queue IRQ is greater than 3. If it is less than 3, repeat steps S1 to step S6; if it is greater than or equal to 3, perform 3 dequeue operations on DQ and IRQ respectively. Reset the lengths of IRQ and DQ to zero and obtain the respective 3 dequeued images. Calculate the LBP-TOP features for the images obtained by dequeuing IRQ and DQ 3 times respectively, and splice the calculated LBP-TOP features of each into TMODEL for a living body judgment score calculation;

[0055] Step S8: Calculate through a multi-frame voting mechanism based on the score obtained in S7 and the scores corresponding to the three dequeued frames respectively. If three out of the four votes are for a living body, it is a living body; otherwise, it is a non-living body.

[0056] In step S2, the bilateral filtering of the depth image and the dynamic range compression of the IR image include the following steps:

[0057] Step S21: Perform bilateral filtering on the depth image;

[0058] Step S22: Calculate the gray histogram of the original IR image;

[0059] Step S23: Calculate the numerical normalization threshold according to the gray histogram calculated in S22;

[0060] Step S24: Perform MinMax normalization calculation on the IR image according to the threshold obtained in S23.

[0061] In step S3, the dynamic range compression of the depth map and the median filtering of the 8-bit IR image, this step includes the following steps:

[0062] Step S31: Calculate the gray histogram of the depth map and calculate the dynamic normalization threshold;

[0063] Step S32: Perform MinMax normalization on the depth map according to the threshold obtained in S31 to obtain the normalized 8-bit depth image;

[0064] Step S33: Perform median filtering on the 8-bit IR image to obtain the filtered IR image;

[0065] In step S4, calculate the surface normal RGB image of the 8-bit depth map. This step includes the following steps:

[0066] Step S41: Perform Pad operations along the H and W directions of the image respectively to obtain an image with a height of H + 2 and a width of W + 2, denoted as HP;

[0067] Step S42: Calculate the gradients in the H and W directions of the image obtained in step S41 respectively to obtain two gradient images with the same size as the image obtained in step S41, denoted as HT and WT respectively. At the same time, construct a gradient map in the Z direction with the same size as HT and WT, and each element value is 1, denoted as ZT;

[0068] Step S43: Calculate the adaptive coefficient Map in the gray space. Map has the same size as the gradient image, and each element on Map is the square root of the sum of the squares of the gradient values at the corresponding positions on HT, WT, and ZT;

[0069] Step S44: Calculate the adaptive gray maps in the H, W, and Z directions respectively. Taking the calculation of the adaptive gray in the H direction as an example, access each element on HT, and denote the element gray value as hi. At the same time, take out the gray value at the corresponding position on Map and denote it as mi; at this time, the gray value at this position on the H adaptive gray map is calculated as (hi / mi + 1) / 2 * 255; similarly, the calculations in the W and Z directions are similar to the H direction, only need to replace hi with the corresponding wi and zi, and denote the three calculated adaptive gray maps as HO, WO, and ZO respectively;

[0070] Step S45: Concatenate HO, WO, and ZO into an RGB three-channel image, where WO corresponds to the R channel, HO corresponds to the G channel, and ZO corresponds to the B channel;

[0071] In step S7, calculate the LBP-TOP features of DB8 and IRM respectively and send them to TMODEL to calculate the scores, including the following steps:

[0072] Step S71: Obtain three frames of images dequeued from the queue DQ three times, and set the length of DQ to 0;

[0073] Step S72: Calculate three frames of images dequeued from the queue IRQ three times, and set the length of IRQ to 0;

[0074] Step S73: Calculate the LBP-TOP features according to the images obtained in step S71;

[0075] Step S74: Calculate the LBP-TOP feature based on the image obtained in Step S72;

[0076] Step S75: Concatenate the LBP-TOP features calculated in Step S73 and Step S74 and send them to the TMODEL.

[0077] The above description of the disclosed embodiments enables those skilled in the art to implement or use the present invention. Various modifications to these embodiments will be apparent to those skilled in the art, and the general principles defined herein can be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention will not be limited to the embodiments shown herein, but rather to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A TOF-based face liveness detection method, characterized in that Including the following steps: Obtain a depth image and a near-infrared image, denoted as Depth and IR respectively; Preprocess the depth image and the near-infrared image; Calculate the surface normal vector RGB image of the preprocessed depth image, denoted as DRGB; the specific steps are as follows: Perform Pad operations along the H direction and W direction of the image respectively to obtain an image with a height of H + 2 and a width of W + 2, denoted as HP; Calculate the gradients in the H direction and W direction of HP respectively to obtain two gradient images with the same size as the image obtained in step S31, denoted as HT and WT respectively. At the same time, construct a gradient map in the Z direction with the same size as HT and WT, and each element value is 1, denoted as ZT; Calculate the adaptive coefficient Map in the gray space. Map has the same size as the gradient image, and each element on Map is the square root of the sum of the gradient values at the corresponding positions on HT, the gradient values at the corresponding positions on WT, and the square of the gradient at the corresponding position on ZT; Calculate the adaptive gray maps in the H, W, and Z directions respectively. Taking the calculation of the adaptive gray map in the H direction as an example, access each element on HT, and denote the gray value of the element as hi. At the same time, take out the gray value at the corresponding position on Map and denote it as mi; at this time, the gray value at this position on the H adaptive gray map is calculated as (hi / mi + 1) / 2 * 255; the calculations in the W direction and Z direction are the same as in the H direction; denote the three calculated adaptive gray maps as HO, WO, and ZO respectively; Stitch HO, WO, and ZO into an RGB three-channel image, where WO corresponds to the R channel, HO corresponds to the G channel, and ZO corresponds to the B channel; Send DRGB into the depth liveness judgment model DMODE for a liveness score judgment once. If DMODEL judges as non-living, directly use the DMODEL output as the previous frame result score; if it judges as living, then stitch the preprocessed depth image and the preprocessed near-infrared image and send them into the GMODEL model for liveness judgment, and multiply and add the GMODEL liveness score and the DMODEL liveness score proportionally to obtain the final score of the current frame; Output the current frame score, and add the current frame IR and Depth to IRQ and DQ respectively; Judge whether the lengths of IRQ and DQ are greater than or equal to 3. If it is judged that the lengths of IRQ and DQ are less than 3, then obtain the next frame of IR and Depth and repeat the above operations; If it is judged that the lengths of IRQ and DQ are greater than or equal to 3, then perform 3 dequeue operations on IRQ and DQ respectively to obtain 3 frames of IR images and 3 frames of Depth images, calculate the LBP-TOP features of the preprocessed depth image and the preprocessed infrared image respectively, and send them into the TMODEL model for a liveness judgment score calculation; Comprehensively perform voting calculation on the scores corresponding to the three frames dequeued from IRQ and DQ and the score calculated by TMODEL based on the three frames, a total of four scores, to obtain the final liveness score. If there are three or more results of living in the voting results, it is judged as living, otherwise it is judged as non-living.

2. The method for face liveness detection based on TOF according to claim 1, characterized in that The preprocessing of the depth image and the near-infrared image is specifically as follows: Perform bilateral filtering on the depth image and dynamic range compression on the near-infrared image to obtain a filtered depth image and a near-infrared image with dynamic compression to 8 bits; Perform dynamic range compression on the depth image and median filtering on the near-infrared image to obtain a depth image compressed to 8 bits and a filtered near-infrared image.

3. The method for face liveness detection based on TOF according to claim 2, wherein, The bilateral filtering of the depth image and the dynamic range compression of the near-infrared image are specifically as follows: S11: Perform bilateral filtering on the depth image; S12: Calculate the gray histogram of the original near-infrared image; S13: Calculate the numerical normalization threshold according to the gray histogram obtained in S12; S14: Perform MinMax normalization calculation on the near-infrared image according to the threshold obtained in S13.

4. A TOF-based face liveness detection method according to claim 2, characterized in that, The dynamic range compression of the depth image and the median filtering of the near-infrared image are specifically as follows: S21: Calculate the gray histogram of the depth map and calculate the dynamic normalization threshold; S22: Perform MinMax normalization on the depth map according to the threshold obtained in S21 to obtain a normalized 8-bit depth image; S23: Perform median filtering on the near-infrared image to obtain a filtered near-infrared image.

5. A TOF-based face liveness detection method according to claim 1, characterized in that, Calculate the LBP-TOP features of the preprocessed depth image and the preprocessed infrared image respectively, specifically as follows: Step S41: Obtain three frames of images dequeued from the queue DQ 3 times, and set the length of DQ to 0; Step S42: Obtain three frames of images dequeued from the queue IRQ 3 times, and set the length of IRQ to 0; Step S43: Calculate the LBP-TOP features according to the images obtained in Step S41; Step S44: Calculate the LBP-TOP features according to the images obtained in Step S42; Step S45: Concatenate the LBP-TOP features obtained in Step S43 and Step S44 and send them to TMODEL to calculate the liveness score.

Images