Iris face multimodal living body detection and recognition method, device, medium and equipment

Abstract

The present invention discloses an iris face multimodal liveness detection and recognition method, device, medium and equipment, belonging to the field of biometrics. The present invention obtains multiple face images and multiple iris images collected by a face lens and an iris lens at multiple collection positions; and when it is determined that the multiple face images are all of the same face, the multiple iris images are spliced ​​together to obtain a spliced ​​image; at least one of the multiple face images, at least one of the multiple iris images and the spliced ​​image are used to perform multi-step liveness detection respectively, and if the multi-step liveness detection is passed, the iris face multimodal liveness detection is passed. The present invention improves the accuracy and efficiency of liveness detection, and improves the accuracy of recognition.

Description

Technical Field

[0001] The present invention relates to the field of biometrics, and in particular to an iris and face multimodal living body detection and recognition method, device, computer-readable storage medium and equipment. Background Art

[0002] Biometric technology is a close combination of computers with high-tech means such as optics, acoustics, biosensors and biostatistics principles, which uses the inherent physiological characteristics of the human body (such as fingerprints, faces, irises, etc.) and behavioral characteristics (such as handwriting, voice, gait, etc.) to identify personal identity.

[0003] Face recognition technology has become the most mainstream biometric technology at present. Its ease of use and richness of data samples are unmatched by other biometric technologies. However, it is slightly lacking in security, for example, face recognition cannot distinguish twins. As an important identity recognition feature, the iris has the advantages of lifelong uniqueness, stability, collectability and non-invasiveness, and is an inevitable trend in the development of identity recognition research and application. However, iris recognition is less easy to use than face recognition, and the amount of data samples is not yet rich. Therefore, combining iris recognition and face recognition for multimodal biometrics can fully combine and utilize the advantages of face recognition and iris recognition.

[0004] Whether it is face recognition, iris recognition or iris-face multimodal recognition, there is a risk of prosthesis attack. Therefore, liveness detection is an important part of the biometric system, which can avoid prosthesis attacks and improve the security of the biometric system.

[0005] Most existing face recognition solutions currently use near-infrared face images (grayscale images) for liveness detection, while the images used for face recognition are color face images. To achieve liveness detection while performing face recognition, it is necessary to add a near-infrared face lens module to the original color face lens module. If this method is applied to the iris face multimodal recognition system, since a near-infrared iris lens module is also required for iris recognition, there will be three lens modules.

[0006] This solution of adding a near-infrared face lens module not only increases the complexity and cost of the hardware system, but also increases the size of the device. In addition, since the video streams collected by the three lens modules in the iris face multimodal recognition system need to be transmitted and processed in real time at the same time, the transmission rate of the line and the real-time processing capability of the CPU are required to be high. In addition, the use of near-infrared faces for liveness detection is limited by the image clarity. The face lens modules currently in use are generally around 2 million pixels, and the details of the near-infrared face images they capture are not very clear. Some highly simulated prosthetic materials can still deceive the algorithm that uses near-infrared face images for liveness detection. If the details of the near-infrared face images are clearer, these problems can theoretically be solved.

[0007] However, the pixel of the lens that requires clear details and full-frame facial information is too high, generally requiring more than 10 million pixels. The video stream shot by this lens is difficult to transmit in real time, and it is also difficult to be processed in real time by the CPU of an ordinary computer. A server with very high performance is required to be competent for this real-time transmission and processing. This limits the application of this type of biometric equipment. In most application scenarios, such as PCs, various mobile terminals, subway station gates, access control systems, bank security systems, etc., it is simply not applicable. Summary of the invention

[0008] In order to solve the problems in the prior art of low accuracy of liveness detection of images taken by ordinary face lenses and poor real-time transmission and processing of images taken by high-precision lenses, the present invention provides an iris face multimodal liveness detection and recognition method, device, medium and equipment, which improves the accuracy and efficiency of liveness detection.

[0009] The present invention provides the following technical solutions:

[0010] In a first aspect, the present invention provides an iris face multimodal liveness detection method, the method comprising:

[0011] Acquire multiple face images and multiple iris images captured by a face lens and an iris lens at multiple capture positions; and when it is determined that the multiple face images are all of the same face, stitch the multiple iris images together to obtain a stitched image; perform multi-step liveness detection using at least one of the multiple face images, at least one of the multiple iris images, and the stitched image, respectively. If the multi-step liveness detection is passed, the iris face multimodal liveness detection is passed.

[0012] In a second aspect, the present invention provides an iris human face multimodal liveness detection device corresponding to the iris human face multimodal liveness detection method described in the first aspect, the device comprising:

[0013] A liveness detection module is used to obtain multiple face images and multiple iris images collected by a face lens and an iris lens at multiple collection positions; and when it is determined that the multiple face images are all of the same face, the multiple iris images are spliced ​​together to obtain a spliced ​​image; at least one of the multiple face images, at least one of the multiple iris images, and the spliced ​​image are used to perform multi-step liveness detection respectively, and if the multi-step liveness detection is passed, the iris face multimodal liveness detection is passed.

[0014] In a third aspect, the present invention provides a computer-readable storage medium for iris and face multimodal liveness detection, comprising a memory for storing processor-executable instructions, wherein the instructions, when executed by the processor, implement the steps of the iris and face multimodal liveness detection method described in the first aspect.

[0015] In a fourth aspect, the present invention provides a device for iris and face multimodal liveness detection, comprising at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the iris and face multimodal liveness detection method described in the first aspect are implemented.

[0016] In a fifth aspect, the present invention provides an iris face multimodal recognition method, the method comprising:

[0017] Performing iris and face multimodal liveness detection using the iris and face multimodal liveness detection method described in the first aspect;

[0018] When the iris-face multimodal liveness detection passes, multimodal recognition is performed using the first face image, at least one of the second face image and the spliced ​​image, and the first iris image.

[0019] In a sixth aspect, the present invention provides an iris face multimodal recognition device corresponding to the iris face multimodal recognition method according to the fifth aspect, the device comprising:

[0020] A multimodal liveness detection module, used to perform iris and face multimodal liveness detection through the iris and face multimodal liveness detection device according to the second aspect;

[0021] The multimodal recognition module is used to perform multimodal recognition using the first face image, the second face image and at least one of the spliced ​​image and the first iris image when the iris face multimodal liveness detection passes.

[0022] In a seventh aspect, the present invention provides a computer-readable storage medium for iris and face multimodal recognition, comprising a memory for storing processor-executable instructions, wherein the instructions, when executed by the processor, implement the steps of the iris and face multimodal recognition method described in the fifth aspect.

[0023] In an eighth aspect, the present invention provides a device for iris and face multimodal recognition, comprising at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the iris and face multimodal recognition method described in the fifth aspect are implemented.

[0024] The present invention has the following beneficial effects:

[0025] The present invention uses a method of splicing multiple iris images at different positions to obtain complete high-definition near-infrared face information, improve the quality of near-infrared face images (image quality and detail information at the iris level), and thus achieve high-precision liveness detection. In addition, when splicing, the face images taken at different positions are used to ensure whether the multiple iris images are from the same user, thereby preventing the error of multiple iris images not being from the same user when splicing.

[0026] In addition, the present invention can realize multi-step liveness detection by splicing, using ordinary pixel face lens and iris lens, without the need for ultra-high pixel lens, and integrates multiple methods of color face liveness detection, near-infrared partial face area liveness detection, and high-definition complete near-infrared face liveness detection without increasing additional hardware costs. Not only is the hardware cost low and the volume small, but the real-time transmission and processing of data is also guaranteed. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 This is a flowchart of an example of the iris and face multimodal living body detection method of the present invention;

[0028] Figure 2 This is a flowchart of another example of the iris and face multimodal living body detection method of the present invention;

[0029] Figure 3 for Figure 2 The flowchart of Example 1 of step S150 in the example of the iris and face multimodal living body detection method shown;

[0030] Figure 4 is a schematic diagram of the first iris image in Example 1;

[0031] Figure 5 is a schematic diagram of the second iris image in Example 1;

[0032] Figure 6 for Figure 4 The first iris image shown and Figure 5 A schematic diagram of a stitched image obtained after stitching the second iris image shown;

[0033] Figure 7 for Figure 2 The flowchart of Example 2 of step S150 in the example of the iris and face multimodal living body detection method shown;

[0034] Figure 8 is a schematic diagram of the first iris image in Example 2;

[0035] Fig. 9 is a schematic diagram of the second iris image in Example 2;

[0036] Fig.10 is a schematic diagram of the first face image or the second face image in Example 2;

[0037] Fig.11 for Figure 8 The first iris image shown and Fig. 9 A schematic diagram of a stitched image obtained after stitching the second iris image shown;

[0038] Fig.12 for Figure 2 The flowchart of step S110 in the example of the iris and face multimodal living body detection method shown;

[0039] Fig.13 A schematic diagram of an example of the iris and face multimodal living body detection device of the present invention;

[0040] Fig.14 is a schematic diagram of another example of the iris and face multimodal living body detection device of the present invention;

[0041] Fig.15 for Fig.14 An example diagram of a splicing unit 150 in an example of an iris and face multimodal living body detection device is shown;

[0042] Fig.16 for Fig.14 Another example diagram of the splicing unit 150 in the example of the iris and face multimodal living body detection device shown;

[0043] Fig.17 for Fig.14 An example diagram of the first acquisition unit 110 in the example of the iris and face multimodal living body detection device shown;

[0044] Fig.18 This is a flowchart of an example of the iris face multimodal recognition method of the present invention;

[0045] Fig.19 The figure is a schematic diagram of an example of the iris and face multimodal recognition device of the present invention. DETAILED DESCRIPTION

[0046] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0047] Embodiment 1:

[0048] The embodiment of the present invention provides a multi-modal liveness detection method for iris and face, such as Figure 1 As shown, the method includes:

[0049] S100: Acquire multiple face images and multiple iris images captured by a face lens and an iris lens at multiple capture positions; and when it is determined that the multiple face images are all of the same face, stitch the multiple iris images together to obtain a stitched image; perform multi-step liveness detection using at least one of the multiple face images, at least one of the multiple iris images, and the stitched image, respectively. If the multi-step liveness detection is passed, the iris face multimodal liveness detection is passed.

[0050] In the present invention, at the same acquisition position, the face lens and the iris lens simultaneously acquire the face image and the iris image, and the multiple iris images acquired at multiple acquisition positions need to ensure that the image obtained after all iris images are spliced ​​includes a complete face.

[0051] Furthermore, the present invention does not limit the specific acquisition method of multiple acquisition positions. Multiple acquisitions can be performed by rotating the rotatable face lens and iris lens to different positions. This method only requires one face lens and one iris lens. It can also be performed by using a fixed face lens and an iris lens. This method includes multiple iris lenses at different angles. Multiple iris images at different positions are collected simultaneously by using multiple iris lenses at different angles. There can be only one face lens because the face lens has a large field of view and can capture the entire face area at one time without the need for multiple lenses to cooperate.

[0052] When the present invention determines that multiple face images are all of the same face, multiple iris images are spliced ​​together to obtain a spliced ​​image, thereby ensuring that the multiple iris images are from the same face, thereby ensuring the correctness of the spliced ​​face image. The spliced ​​image contains a complete near-infrared face, and the details such as texture are very clear, so that very accurate near-infrared face liveness detection can be performed.

[0053] Then, at least one of the multiple face images, at least one of the multiple iris images, and the spliced ​​image are used to perform multi-step liveness detection. The present invention does not limit the order of the multi-step liveness detection, and the order of simple first and complex later can be followed, that is, the simpler ones in the multi-step liveness detection are put in front and executed first; or the order of first obtained first performed can be followed, that is, the face / iris image obtained first is first subjected to liveness detection.

[0054] The present invention has the following beneficial effects:

[0055] The present invention uses a method of splicing multiple iris images at different positions to obtain complete high-definition near-infrared face information, improve the quality of near-infrared face images (image quality and detail information at the iris level), and thus achieve high-precision liveness detection. In addition, when splicing, the face images taken at different positions are used to ensure whether the multiple iris images are from the same user, thereby preventing the error of multiple iris images not being from the same user when splicing.

[0056] In addition, the present invention can realize multi-step liveness detection by splicing, using ordinary pixel face lens and iris lens, without the need for ultra-high pixel lens, and integrates multiple methods of color face liveness detection, near-infrared partial face area liveness detection, and high-definition complete near-infrared face liveness detection without increasing additional hardware costs. Not only is the hardware cost low and the volume small, but the real-time transmission and processing of data is also guaranteed.

[0057] As a more specific implementation of the present invention, Figure 2 As shown, the aforementioned S100 includes:

[0058] S110: Acquire a first face image and a first iris image simultaneously captured at a target capture position by a face lens and an iris lens simultaneously mounted on a rotating pan-tilt platform.

[0059] The face lens and iris lens of the present invention are simultaneously mounted on a rotating platform, which can rotate up and down and / or left and right, wherein the field of view of the face lens is relatively large, and the field of view of the iris lens is relatively small.

[0060] The present invention does not limit the specific structure of the rotating pan-tilt platform. For example, it can be an electric two-dimensional pan-tilt platform controlled by a stepper motor (which can rotate up and down and left and right), or it can be a simple shaft (which can only rotate up and down or left and right) controlled by a stepper motor.

[0061] The face lens and iris lens of the present invention can be turned on at the same time to collect face images and iris images in real time. It is also possible to decide to turn on a certain lens after a condition is met.

[0062] The target acquisition position is determined in advance, and a complete and clear iris image with the human eye in the middle of the picture can be acquired at the target acquisition position. Although the target acquisition position is determined in advance, the target acquisition position is not a fixed position corresponding to the actual spatial position. Because in the iris image acquired at the target position, the human eye is in the middle of the picture, the target acquisition position is determined by the human eye position in the picture, and the position of the human eye in the picture is fixed. If the user's height, distance from the lens, etc. are different, then the actual spatial position corresponding to the target acquisition position will also be different.

[0063] When the pan / tilt of the present invention rotates to a target acquisition position, a first face image and a first iris image are simultaneously acquired at the target acquisition position, thereby ensuring that the simultaneously acquired first face image and first iris image are from the same user.

[0064] S120: Performing a first step of living body detection using the first face image and the first iris image respectively.

[0065] The aforementioned first face image includes the entire face and is a color image, and living body detection can be performed through the image.

[0066] The aforementioned first iris image includes the eyes and the area around the eyes, for example, it can be half of a face (because the first iris image is to capture the eyes, and the eyes are in the upper half of the face, so it is generally the upper half of the face), and it is a near-infrared grayscale image. Because the requirements for the accuracy of iris texture and other aspects during iris recognition are much higher than the requirements for the accuracy of facial texture and other aspects during face recognition, it is necessary to make the number of pixels captured in the iris area larger (about 20 pixels / mm), so the accuracy of the iris lens is required to be much higher than the accuracy of the facial lens. However, the pixels of the iris lens cannot be made very large (otherwise it will not meet the requirements of real-time transmission and processing), so although the iris lens has high accuracy, the field of view is small, and the image size is small, and it can only capture a part of the face including the eyes (although the size is small, it has high accuracy and rich details such as texture), but cannot capture the entire face.

[0067] Therefore, liveness detection can be performed through the first iris image. Since the first iris image is taken by an iris lens with higher accuracy than a face lens, many details that cannot be captured by traditional face infrared cameras can be captured in the near-infrared first iris image. Therefore, the accuracy of liveness detection using the first iris image will be much higher than that of traditional near-infrared face liveness detection methods.

[0068] The liveness detection through the first face image and the liveness detection through the first iris image are the first liveness detection. When both liveness detections pass, the first liveness detection passes.

[0069] The present invention does not limit the method used for liveness detection through the first face image, for example, a traditional deep learning-based color face silent liveness detection method can be used. The present invention also does not limit the method used for liveness detection through the first iris image, for example, a deep learning-based near-infrared iris liveness detection method can be used.

[0070] There is no restriction on the execution order of performing liveness detection through the first face image and performing liveness detection through the first iris image, and they can be executed simultaneously or one after another.

[0071] S130: extracting an iris region from the first iris image, and using the iris region to perform a second step of body detection.

[0072] In the aforementioned S120, liveness detection is performed using the first iris image through the entire first iris image, and mainly uses detail information such as texture of the iris surrounding area for liveness detection. Since the first iris image not only captures detail information such as texture of the iris surrounding area, but also captures clear information within the iris area, the information within the iris area can also be used to perform rapid iris liveness detection, i.e., the second step liveness detection. Before performing the second step liveness detection, the iris area needs to be extracted first.

[0073] The present invention does not limit the method of the second step of liveness detection, and an example is given below:

[0074] 1. It can be done by observing pupil tremor. The observation method is to use the rhythm of the white light flashing emitted by the biometric device to synchronously observe the dilation and contraction of the pupil. If the changing rhythm is consistent with the rhythm of the white light flashing or the difference is within a certain range, it means that the iris is alive.

[0075] 2. You can also use deep learning-based iris area silent liveness detection to determine the liveness status of the iris area.

[0076] S140: Acquire at least one second face image and at least one second iris image simultaneously captured by the face lens and the iris lens when the pan-tilt head rotates to other capture positions other than the target capture position, and the other capture positions ensure that all the captured second iris images can form a complete face after being spliced ​​with the first iris image.

[0077] In the aforementioned first step of liveness detection, liveness detection is performed through the first face image. The texture details and other features of the color first face image used are not rich, and the accuracy of liveness detection is not very high. The accuracy of liveness detection performed through the first iris image in the first step of liveness detection and the second step of liveness detection is higher, but the image used is the image of the eye and its surroundings, and does not include other parts of the whole face image. If other parts of the face image (such as the lower half of the face) are fake, it still cannot be detected. The solution is to perform liveness detection on the whole face that is rich in details and includes other parts.

[0078] As known from the background technology, it is unrealistic to directly capture a whole near-infrared face image with rich texture details through a lens. Therefore, the present invention seeks a method for obtaining a whole near-infrared face image with rich texture details under the hardware conditions of existing lenses.

[0079] The present invention rotates the pan / tilt to other collection positions outside the target collection position, so that the iris lens collects at least one near-infrared iris image at the other collection position, and it is necessary to ensure that the at least one near-infrared iris image (second iris image) collected at the other collection position can form a complete near-infrared face after being spliced ​​with the first iris image. When collecting the second iris image, the face lens simultaneously captures the second face image, and the second iris image and the second face image captured at the same time can be guaranteed to be from the same user.

[0080] Generally, the number of the second iris image is one, for example, the first iris image is the upper half of the face, and the second iris image is the lower half of the face, and the entire face can be obtained by splicing the upper half of the face and the lower half of the face.

[0081] In addition, the present invention can execute the first step of liveness detection and the second step of liveness detection in S120 and S130 during the rotation of the rotating platform. In this way, the time gap of the rotating platform can be fully utilized, and the CPU and other processors perform the first step of liveness detection and the second step of liveness detection in the time gap, which fully utilizes the time of the entire liveness detection process and improves the processing efficiency of the CPU. Of course, the rotating platform can also be rotated after completing S120 and S130, and the present invention does not limit this.

[0082] S150: When it is determined that the first face image and each second face image are of the same face, all the second iris images are spliced ​​together with the first iris image to obtain a spliced ​​image containing a complete near-infrared face.

[0083] This step is used to stitch the second iris image with the first iris image to obtain a complete near-infrared face. Since when the pan / tilt rotates to other acquisition positions, it cannot be guaranteed that the user in front of the camera is always the same person (i.e., a person may be replaced to create a fake image, etc.), it cannot be guaranteed that the second iris image and the first iris image are of the same person. If the second iris image and the first iris image are directly stitched together, errors may occur. Also, because the second iris image and the first iris image are both different parts of the face (even if there are overlapping similar areas, the areas are not large), it is not possible to simply determine whether the second iris image and the first iris image are the same face based on the second iris image and the first iris image themselves.

[0084] To solve the above problem, the present invention determines whether the first face image and the second face image are from the same person (i.e., whether they are the same face). Since the field of view of the face lens is large, the first face image and the second face image both include complete faces, so it can be determined whether the first face image and the second face image are the same face.

[0085] The present invention does not limit the method for determining whether the first face image and the second face image are the same face. In one example, the method includes: comparing the first face image and the second face image by a face comparison method to obtain a comparison score; if the comparison score is greater than a set judgment threshold, they are the same face. Since the first face image and the second face image are taken at a short interval and almost at the same time, a very high judgment threshold can be used.

[0086] Since the first face image and the first iris image are taken at the same time, it can be guaranteed that the first face image and the first iris image are from the same face (user), and the second face image and the second iris image are taken at the same time, it can also be guaranteed that the second face image and the second iris image are from the same face (user). Therefore, after determining that the first face image and the second face image are the same face, it can be guaranteed that the first iris image and the second iris image are from the same face of the same user, and the second iris image can be spliced ​​with the first iris image to obtain a spliced ​​image containing a complete near-infrared face.

[0087] S160: Use the stitched image to perform the third step of living body detection.

[0088] The present invention obtains a stitched image containing a complete near-infrared face through stitching in S150, and this step performs the third step of liveness detection based on the stitched image. Although the stitched image is stitched, its texture and other details are very clear, and very accurate near-infrared face liveness detection can be completed.

[0089] In most cases, the previous steps including S130 are able to meet the requirements of liveness detection in most cases. S140 to S160 are mainly aimed at the situation where the lower half of the face is wearing a prosthetic mask (or respirator, etc.).

[0090] The first step of liveness detection, the second step of liveness detection and the third step of liveness detection are executed in sequence. When executed in sequence, the next step is executed only if the previous step of liveness detection passes. This sequential execution of multi-step liveness detection not only improves the accuracy of liveness detection, but also ensures fast and efficient liveness detection in the order from simple to complex and from partial to complete.

[0091] When executed sequentially, if the first step of liveness detection, the second step of liveness detection and the third step of liveness detection are all passed, the iris face multimodal liveness detection is passed. If any step of liveness detection fails, the iris face multimodal liveness detection fails, a prompt indicating that the liveness detection failed is sent, and the subsequent liveness detection process is no longer executed.

[0092] The present invention first uses the acquired color first face image and near-infrared first iris image to perform liveness detection respectively; then performs liveness detection on the iris area on the first iris image; then splices the acquired second iris image with the first iris image to obtain a complete near-infrared face image, and finally uses the complete near-infrared face image to perform liveness detection. Only when all liveness detections pass, the entire iris face multimodal liveness detection passes.

[0093] The present invention has the following beneficial effects:

[0094] 1. The present invention adopts a multi-step liveness detection method that is executed sequentially from simple to complex and from partial to complete, which not only improves the accuracy of liveness detection, but also ensures that liveness detection is fast and efficient.

[0095] 2. The present invention uses a method of splicing multiple iris images at different positions to obtain complete high-definition near-infrared face information, improve the quality of near-infrared face images (image quality and detail information at the iris level), and thus achieve high-precision liveness detection. In addition, when splicing, the face images taken at different positions are used to ensure whether the multiple iris images are from the same user, thereby preventing the error of multiple iris images not being from the same user when splicing.

[0096] 3. Through the splicing method, the present invention can realize multi-step liveness detection using two lenses, the face lens and the iris lens, which can reduce a near-infrared face lens and does not require a lens with ultra-high pixels. Without increasing the additional hardware cost, it integrates multiple methods of color face liveness detection, near-infrared partial face area liveness detection, iris liveness detection, and high-definition complete near-infrared face liveness detection. Not only is the hardware cost low and the volume small, but the real-time transmission and processing of data is also guaranteed.

[0097] The present invention does not limit the specific method of splicing all the second iris images with the first iris image in S150 to obtain a spliced ​​image containing a complete near-infrared face. Two specific examples are given below.

[0098] Example 1:

[0099] This example is based on overlapping area stitching, which is suitable for the case where there is a large overlapping area between the second iris image and the first iris image. Figure 3 As shown, the method of this example includes:

[0100] S151: extracting feature points of the first iris image and the second iris image, and counting the feature points at the same positions of the first iris image and the second iris image.

[0101] The first iris image and the second iris image both include a portion of a human face, and feature points on a portion of the human face can be extracted by various face positioning and feature point extraction algorithms in the prior art. Since the second iris image and the first iris image have overlapping areas of appropriate size, feature points at the same position also exist in these overlapping areas. For example, if the feature points of the nose are distributed on both the first iris image and the second iris image, or distributed on multiple images on the second iris image, then the feature points of the nose are feature points at the same position.

[0102] S152: performing image registration on the first iris image and the second iris image according to feature points at the same position, and finding an overlapping area between the first iris image and the second iris image.

[0103] The positions of the feature points distributed at the same position on different images are the same on the entire face image. Therefore, the positions of the images can be registered according to the feature points at the same position on different images, that is, image registration. Specifically, image registration can be performed by aligning the feature points at the same position on different images. After image registration, the overlapping part of multiple images is the overlapping area.

[0104] S153: The overlapping areas are fused to obtain a stitched image containing a complete near-infrared face.

[0105] After the images are registered, the overlapping area includes multiple layers of images. In this step, fusing the overlapping area means fusing the multiple layers of images in the overlapping area into one layer of image, so that the multiple images are stitched into one image, thereby obtaining the stitched whole image.

[0106] Figure 4 is an example of a first iris image, which is the upper half of a human face and includes multiple feature points of the upper half of the human face. Figure 5 is an example of the second iris image, which is the lower half of the face, including multiple feature points of the lower half of the face. The first iris image and the second iris image both include feature points of the nose area, which are feature points at the same position, and the overlapping part of the first iris image and the second iris image (the area where the nose is located) is the overlapping area. Figure 4 ,and Figure 5 The stitched image obtained after stitching is as follows Figure 6 As shown, it is a complete face image.

[0107] Example 2:

[0108] The above-mentioned example 1 is applicable to the case where there is a large overlapping area between the second iris image and the first iris image. If the overlapping area is small or almost non-existent, then feature points at the same position cannot be detected in the overlapping area, and stitching cannot be achieved using the method of example 1.

[0109] This example can solve the problem of Example 1. It can be applied to the case where multiple iris images do not have a large overlapping area. Of course, it can also be applied to the case where multiple iris images have a large overlapping area. Figure 7 As shown, the method of this example includes:

[0110] S151 ′: extracting feature points of the first face image and / or the second face image, and extracting feature points of the first iris image and the second iris image.

[0111] The first iris image and the second iris image both include a part of a human face, and the first face image and the second face image both include a complete human face, and feature points are extracted from them respectively through face positioning and feature point extraction algorithms.

[0112] S152 ′: performing image registration on the first iris image and the second iris image according to feature points of the first iris image and the second iris image and feature points at the same position of the first face image and / or the second face image.

[0113] The first face image and the second face image both include a complete face, so their feature points include all feature points of the face. Since the positions of the feature points of the face of the same user are unchanged, the same feature points on the first iris image, the second iris image, the first face image, and the second face image should be in the same position, so the first iris image and the second iris image can be image registered based on the feature points of the first face image and / or the second face image.

[0114] A specific registration method may be: aligning feature points on the first iris image and the second iris image with feature points at the same position on the first face image and / or the second face image to achieve image registration.

[0115] S153': If the first iris image and the second iris image after image registration have an overlapping area, the overlapping area is fused to obtain a spliced ​​image containing a complete near-infrared face.

[0116] Figure 8 is an example of a first iris image, which is the upper half of a human face and includes multiple feature points of the upper half of the human face. Fig. 9 is an example of the second iris image, which is the lower half of a human face, including multiple feature points of the lower half of the human face. The overlapping area of ​​the first iris image and the second iris image is very small, and there are no feature points in the same position. Fig.10 is an example of a first face image or a second face image, which is a complete face image. Fig.10 As a reference, Figure 8 ,and Fig. 9 The stitched image obtained after stitching is as follows Fig.11 As shown, it is a complete face image.

[0117] The above-mentioned Example 1 and Example 2 are examples of the method for splicing the second iris image with the first iris image, and are not limited to the method described in Example 1 and Example 2.

[0118] In the above-mentioned example 1 and example 2, the integrity of the spliced ​​image obtained after splicing can also be verified by using the first face image and / or the second face image as a reference to ensure that the spliced ​​image includes a complete face.

[0119] In the process of collecting the first iris image, it is necessary to quickly and accurately capture the user's iris information, but the iris area of ​​each person is small (about 1 cm in diameter), and the number of pixels captured in the iris area is large (about 20 pixels / mm). Therefore, traditional iris lenses face the problems of small depth of field, small range, and high coordination when collecting irises.

[0120] To solve this problem, Fig.12As shown, S110 of the present invention includes:

[0121] S111: Acquire a front face image captured by a face camera.

[0122] S112: performing eye positioning on the front face image to obtain eye coordinates, calculating a first rotation angle of the rotating platform according to the eye coordinates and eye reference coordinates of the target acquisition position, and causing the rotating platform to rotate according to the first rotation angle.

[0123] Since the field of view of the face lens is relatively large, it is easier to detect human eyes in the face image. Therefore, the present invention first obtains the front face image captured by the face lens, and preliminarily calculates the angle that the rotating pan / tilt head needs to rotate according to the eye coordinates located on the front face image, which is the first rotation angle.

[0124] The target acquisition position is predetermined, and a complete and clear iris image with the human eye in the middle of the image can be acquired at the target acquisition position. The eye reference coordinates of the target acquisition position are the eye coordinates on the face image taken at the target acquisition position, and the coordinates are predetermined based on the target acquisition position.

[0125] As mentioned above, although the target acquisition position is determined in advance, the actual spatial position corresponding to the target acquisition position is not a fixed position. If the user's height, distance from the camera, etc. are different, the actual spatial position corresponding to the target acquisition position will also be different.

[0126] After the eye coordinates are located on the front face image, the pixel difference between the eye coordinates and the preset eye reference coordinates is converted into the angle that the stepper motor needs to rotate, that is, the first rotation angle, and a rotation instruction is sent to the motor, and the motor rotates according to the first rotation angle.

[0127] Specifically, when the rotating gimbal can perform two-dimensional rotation, the lateral rotation angle and the longitudinal rotation angle are calculated based on the pixel difference between the horizontal and vertical coordinates of the human eye position and the reference horizontal and vertical coordinates; when the rotating gimbal can only rotate in one dimension up and down or left and right (because the present invention is mostly for users of different heights, so the one-dimensional rotation is generally up and down), the lateral rotation angle or the longitudinal rotation angle is calculated based on the pixel difference between the horizontal or vertical coordinate of the human eye position and the reference horizontal or vertical coordinate.

[0128] S113: Acquire a front iris image captured by the iris lens when the rotating platform rotates according to the first rotation angle.

[0129] When the rotating platform rotates according to the first rotation angle, the iris lens collects the iris image in real time as the front iris image, and the front iris image is acquired.

[0130] S114: performing iris positioning on the front iris image to obtain iris coordinates, calculating a second rotation angle of the rotating platform according to the iris coordinates and the iris reference coordinates of the target acquisition position, and rotating the rotating platform according to the second rotation angle.

[0131] Although the field of view of the iris lens is relatively small, its resolution is relatively high and the coordinates of the iris are more precise. Therefore, during the rotation of the pan-tilt head, if the iris is found on the collected front iris image, the rotation angle is corrected according to the front iris image to obtain a second rotation angle. The pan-tilt head rotates according to the second rotation angle, which can make the position of the pan-tilt head after rotation more precise and closer to the target collection position.

[0132] The target acquisition position is predetermined, and a complete and clear iris image with the human eye in the middle of the image can be acquired at the target acquisition position. The iris reference coordinates of the target acquisition position are the iris coordinates on the iris image taken at the target acquisition position, and the coordinates are predetermined according to the target acquisition position.

[0133] The actual spatial position (i.e., the target acquisition position) corresponding to the iris reference coordinates and the eye reference coordinates is the same position, but since the iris reference coordinates are set based on the iris image and the eye reference coordinates are set based on the face image, and the size and resolution of the iris image and the face image are different, the specific values ​​of the iris reference coordinates and the eye reference coordinates may be different.

[0134] S115: Acquire a first face image and a first iris image that are respectively captured by the face lens and the iris lens simultaneously when the rotating platform rotates to the target capture position according to the second rotation angle.

[0135] The rotating platform rotates according to the second rotation angle, and is located at the target acquisition position when the rotation is completed. At this time, the face lens and the iris lens simultaneously acquire the first face image and the first iris image.

[0136] The present invention utilizes a face lens and an iris lens that can rotate simultaneously, and adopts a secondary positioning method. The face lens is quickly positioned once, and the iris lens is accurately positioned twice. This can quickly and accurately adapt the height and position of the user, and help users of different heights / distances quickly complete iris information collection without almost any cooperation, allowing the user to accurately complete the collection of face and iris information with almost no perception, and simultaneously realize liveness detection.

[0137] In some embodiments of the present invention, the pixel of the iris lens is generally greater than or equal to 2 million pixels, because iris recognition requires capturing relatively clear iris texture, so iris lenses with less than 2 million pixels generally cannot meet the requirements of iris recognition. In addition, the pixel of the iris lens of the present invention is generally less than or equal to 8 million pixels, because the smaller the pixel of the iris lens, the better the real-time transmission and real-time processing of the image captured by it. The image captured by the iris lens with more than 8 million pixels is difficult to process in real time by a general CPU, and the image captured by the iris lens with more than 8 million pixels can basically cover the entire face, so there is no need for splicing.

[0138] The present invention is described in detail below with a specific example:

[0139] 1. Synchronous acquisition of the first face image and the first iris image

[0140] The face lens is turned on to obtain the front face image, and the face information is located on the front face image to find the eye coordinates; the pixel difference from the target acquisition position is calculated based on the vertical axis data value of the eye coordinates and the eye reference coordinates, and the pixel difference is converted into the first rotation angle of the stepper motor, and a rotation instruction is sent to the motor, and the motor is ready to rotate. During the rotation of the motor, the iris lens collects the front iris image in real time. Once the iris position information data is found on the front iris image, the vertical coordinate pixel number of the iris coordinate is immediately obtained, and the pixel difference between the vertical coordinate of the iris and the vertical coordinate of the iris reference coordinate is calculated twice, and the second rotation angle of the stepper motor is given according to the pixel difference. After rotating to the target acquisition position, the face and iris positions are suitable, and the first face image and the first iris image are synchronously collected.

[0141] 2. The first face image and the first iris image perform the first step of liveness detection

[0142] The first face image (full face full color image information) and the first iris image (upper half near infrared face, including iris and surrounding area information) collected on site are respectively subjected to liveness detection. The judgment method can adopt the traditional color face silent liveness detection method and the near infrared deep learning training method for judging liveness.

[0143] Since the camera that captures the upper half of the near-infrared face is an iris camera, it can capture many details that traditional facial infrared cameras cannot capture. The accuracy of liveness detection using the first iris image will be much higher than traditional facial liveness detection methods.

[0144] 3. The first iris image (iris internal area information) is used for the second step of liveness detection

[0145] Since the half-frame near-infrared face of the first iris image captures clear iris video information, the internal area information of the iris can be used to perform rapid iris liveness detection.

[0146] The iris liveness detection method can be performed by observing pupil tremor. The observation method uses the rhythm of the flashing white light emitted by the device to synchronously observe the dilation and contraction of the pupil. If the changing rhythm is consistent with the rhythm of the flashing white light or the difference is within a certain range, it represents a live iris.

[0147] The liveness status of the iris area can also be determined by using a deep learning-based iris area silent live iris method.

[0148] 4. Stitching of multiple iris images from different acquisition positions

[0149] Since only the upper half of the near-infrared face (iris and its surrounding area information) cannot fully determine whether there is prosthetic information for the lower half of the face, this is also the key problem of liveness detection in iris face multimodal recognition.

[0150] Since the device used in this hairstyle is a self-rotating device that can adapt to height, the face lens and iris lens on the device are not fixed and can be rotated and controlled according to needs.

[0151] Therefore, after completing the first and second steps of liveness detection, it is necessary to rotate the face lens and iris lens to other acquisition positions through a rotating gimbal to capture images and complete image stitching.

[0152] The information captured before the rotation is: the first face image (a) representing the full-frame color face information and the first iris image (b) representing the upper half near-infrared face information (iris and its surrounding area information), and the information captured after the rotation is: the second face image (c) representing the full-frame color face information and the second iris image (d) representing the lower half near-infrared face information (mouth and its surrounding area information).

[0153] First, we need to determine whether a and c are the same face. We can use a face recognition algorithm to make this determination. And because a and c are taken at almost the same time, we can take a very high judgment threshold.

[0154] Then, b and d are spliced ​​together. The splicing methods can be the two methods described in the above-mentioned Example 1 and Example 2.

[0155] 5. Stitched image (full-frame near-infrared face) for the third step of liveness detection

[0156] Through the stitching in step 4, a complete high-definition near-infrared face image can be obtained. At this time, liveness detection is performed based on this near-infrared face image. Although this near-infrared face image is stitched, the details are very clear, and because the color face (first face image, second face image) is used as a reference during the stitching in step 4, there is no problem with integrity, and very accurate near-infrared face liveness detection can be completed.

[0157] Embodiment 2:

[0158] The embodiment of the present invention provides an iris face multimodal living body detection device, such as Fig.13 As shown, the device comprises:

[0159] The liveness detection module 100 is used to obtain multiple face images and multiple iris images collected by the face lens and the iris lens at multiple collection positions; and when it is determined that the multiple face images are all of the same face, the multiple iris images are spliced ​​together to obtain a spliced ​​image; at least one of the multiple face images, at least one of the multiple iris images, and the spliced ​​image are used to perform multi-step liveness detection respectively. If the multi-step liveness detection is passed, the iris face multimodal liveness detection is passed.

[0160] The present invention uses a method of splicing multiple iris images at different positions to obtain complete high-definition near-infrared face information, improve the quality of near-infrared face images (image quality and detail information at the iris level), and thus achieve high-precision liveness detection. In addition, when splicing, the face images taken at different positions are used to ensure whether the multiple iris images are from the same user, thereby preventing the error of multiple iris images not being from the same user when splicing.

[0161] In addition, the present invention can realize multi-step liveness detection by splicing, using ordinary pixel face lens and iris lens, without the need for ultra-high pixel lens, and integrates multiple methods of color face liveness detection, near-infrared partial face area liveness detection, and high-definition complete near-infrared face liveness detection without increasing additional hardware costs. Not only is the hardware cost low and the volume small, but the real-time transmission and processing of data is also guaranteed.

[0162] As a more specific implementation of the present invention, Fig.14 As shown, the aforementioned living body detection module 100 includes:

[0163] The first acquisition unit 110 is used to acquire a first face image and a first iris image which are simultaneously acquired at a target acquisition position by the face lens and the iris lens which are simultaneously installed on the rotating pan-tilt head.

[0164] The first living body detection unit 120 is used to perform a first step of living body detection using the first face image and the first iris image.

[0165] The second living body detection unit 130 is used to extract an iris area from the first iris image and use the iris area to perform a second step of living body detection.

[0166] The second acquisition unit 140 is used to acquire at least one second face image and at least one second iris image simultaneously acquired by the face lens and the iris lens when the rotating platform rotates to other acquisition positions outside the target acquisition position, and the other acquisition positions ensure that all the acquired second iris images can form a complete face after being spliced ​​with the first iris image.

[0167] The stitching unit 150 is used to stitch all the second iris images with the first iris image together to obtain a stitched image containing a complete near-infrared face when it is determined that the first face image and each second face image are of the same face.

[0168] The third living body detection unit 160 is used to perform a third step of living body detection using the stitched image.

[0169] Among them, if the first step of liveness detection, the second step of liveness detection and the third step of liveness detection are all passed, the iris and face multimodal liveness detection is passed.

[0170] like Fig.15 As shown, the aforementioned splicing unit 150 includes:

[0171] The first extraction subunit 151 is used to extract feature points of the first iris image and the second iris image, and count the feature points at the same position of the first iris image and the second iris image.

[0172] The first registration subunit 152 is used to perform image registration on the first iris image and the second iris image according to the feature points at the same position, and find the overlapping area between the first iris image and the second iris image.

[0173] The first fusion subunit 153 is used to fuse the overlapping areas to obtain a spliced ​​image containing a complete near-infrared face.

[0174] Or, if Fig.16 As shown, the splicing unit 150 includes:

[0175] The second extraction subunit 151 ′ is used to extract feature points of the first face image and / or the second face image, and to extract feature points of the first iris image and the second iris image.

[0176] The second registration subunit 152' is used to perform image registration on the first iris image and the second iris image according to feature points of the first iris image and the second iris image and feature points at the same position of the first face image and / or the second face image.

[0177] The second fusion subunit 153' is used for fusing the overlapped area between the first iris image and the second iris image after image registration to obtain a spliced ​​image containing a complete near-infrared face if the overlapped area exists between the first iris image and the second iris image after image registration.

[0178] like Fig.17 As shown, the first acquisition unit 110 includes:

[0179] The first acquisition subunit 111 is used to acquire a front face image captured by a face lens.

[0180] The first calculation subunit 112 is used to locate the eyes of the front face image to obtain eye coordinates, calculate the first rotation angle of the rotating platform according to the eye coordinates and the eye reference coordinates of the target acquisition position, and make the rotating platform rotate according to the first rotation angle.

[0181] The second acquisition subunit 113 is used to acquire a front iris image captured by the iris lens when the rotating platform rotates according to the first rotation angle.

[0182] The second calculation subunit 114 is used to perform iris positioning on the front iris image to obtain iris coordinates, calculate a second rotation angle of the rotating platform according to the iris coordinates and the iris reference coordinates of the target acquisition position, and rotate the rotating platform according to the second rotation angle.

[0183] The third acquisition subunit 115 is used to acquire a first face image and a first iris image which are respectively captured by the face lens and the iris lens simultaneously when the rotating platform rotates to the target acquisition position according to the second rotation angle.

[0184] In the present invention, the pixel of the iris lens is 2 million ≤ 8 million ≤.

[0185] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned method embodiment 1. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment 1. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment 1, and will not be repeated here.

[0186] Embodiment 3:

[0187] The method described in the above embodiment 1 provided by the present invention can realize the business logic through a computer program and record it on a storage medium, and the storage medium can be read and executed by a computer to achieve the effect of the solution described in embodiment 1 of this specification. Therefore, the present invention also provides a computer-readable storage medium for iris and face multimodal liveness detection, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the iris and face multimodal liveness detection method including embodiment 1 are implemented.

[0188] The storage medium may include a physical device for storing information, which is usually a medium that digitizes the information and then stores it in an electrical, magnetic or optical manner. The storage medium may include: a device that stores information in an electrical energy manner, such as various memories, such as RAM, ROM, etc.; a device that stores information in a magnetic energy manner, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information in an optical manner, such as a CD or a DVD. Of course, there are other readable storage media, such as quantum memory, graphene memory, etc.

[0189] The storage medium described above may also include other implementation methods according to the description of method embodiment 1. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment 1. For details, please refer to the description of the relevant method embodiment 1, and no further details will be given here.

[0190] Embodiment 4:

[0191] The present invention also provides a device for iris face multimodal liveness detection, which may be a separate computer, or may include an actual operating device using one or more of the methods or one or more of the embodiments of this specification. The device for iris face multimodal liveness detection may include at least one processor and a memory storing computer executable instructions, and when the processor executes the instructions, the steps of the iris face multimodal liveness detection method described in any one or more of Embodiment 1 are implemented.

[0192] The above-mentioned equipment may also include other implementation methods according to the description of the method or device embodiment. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment 1. For details, please refer to the description of the relevant method embodiment 1, and no further description will be given here.

[0193] Embodiment 5:

[0194] The embodiment of the present invention provides a method for multimodal iris face recognition, such as Fig.18 As shown, the method includes:

[0195] S100': performing iris human face multimodal liveness detection using the iris human face multimodal liveness detection method described in Example 1.

[0196] S200 ′: when the iris-face multimodal liveness detection passes, use the first iris image and at least one of the first face image, the second face image and the spliced ​​image to perform multimodal recognition.

[0197] After completing the iris face multimodal liveness detection method shown in Example 1, and when the iris face multimodal liveness detection passes, after determining the valid state of the iris face liveness of the collected user, the color face information (first face image a or second face image c), binocular iris information (first iris image b) and near-infrared face information (stitched image b+d) obtained in Example 1 are subjected to multimodal identity recognition.

[0198] This embodiment 1 not only achieves the beneficial effects described in embodiment 1, but also realizes the comparison of multimodal feature (visible light face, binocular iris, near-infrared face) information. Multimodality makes the recognition and comparison results more accurate, and the spliced ​​image has more detailed features, further improving the recognition accuracy.

[0199] This embodiment 5 includes all the features of the aforementioned embodiment 1. For the sake of brief description, for matters not mentioned in this embodiment 5, reference can be made to the corresponding contents of the aforementioned embodiment 1, and no further details will be given here.

[0200] Embodiment 6:

[0201] The embodiment of the present invention provides an iris face multimodal recognition device, such as Fig.19 As shown, the device comprises:

[0202] The multimodal living body detection module 10 ′ is used to perform iris and face multimodal living body detection through the iris and face multimodal living body detection device described in Example 5.

[0203] The multimodal recognition module 20' is configured to perform multimodal recognition using the first face image, the second face image and the spliced ​​image and the first iris image when the iris face multimodal liveness detection passes.

[0204] The device provided in this embodiment has the same implementation principle and technical effects as those of the aforementioned method embodiment 5. For the sake of brief description, for matters not mentioned in the device embodiment, reference can be made to the corresponding contents in the aforementioned method embodiment 5, which will not be repeated here.

[0205] Embodiment 7:

[0206] The method described in the above-mentioned embodiment 5 provided by the present invention can realize the business logic through a computer program and record it on a storage medium, and the storage medium can be read and executed by a computer to realize the effect of the solution described in embodiment 5 of this specification. Therefore, the present invention also provides a computer-readable storage medium for iris and face multimodal recognition, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the iris and face multimodal recognition method including embodiment 5 are realized.

[0207] The storage medium may include a physical device for storing information, which is usually a medium that digitizes the information and then stores it in an electrical, magnetic or optical manner. The specific form of the storage medium can be found in Example 3.

[0208] The storage medium described above may also include other implementations according to the description of method embodiment 5. The specific implementations and beneficial effects may refer to the description of the relevant method embodiment 5, and will not be described one by one here.

[0209] Embodiment 8:

[0210] The present invention also provides a device for iris face multimodal recognition, which may be a separate computer, or may include an actual operating device using one or more of the methods or one or more of the embodiments of this specification. The iris face multimodal recognition device may include at least one processor and a memory storing computer executable instructions, and when the processor executes the instructions, the steps of the iris face multimodal recognition method described in any one or more of Embodiment 5 are implemented.

[0211] The above-mentioned device may also include other implementation modes according to the description of the method or device embodiment. The specific implementation methods and beneficial effects can be referred to the description of the relevant method embodiment 5, which will not be described one by one here.

[0212] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multimodal liveness detection method for iris and face, characterized in that: The method comprises: Acquire multiple face images and multiple iris images captured by a face lens and an iris lens at multiple acquisition positions; and when it is determined that the multiple face images are all of the same face, stitch the multiple iris images together to obtain a stitched image including complete high-definition near-infrared face information; respectively use at least one of the multiple face images, at least one of the multiple iris images, and the stitched image to perform multi-step liveness detection including color face liveness detection, near-infrared partial face area liveness detection, and high-definition complete near-infrared face liveness detection. If the multiple-step liveness detection is passed, the iris face multimodal liveness detection is passed; The method comprises: obtaining a plurality of face images and a plurality of iris images collected by a face lens and an iris lens at a plurality of collection positions; and splicing the plurality of iris images together to obtain a spliced ​​image including complete high-definition near-infrared face information when it is determined that the plurality of face images are all of the same face; and using at least one of the plurality of face images, at least one of the plurality of iris images, and the spliced ​​image to perform multi-step liveness detection including color face liveness detection, near-infrared partial face area liveness detection, and high-definition complete near-infrared face liveness detection, respectively. If the multi-step liveness detection is passed, then the iris face multimodal liveness detection is passed, including: Acquire a first face image and a first iris image simultaneously captured at a target capture position by the face lens and the iris lens simultaneously mounted on the rotating pan-tilt platform; Performing a first step of living body detection using the first face image and the first iris image; Extracting an iris region from the first iris image, and using the iris region to perform a second step of living body detection; Acquire at least one second face image and at least one second iris image simultaneously captured by the face lens and the iris lens when the pan / tilt head rotates to other capture positions other than the target capture position, and the other capture positions ensure that all the captured second iris images can form a complete face after being spliced ​​with the first iris image; When it is determined that the first face image and each of the second face images are of the same face, stitching all the second iris images with the first iris image to obtain a stitched image containing a complete near-infrared face; Using the stitched image to perform a third step of liveness detection; Among them, if the first step of liveness detection, the second step of liveness detection and the third step of liveness detection are all passed, the iris and face multimodal liveness detection is passed.

2. The iris face multimodal living body detection method according to claim 1, characterized in that: The step of stitching all the second iris images with the first iris image to obtain a stitched image containing a complete near-infrared face includes: Extracting feature points of the first iris image and the second iris image, and counting the feature points at the same positions of the first iris image and the second iris image; Performing image registration on the first iris image and the second iris image according to the feature points at the same position, and finding an overlapping area between the first iris image and the second iris image; The overlapping areas are fused to obtain a spliced ​​image containing a complete near-infrared face.

3. The iris face multimodal living body detection method according to claim 1, characterized in that: The step of stitching all the second iris images with the first iris image to obtain a stitched image containing a complete near-infrared face includes: Extracting feature points of the first face image and / or the second face image, and extracting feature points of the first iris image and the second iris image; Performing image registration on the first iris image and the second iris image according to feature points of the first iris image and the second iris image and feature points at the same position of the first face image and / or the second face image; If the first iris image and the second iris image after image registration have an overlapping area, the overlapping area is fused to obtain a spliced ​​image containing a complete near-infrared face.

4. The iris face multimodal liveness detection method according to any one of claims 1 to 3, characterized in that: The step of acquiring a first face image and a first iris image simultaneously captured at a target capture position by the face lens and the iris lens simultaneously mounted on the rotating pan-tilt head comprises: Acquire a front face image captured by a face camera; Performing eye positioning on the front face image to obtain eye coordinates, calculating a first rotation angle of the rotating platform according to the eye coordinates and eye reference coordinates of a target acquisition position, and causing the rotating platform to rotate according to the first rotation angle; Acquire a front iris image captured by an iris lens when the rotating platform rotates according to the first rotation angle; Performing iris positioning on the front iris image to obtain iris coordinates, calculating a second rotation angle of the rotating platform according to the iris coordinates and the iris reference coordinates of the target acquisition position, and causing the rotating platform to rotate according to the second rotation angle; A first face image and a first iris image are acquired which are respectively captured by the face lens and the iris lens simultaneously when the rotating platform rotates to a target capture position according to the second rotation angle.

5. An iris face multimodal living body detection device, characterized in that: The device comprises: A liveness detection module is used to obtain multiple face images and multiple iris images collected by a face lens and an iris lens at multiple collection positions; and when it is determined that the multiple face images are all of the same face, the multiple iris images are spliced ​​together to obtain a spliced ​​image including complete high-definition near-infrared face information; at least one of the multiple face images, at least one of the multiple iris images, and the spliced ​​image are used to perform multi-step liveness detection including color face liveness detection, near-infrared partial face area liveness detection, and high-definition complete near-infrared face liveness detection. If the multiple-step liveness detection is passed, the iris face multimodal liveness detection is passed; The living body detection module comprises: A first acquisition unit, used for acquiring a first face image and a first iris image simultaneously acquired at a target acquisition position by the face lens and the iris lens simultaneously mounted on the rotating pan-tilt head; A first living body detection unit, configured to perform a first step of living body detection using the first face image and the first iris image; A second living body detection unit, configured to extract an iris area from the first iris image, and use the iris area to perform a second step of living body detection; a second acquisition unit, configured to acquire at least one second face image and at least one second iris image simultaneously acquired by the face lens and the iris lens when the pan / tilt head rotates to other acquisition positions other than the target acquisition position, wherein the other acquisition positions ensure that all the acquired second iris images can form a complete face after being spliced ​​with the first iris image; a splicing unit, configured to splice all the second iris images with the first iris image to obtain a spliced ​​image containing a complete near-infrared face when it is determined that the first face image and each second face image are of the same face; A third living body detection unit, used for performing a third step of living body detection using the stitched image; Among them, if the first step of liveness detection, the second step of liveness detection and the third step of liveness detection are all passed, the iris and face multimodal liveness detection is passed.

6. A computer-readable storage medium for iris and face multimodal liveness detection, characterized in that: It comprises a memory for storing processor executable instructions, and when the instructions are executed by the processor, the steps of the iris and face multimodal living body detection method according to any one of claims 1 to 4 are implemented.

7. A device for iris and face multimodal liveness detection, characterized in that: The method comprises at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the iris and face multimodal living body detection method according to any one of claims 1 to 4 are implemented.

8. A multimodal iris face recognition method, characterized in that: The method comprises: Performing iris and face multimodal liveness detection by using the iris and face multimodal liveness detection method described in any one of claims 1 to 4; When the iris-face multimodal liveness detection passes, multimodal recognition is performed using the first face image, at least one of the second face image and the spliced ​​image, and the first iris image.

9. An iris face multimodal recognition device, characterized in that: The device comprises: A multimodal liveness detection module, used to perform iris and face multimodal liveness detection through the iris and face multimodal liveness detection device according to claim 5; The multimodal recognition module is used to perform multimodal recognition using the first face image, the second face image and at least one of the spliced ​​image and the first iris image when the iris face multimodal liveness detection passes.

Images