Multimodal biometric acquisition method, device, readable storage medium and equipment

Abstract

The present invention discloses a multimodal biometric feature acquisition method, device, readable storage medium and equipment, belonging to the field of biometric identification. The present invention acquires a visible light 2D face image through a face lens, and rotates the carrier to a first acquisition position to mutually light up a fill light and a structured light to respectively acquire an iris image and a local face 3D structured light image; then rotates the carrier to at least one second acquisition position to mutually light up a fill light and a structured light to respectively acquire a local near-infrared face image and a local face 3D structured light image; finally, the iris image and all local near-infrared face images are spliced ​​to obtain a 2D near-infrared face image, and all local face 3D structured light images are spliced ​​to obtain a 3D face image. The present invention reuses an iris lens to acquire an iris image, a 2D near-infrared face image and a 3D face image, thereby reducing the complexity and cost of the equipment, reducing the size of the equipment, and improving the security and reliability of biometric feature identification.

Description

Technical Field

[0001] The present invention relates to the field of biometrics, and in particular to a multimodal biometric feature acquisition method, device, readable storage medium and equipment. Background Art

[0002] 2D face recognition technology is the most mainstream biometric technology at present. Its ease of use and richness of data samples are unmatched by other biometric technologies. However, it is slightly lacking in security, such as being unable to distinguish between twins and being vulnerable to prosthetic attacks. With the further development and research of face recognition technology by face recognition companies, 3D face recognition technology has emerged, and it has overcome the shortcomings of 2D face recognition technology, such as high environmental requirements and easy to be hacked, and can recognize more quickly.

[0003] At the same time, iris recognition is recognized as the safest and most accurate identity recognition method in biometrics due to the high uniqueness, stability and unchangeable characteristics of the iris, and has been widely used in many fields.

[0004] Therefore, combining 2D face recognition technology, 3D face recognition technology and iris recognition technology for multimodal biometric recognition can make good use of their respective advantages and significantly improve the recognition rate.

[0005] To perform the above-mentioned multimodal biometric recognition, it is first necessary to obtain 2D face images, 3D face images and iris images that meet the requirements. However, in the prior art, it is very difficult to simultaneously obtain 2D face images, 3D face images and iris images through one device.

[0006] The existing technology collects 3D face images through the structured light method, projects active structured light structure information such as laser stripes, Gray code, sinusoidal stripes, etc. onto the surface of the object to be measured through a projector, and then photographs the surface of the object to be measured through a lens to obtain a structured light image, and obtains 3D face information based on the structured light image.

[0007] The acquisition range (field of view) of the existing face lens is large, and the entire face image can be acquired at one time. However, the resolution of the face lens is low. If the structured light image is captured through the face lens, the resolution of the obtained structured light image is low, and it is difficult to meet the high-precision recognition requirements. Therefore, the existing face lens cannot be used to capture 3D face structured light images. The resolution of the iris lens is high, but its field of view is small (about 30°), and it can only capture images of the eye area. Therefore, the existing iris lens cannot be used to capture 3D face structured light images.

[0008] It can be seen that the existing technology cannot use the face lens and iris lens to collect 2D face images, 3D face images and iris images on the same device. If an additional lens is specially equipped for collecting 3D face structured light images, the device structure will be complicated and costly. The lens resolution must be very high and the field of view must be large. The cost of such a lens is very high, and the resolution and size of the image collected by such a lens are very large, which makes it difficult to achieve real-time processing with the processing performance of existing equipment. Summary of the invention

[0009] In order to solve the defect in the prior art that 3D face images cannot be collected through the existing face lens and iris lens, the present invention provides a multimodal biometric feature collection method, device, readable storage medium and equipment, which can simultaneously collect visible light 2D face images, iris images, 2D near-infrared face images and 3D face images by using only two lenses, namely, face lens and iris lens, on one device, thereby reducing the complexity and cost of the device, reducing the size of the device, and improving the security and reliability of biometric feature recognition.

[0010] The present invention provides the following technical solutions:

[0011] In a first aspect, the present invention provides a multimodal biometric feature acquisition method, wherein a face lens and an iris lens are fixedly arranged on a rotatable carrier, and a fill light and a structured light are fixedly arranged on the carrier, and the method comprises:

[0012] Acquire a face image captured by the face lens when the platform is located at an initial position;

[0013] Controlling the stage to rotate to a first acquisition position, wherein the first acquisition position makes the iris centered within the field of view of the iris lens;

[0014] Control the fill light and structured light to be turned on in a time-sharing and mutually exclusive manner, and obtain the iris image and the partial face 3D structured light image collected by the iris lens in a time-sharing manner;

[0015] Control the stage to rotate to at least one second acquisition position, and for each second acquisition position, control the fill light and the structured light to be turned on in a time-sharing and mutually exclusive manner, so as to obtain a local near-infrared face image and a local face 3D structured light image collected in a time-sharing manner by the iris lens;

[0016] The iris image and all the local near-infrared face images are spliced ​​together to obtain a near-infrared face image containing a complete face, and all the local face 3D structured light images are spliced ​​together to obtain a 3D face image containing a complete face.

[0017] In a second aspect, the present invention provides a multimodal biometric feature acquisition device, wherein a face lens and an iris lens are fixedly arranged on a rotatable carrier, a fill light and a structured light are fixedly arranged on the carrier, and the device comprises:

[0018] A first acquisition module, used for acquiring a face image captured by the face lens when the platform is located at an initial position;

[0019] A rotation module, used for controlling the stage to rotate to a first acquisition position, wherein the first acquisition position makes the iris centered within the field of view of the iris lens;

[0020] The second acquisition module is used to control the fill light and the structured light to be turned on in a time-sharing and mutually exclusive manner, and to acquire the iris image and the partial face 3D structured light image collected by the iris lens in a time-sharing manner;

[0021] A third acquisition module is used to control the stage to rotate to at least one second acquisition position, and for each second acquisition position, control the fill light and the structured light to be turned on in a time-sharing and mutually exclusive manner, so as to acquire a local near-infrared face image and a local face 3D structured light image acquired in a time-sharing manner by the iris lens;

[0022] The splicing module is used to splice the iris image and all the local near-infrared face images together to obtain a near-infrared face image containing a complete face, and to splice all the local face 3D structured light images together to obtain a 3D face image containing a complete face.

[0023] In a third aspect, the present invention provides a computer-readable storage medium for multimodal biometric acquisition, comprising a memory for storing processor-executable instructions, wherein the instructions, when executed by the processor, implement the steps of the multimodal biometric acquisition method described in the first aspect.

[0024] In a fourth aspect, the present invention provides a device for multimodal biometric collection, comprising at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the multimodal biometric collection method described in the first aspect are implemented.

[0025] The present invention has the following beneficial effects:

[0026] The present invention reuses an iris lens to collect iris images, 2D near-infrared face images and 3D face images, so that only two lenses, face lens and iris lens, are used on one device to collect visible light 2D face images, iris images, 2D near-infrared face images and 3D face images at the same time, and obtain 2D face information, iris information and 3D face information, thereby reducing the complexity and cost of the device and reducing the size of the device. In addition, the obtained 2D face information, iris information and 3D face information can be used for multi-modal biometric feature recognition and liveness detection, thereby improving the security and reliability of biometric feature recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 A flowchart of an example of the multimodal biometric feature acquisition method of the present invention;

[0028] Figure 2 A schematic diagram of various components in a multimodal acquisition device using the multimodal biometric acquisition method of the present invention;

[0029] Figure 3 This is a schematic diagram of the field of view of the face lens and the iris lens;

[0030] Figure 4 It is a schematic diagram of the field of view angle of the face lens and the iris lens when the stage rotates to the first acquisition position;

[0031] Figure 5 is a schematic diagram of a first current position and a first target position of a human eye;

[0032] Figure 6 for Figure 1 A flowchart of an example of step S200 in the example of the multimodal biometric feature collection method shown;

[0033] Figure 7 for Figure 1 A flowchart of an example of step S500 in the example of the multimodal biometric feature collection method shown;

[0034] Figure 8 for Figure 1 A flowchart of another example of step S500 in the example of the multimodal biometric feature collection method shown;

[0035] Fig. 9 is a schematic diagram of an iris image;

[0036] Fig.10 Schematic diagram of a local near-infrared face image;

[0037] Fig.11 is a schematic diagram of a near-infrared face image;

[0038] Fig.12A schematic diagram of an example of a multimodal biometric feature acquisition device of the present invention;

[0039] Fig.13 for Fig.12 A schematic diagram of an example of a rotation module in the example of a multimodal biometric feature acquisition device shown;

[0040] Fig.14 for Fig.12 A schematic diagram of an example of a splicing module in an example of a multimodal biometric feature acquisition device shown;

[0041] Fig.15 for Fig.12 A schematic diagram of another example of a splicing module in the example of a multimodal biometric feature acquisition device is shown. DETAILED DESCRIPTION

[0042] In order to make the technical problems, technical solutions and advantages to be solved by the present invention clearer, the technical solutions of the present invention will be clearly and completely described below in conjunction with the accompanying drawings and specific embodiments. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. The components of the embodiments of the present invention generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the drawings is not intended to limit the scope of the claimed invention, but merely represents selected embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present invention.

[0043] Embodiment 1:

[0044] The embodiment of the present invention provides a multimodal biometric feature collection method, which is based on Figure 2 In the structure shown, the face lens and the iris lens are fixedly arranged on a rotatable carrier, the face lens is a visible light lens, and the iris lens is a near-infrared light lens. A fill light and a structured light are fixedly arranged on the carrier. Preferably, there are two fill lights, which are distributed on the left and right sides of the iris lens. The carrier is driven to rotate by a driving device such as a motor, and the rotation of the motor is controlled by instructions sent by a processor.

[0045] like Figure 1 As shown, the method includes:

[0046] S100: Acquire a facial image captured by the facial lens when the platform is located at an initial position.

[0047] The initial position of the stage refers to the position of the stage when the device starts working, which is not necessarily a fixed position. In some embodiments, the initial position can be fixed, and in other embodiments, the initial position can be the position of the stage when the device stopped working last time, or it can be the average value of the initial positions corresponding to multiple recognitions, which can reduce the degree of rotation of the stage and speed up the recognition speed.

[0048] S200: Control the stage to rotate to a first acquisition position, wherein the first acquisition position makes the iris centered within the field of view of the iris lens.

[0049] Since the field of view of the iris lens is small (generally about 30°), the iris image may not be captured at the initial position, so the stage needs to be rotated to the first capture position. At the first capture position, the iris image can be captured and the iris is located in the center of the image, so the captured iris image is of high quality.

[0050] The present invention does not limit the method used for rotating the stage to the first acquisition position. In some embodiments where the initial position is fixed, the stage can be rotated at a fixed angle to reach the first acquisition position. In some embodiments where the initial position is not fixed, the angle of rotation of the stage can be determined based on the position of specific features in the facial image.

[0051] S300: Control the fill light and the structured light to be turned on in a time-sharing and mutually exclusive manner, and obtain the iris image and the partial face 3D structured light image collected by the iris lens in a time-sharing manner.

[0052] In the prior art, the near-infrared imaging band of the structured light lamp during structured light imaging is different from the near-infrared imaging band of the fill light lamp during iris imaging, and the two will interfere with each other, so the existing 3D structured light face acquisition equipment cannot obtain iris information. The present invention uses an iris lens to acquire iris images and 3D structured light images of partial faces. To prevent the fill light lamp and the structured light from interfering with each other, when acquiring iris images and 3D structured light images of partial faces, the fill light lamp and the structured light lamp are turned on in a time-sharing and mutually exclusive manner, and the iris images and 3D structured light images of partial faces are acquired in a time-sharing manner through the iris lens.

[0053] For example, the fill light may be turned on first and the structured light may be turned off to collect the iris image, and then the structured light may be turned on and the fill light may be turned off to collect the 3D structured light image of the partial face. Alternatively, the structured light may be turned on and the fill light may be turned off to collect the 3D structured light image of the partial face, and then the fill light may be turned on and the structured light may be turned off to collect the iris image.

[0054] S400: Control the stage to rotate to at least one second acquisition position, and for each second acquisition position, control the fill light and the structured light to be turned on in a time-sharing and mutually exclusive manner, so as to obtain a local near-infrared face image and a local face 3D structured light image collected by the iris lens in a time-sharing manner.

[0055] Since the iris lens uses a camera with a small field of view, the captured image is only a part of the face. In order to obtain a complete near-infrared face image and a complete 3D face image, it is necessary to control the stage to rotate to other collection positions through the processor to collect local near-infrared face images and local face 3D structured light images at other collection positions.

[0056] S500: splicing the iris image and all partial near-infrared face images together to obtain a near-infrared face image containing a complete face, and splicing all partial face 3D structured light images together to obtain a 3D face image containing a complete face.

[0057] The present invention collects a visible light 2D face image through a face lens, and rotates a carrier to a first collection position, and mutually lights a fill light and a structured light at the first collection position to respectively collect an iris image and a local face 3D structured light image; then the carrier is rotated to at least one second collection position, and mutually lights a fill light and a structured light at each second collection position to respectively collect a local near-infrared face image and a local face 3D structured light image; finally, the iris image and all local near-infrared face images are spliced ​​to obtain a 2D near-infrared face image, and all local face 3D structured light images are spliced ​​to obtain a 3D face image.

[0058] The visible light 2D face image, iris image, 2D near-infrared face image and 3D face image obtained by the present invention can be used for identity recognition and / or liveness detection.

[0059] When performing identity recognition and / or liveness detection, the obtained iris image and 3D face image can be stored as two different feature sets respectively, wherein the iris features in the iris feature set are used for identity recognition and liveness detection, and the depth information in the 3D face feature set is used for identity recognition and liveness detection.

[0060] The visible light 2D face image and / or iris image and / or near-infrared face image and / or 3D face image may also be input into a pre-trained deep convolutional neural network for liveness detection.

[0061] The present invention reuses a near-infrared lens (iris lens) to collect iris images, 2D near-infrared face images and 3D face images, so that only two lenses, face lens and iris lens, can be used on one device to simultaneously collect visible light 2D face images, iris images, 2D near-infrared face images and 3D face images, and obtain 2D face information, iris information and 3D face information, thereby reducing the complexity and cost of the device and reducing the size of the device. In addition, the obtained 2D face information, iris information and 3D face information can be used for multi-modal biometric feature recognition and liveness detection, thereby improving the security and reliability of biometric feature recognition.

[0062] The inventors found during the research that the existing iris lens is usually installed at a fixed height and fixed position. At this time, the user needs to actively cooperate (bending / squatting, standing on tiptoe, etc.) to successfully capture the iris image. It is not suitable for users of different heights. Therefore, it is necessary to design the iris lens to be rotatable. However, in actual applications, the field of view of the iris lens is small (about 30°), so it often needs to be rotated and adjusted multiple times to make the user's eye position fall within the field of view of the iris lens. The adjustment speed is slow and the user experience is poor.

[0063] To solve the above problem, the method of controlling the stage to rotate to the first collection position of the present invention is as follows: Figure 6 As shown, including:

[0064] S201: Performing face detection on a face image to obtain a first current position of a human eye in the face image.

[0065] In this step, any known method in the prior art can be used for face detection, for example: face detection is performed on the face image using FaceCraft, a face detection algorithm based on Cascade CNN, and the feature points of the face can be located in combination with the SDM (Supvised Descent Method) method to obtain the first current position of the human eye. It should be pointed out that the face detection method is not limited to FaceCraft, and can also be, for example, Harr-AdaBoost, SSD (Single Shot MultiBox Detector), Faster RCNN, etc.; the face feature point positioning method is not limited to SDM, and can also be, for example, LBF (Local binary feature), LAB (Locally Assembled Binary), etc.

[0066] Preferably, if there are multiple faces in the face image, the largest and most centered face in the face image is used as the standard to obtain the first current position of the human eye. In this way, when multiple people stand in front of the acquisition device at the same time, the large face in the center is used as the standard, which is more in line with the user's understanding and usage habits, and the user experience is good.

[0067] S202: Calculate a first difference between a first current position of a human eye and a first target position, and when an absolute value of the first difference is greater than a first preset threshold, calculate a first angle difference according to the first difference.

[0068] The first difference value of this step is preferably the vertical difference between the first current position of the human eye and the first target position. Because the inventors found in the research process that the field of view angles of the visible light face lens and the near-infrared iris lens are usually large in the horizontal direction, when the user stands in front of the device, the position of the human eye can basically fall within the horizontal field of view, so the stage only needs to be rotated up and down, without rotating left or right, and only needs to calculate the vertical difference value.

[0069] The vertical field of view angle α of the face lens is relatively large (usually 90 degrees), and the vertical field of view angle β of the iris lens is relatively small (usually 30 degrees). The position of the human eye is very likely to fall into the field of view of the face lens, but not the field of view of the iris lens. Figure 3 As shown, in order to capture a clear iris image, it is necessary to adjust the stage to drive the face lens and the iris lens to rotate upward or downward so that the position of the human eye falls within the field of view of the iris lens, as shown in FIG. Figure 4 shown.

[0070] Since both the face lens and the iris lens are set on the stage and their positions are relatively fixed, there is a certain area in the collected face image. If the human eye is located in the certain area, the human eye falls into the field of view of the iris lens at the same time, and if the human eye is not in the certain area, the human eye does not fall into the field of view of the iris lens. The first target position is obtained by selecting the central position from the certain area.

[0071] The first current position and the first target position can be either an area range or a location point. The following description takes both as location points. Figure 5 As shown, in the face image displayed on the screen, assuming that the human eye is located in the dotted circle, the human eye falls into the field of view of the iris lens, and at this time the center of the dotted circle can be selected as the first target position; taking the left eye as an example, the center position of the left eye is taken as the first current position, assuming that the coordinates of the current center position of the left eye (i.e., the first current position) are (100, 250), and the coordinates of the center of the left dotted circle (i.e., the first target position) are (100, 450), then the first difference (pixel difference) between the first current position and the first target position in the vertical direction is 250-400=-150.

[0072] When the absolute value of the first difference is less than or equal to the first preset threshold, it means that the human eye falls within the field of view of the iris lens. At this time, there is no need to use the relevant data of the face image to perform rotation adjustment, and the process can jump directly to step S204. When the absolute value of the first difference is greater than the first preset threshold, it means that the human eye does not fall within the field of view of the iris lens. At this time, it is necessary to use the relevant data of the face image to perform rotation adjustment. The size of the first preset threshold can be flexibly set according to actual conditions. For example, in this embodiment, it can be set to 30. Since the absolute value of -150 is greater than 30, it means that the subsequent step S203 needs to be rotated.

[0073] S203: Calculating a first angle that the platform needs to rotate according to the first angle difference, and controlling the platform to rotate according to the first angle.

[0074] It can be understood that the first difference d between the first current position of the human eye and the first target position in the vertical direction is substantially linearly corresponding to the first angle at which the stage needs to rotate. The first angle at which the stage / lens needs to rotate can be calculated based on the first difference, that is, Figure 4 As shown in the first angle θ, in this embodiment, assuming that the calculated first angle θ is 40 degrees, it means that after the stage rotates downward 40 degrees, the human eye can fall into the field of view of the iris lens.

[0075] After calculating the first angle, the processor generates a first control instruction according to the first angle, so that a driving device such as a motor drives the carrier to rotate the first angle according to the first control instruction.

[0076] In this step, after the stage rotates by the first angle, the position of the human eye in the face image can be changed from the first current position ( Figure 5 The solid line eye position) moves to the first target position ( Figure 5 The user's eyes fall into the field of view of the iris lens (such as Figure 4 shown).

[0077] The above steps S201-S203 use the face image to achieve a rough adjustment of the "large rotation" so that the position of the human eye falls into the field of view of the iris lens as quickly as possible.

[0078] S204: Acquire a front iris image captured by the iris lens, and perform human eye detection on the front iris image to obtain a second current position of the human eye in the front iris image.

[0079] After the human eye position falls within the field of view of the iris lens, the front iris image is collected through the iris lens and human eye detection is performed. This step can use any known method in the prior art to perform human eye detection, such as the SDM, LBF, LAB, etc. described above.

[0080] S205: Calculate a second difference between a second current position of the human eye and a second target position, and when an absolute value of the second difference is greater than a second preset threshold, calculate a second angle difference according to the second difference.

[0081] When taking a face image, factors such as the size of the face and the distance from the lens may affect the accuracy of the first angle rotation of the stage, resulting in the iris not being in a stable and centered position in the iris lens. Therefore, it is necessary to perform a second rotation correction on the stage to make the iris in a stable and centered position for easier capture.

[0082] The principle of the second rotation is the same as the first rotation in S202, which will not be described in detail here. The difference between this step and S202 is that the relevant calculation is performed in the iris image. Since the resolution of the iris image is high, the calculation is more accurate.

[0083] The second target position of this step is different from the first target position. The first target position refers to the optimal position preset in the face image, and the second target position refers to the optimal position preset in the iris image.

[0084] When the absolute value of the second difference is less than or equal to the second preset threshold, it means that the iris is relatively centered within the field of view of the iris lens, and the iris image acquisition quality is relatively good. In this case, step S206 can be skipped, and the user identification is performed directly with the currently acquired iris image. When the absolute value of the second difference is greater than the second preset threshold, it means that the iris is not centered within the field of view of the iris lens, and the iris image acquisition quality is not guaranteed. In this case, it is necessary to use the relevant data of the iris image to perform rotation adjustment.

[0085] The size of the second preset threshold value can be flexibly set according to actual conditions, so that when the stage is rotated by the second angle, the iris can be centered within the field of view of the iris lens.

[0086] S206: Calculating a second angle that the stage needs to rotate according to the second angle difference, and controlling the stage to rotate to a first acquisition position according to the second angle.

[0087] After the stage rotates the second angle, it turns to the first acquisition position, at which the iris is centered within the field of view of the iris lens. Steps S204-S206 use the iris image to implement a "small correction" adjustment, so that the iris is centered within the field of view of the iris lens to ensure the quality of iris image acquisition.

[0088] The present invention sets the face lens and the iris lens on a horizontally rotatable platform. In the process of collecting iris images, the face lens is first used to collect face images, and the platform is adjusted by "large rotation" so that the position of the human eye falls into the field of view of the iris lens more quickly; then the iris lens is used to collect iris images, and the platform is adjusted by "small correction" so that the iris is centered within the field of view of the iris lens to ensure the quality of iris image collection. The present invention can collect clear iris images through two-step adjustment, and the adjustment speed is fast. In addition, the present invention can adapt to users of different heights, and users do not need to actively find a suitable collection position, thereby improving the user experience.

[0089] Existing 3D face acquisition devices generally require a distance sensor. When the distance sensor detects that the user is within the preset working range of the device, the device is started to work. In the present invention, no distance sensor is required. When the face lens detects the face or the iris lens detects the iris, the device is started to work. Since no distance sensor is required, the device cost is reduced.

[0090] In some other embodiments of the present invention, an angle sensor may be provided at the rotating shaft of the stage, and the signal output end of the angle sensor is connected to the processor. The angle sensor is used to detect whether the rotation angle of the stage reaches the aforementioned first angle and second angle. If not, the processor controls the motor to continue rotating until the angle sensor detects that the rotation angle of the stage reaches the aforementioned first angle and second angle. Among them, the angle sensor can use a magnetic encoder, which has the advantages of small size and large rotation range, thereby reducing the size of the multimodal acquisition device.

[0091] In the aforementioned S202, the inventor found that the first angle difference is slightly related to the distance between the user and the face lens. When the user is closer, the face image on the screen is slightly larger, and the first angle of rotation of the stage can be slightly smaller; when the user is farther away, the face image on the screen is slightly smaller, and the first angle of rotation can be slightly larger.

[0092] Therefore, before calculating the first angle that the platform needs to rotate according to the first angle difference, the present invention further includes: multiplying the first angle difference by a coefficient k for correction, where k=standard face size / face size in the face image.

[0093] The standard face size can be obtained by pre-collecting the face sizes of different users in face images at the optimal usage distance in front of the face camera, and taking the average of the face sizes.

[0094] In one example, assuming that there is one second acquisition position, S300 includes:

[0095] S301: Control the fill light to turn on and keep the structured light off, and obtain an iris image captured by the iris lens at a first capturing position.

[0096] In this step, the iris lens captures an iris image at the first capture position. The iris image includes the iris and is the upper half of the entire near-infrared face image.

[0097] S302: Control the fill light to turn off and the structured light to turn on, and obtain a first partial face 3D structured light image collected by the iris lens.

[0098] When the iris image acquisition is completed, the stage does not move, the processor controls the fill light to turn off, the structured light to light up, and the first partial face 3D structured light image is acquired through the iris lens. The first partial face 3D structured light image is the upper half of the 3D face image.

[0099] In this step, since the imaging bands of the iris image and the 3D structured light image are different, when the processor determines that the complete iris image has been collected, it controls the fill light to turn off and controls the structured light to light up, so as to collect the first partial face 3D structured light image of the user. Since the field of view of the iris lens is small, the first partial face 3D structured light image is the upper half of the 3D face image, and the first partial face 3D structured light image and the iris image are located in the same coordinate system.

[0100] S400 includes:

[0101] S401: Control the stage to rotate to a second acquisition position, and control the fill light to turn on and the structured light to turn off, to obtain a local near-infrared face image including a mouth area acquired by an iris lens.

[0102] Since the iris lens uses a camera with a small field of view, it is unable to capture the entire near-infrared face image. Therefore, after the upper half of the 3D face image is captured, in order to obtain the entire near-infrared face image, it is necessary to control the rotation of the stage through the processor to capture the lower half of the near-infrared face image.

[0103] When the processor controls the stage to rotate downward, the mouth area is detected on the near-infrared image collected in real time. When the mouth area is detected to be centered in the image, the stage stops rotating and collects a local near-infrared face image including the mouth area. When collecting the local near-infrared face image, the processor controls the structured light to turn off and the fill light to turn on. The local near-infrared face image collected in this step is the lower half of the near-infrared face image, and the local near-infrared face image and the aforementioned iris image are located in the same coordinate system.

[0104] S402: Control the fill light to turn off and the structured light to turn on, and obtain a second partial face 3D structured light image collected by the iris lens.

[0105] When the partial near-infrared face image is acquired, the stage does not move, the processor controls the fill light to turn off and the structured light to turn on, and acquires the second partial face 3D structured light image. The second partial face 3D structured light image is the upper half of the 3D face image, and is located in the same coordinate system as the aforementioned partial near-infrared face image.

[0106] After obtaining the iris image, the partial near-infrared face image, the first partial face 3D structured light image, and the second partial face 3D structured light image, the iris image and the partial near-infrared face image can be spliced ​​to obtain a complete near-infrared face image, and the first partial face 3D structured light image and the second partial face 3D structured light image can be spliced ​​to obtain a complete 3D face image.

[0107] In the present invention, when the iris image and the partial near-infrared face image are spliced ​​to obtain a complete near-infrared face image, any image splicing method in the prior art can be used, or the splicing method provided in the present invention can be used, and the present invention does not limit this.

[0108] As an example of the splicing method provided by the present invention, Figure 7 As shown, it is applicable to the case where the iris image and the local near-infrared face image have a certain overlapping area. The method of this example includes:

[0109] S501: extracting feature points of the iris image and all local near-infrared face images, and counting the feature points at the same positions of the iris image and all local near-infrared face images.

[0110] Both the iris image and the local near-infrared face image include a part of the face, and the feature points of the iris image and the local near-infrared face image can be extracted by various face positioning and feature point extraction algorithms in the prior art. Since there are overlapping areas of appropriate size between the iris image and the local near-infrared face image, there are feature points at the same position in these overlapping areas.

[0111] Taking the second acquisition position as an example and the local near-infrared face image as one, assuming that the feature points of the nose are both located on the iris image and the local near-infrared face image, then the feature points of the nose are the feature points at the same position.

[0112] S502: Align the feature points at the same position of the iris image and all local near-infrared face images to find the overlapping area.

[0113] The feature points located at the same position on the iris image and the local near-infrared face image have the same position on the entire face image, and the iris image and the local near-infrared face image are located in the same coordinate system. Therefore, the iris image and the local near-infrared face image can be aligned to the corresponding positions of the same face by aligning the feature points at the same position on the iris image and the local near-infrared face image according to the coordinates of the feature points at the same position. After the iris image and the local near-infrared face image are aligned, the overlapping part of the images is the overlapping area.

[0114] S503: Based on the aforementioned overlapping area, the iris image and all local near-infrared face images are fused to obtain the near-infrared face image.

[0115] After the iris image and the local near-infrared face image are aligned, the overlapping area includes multiple layers of images. In this step, when the iris image and all the local near-infrared face images are fused, the multiple layers of images in the overlapping area are fused into one layer of image, so that the iris image and the local near-infrared face image are pieced together into a complete near-infrared face image.

[0116] For example, Fig. 9 Schematic diagram of an iris image, which is the upper half of a face, including multiple feature points of the upper half of the face. Fig.10 is a schematic diagram of a partial near-infrared face image, which is the lower half of the face, including multiple feature points of the lower half of the face. Both the iris image and the partial near-infrared face image include feature points in the nose area, which are feature points at the same position. The overlapping part of the iris image and the partial near-infrared face image (the area where the nose is located) is the overlapping area. Fig. 9 ,and Fig.10 The stitched image obtained after stitching is as follows Fig.11 As shown, it is a complete near-infrared face image.

[0117] The above-mentioned overlapping area-based stitching method is simple, convenient and has high precision.

[0118] When the iris image and the local near-infrared face image do not have an overlapping area or the overlapping area is small and thus do not have feature points at the same position, the present invention achieves the splicing of the iris image and the local near-infrared face image by the following method: Figure 8 shown.

[0119] S501': extracting feature points of the face image, iris image and all local near-infrared face images.

[0120] S502': taking the feature points of the face image as a reference, aligning the feature points of the iris image and all local near-infrared face images with the feature points of the face image at the same position.

[0121] Since the face image, iris image and local near-infrared face image are in the same coordinate system, the feature points of the face image can be used as a reference.

[0122] The face image includes a complete face, the feature points extracted from the face image include all the face feature points, and the iris image and the partial near-infrared face image both include some of the face feature points. The face feature points of the iris image and the partial near-infrared face image are aligned with the feature points at the same position on the face image, so that the iris image and the partial near-infrared face image are aligned to the corresponding positions of the same face.

[0123] S503': Fusing the aligned iris image and all local near-infrared face images together to obtain the near-infrared face image.

[0124] The above-mentioned stitching method based on the feature points of the face image has a wide range of applications.

[0125] The specific method of stitching the partial face 3D structured light images to obtain the 3D face image can refer to the aforementioned method of stitching the iris image and the partial near-infrared face image to obtain the near-infrared face image. When each partial face 3D structured light image has a certain overlapping area, refer to S501 to 503, and when each partial face 3D structured light image does not have an overlapping area, refer to S501' to 503'. At this time, the feature points of the face image can be used as the reference, or the feature points of the near-infrared face image obtained after stitching can be used as the reference.

[0126] To summarize, the present invention firstly utilizes a face lens to capture a face image and performs face detection. After locating the eye coordinates, a first angle is calculated based on the difference between the located eye coordinates and the first target position. After the stage drives the face lens and the iris lens to rotate by the first angle, the eye can quickly fall into the acquisition range of the iris lens.

[0127] Then, the iris lens is used to collect the user's front iris image, detect the iris image, locate the eye coordinates, and calculate the second angle based on the difference between the eye coordinates in the front iris image collected by the iris lens and the second target position. When the platform drives the face lens and the iris lens to rotate the second angle, the eye coordinates fall into the best collection position of the iris lens, and the iris lens is used to collect the iris image at this time; when the iris image collection is completed, the fill light is turned off and the structured light is turned on, and the iris lens is used to collect the first part of the 3D structured light image; then, the fill light is turned on again.

[0128] The stage drives the face lens and iris lens to rotate downward to collect a local near-infrared face image of the mouth area. After the local near-infrared face image of the mouth area is collected, the fill light goes out and the structured light turns on to collect the second part of the 3D structured light image.

[0129] The collected partial near-infrared face images of the mouth area are stitched together to obtain the entire near-infrared face image, and the first part of the 3D structured light image and the second part of the 3D structured light image are stitched together to obtain the entire 3D face image. Thus, the iris image and the 3D face image can be collected using one near-infrared lens, and the visible light 2D face image, iris image, 2D near-infrared face image and 3D face image can be obtained simultaneously.

[0130] Embodiment 2:

[0131] An embodiment of the present invention provides a multimodal biometric feature acquisition device, wherein a face lens and an iris lens are fixedly arranged on a rotatable carrier, and a fill light and a structured light are fixedly arranged on the carrier.

[0132] like Fig.12 As shown, the device comprises:

[0133] The first acquisition module 100 is used to acquire a face image captured by the face lens when the platform is located at an initial position.

[0134] The rotation module 200 is used to control the stage to rotate to a first acquisition position, wherein the first acquisition position makes the iris centered within the field of view of the iris lens.

[0135] The second acquisition module 300 is used to control the fill light and the structured light to be turned on in a time-sharing and mutually exclusive manner, and to acquire the iris image and the partial face 3D structured light image collected by the iris lens in a time-sharing manner.

[0136] The third acquisition module 400 is used to control the stage to rotate to at least one second acquisition position, and for each second acquisition position, control the fill light and the structured light to be turned on time-sharing and mutually exclusive, so as to obtain the local near-infrared face image and the local face 3D structured light image collected by the iris lens time-sharing.

[0137] The stitching module 500 is used to stitch the iris image and all the local near-infrared face images together to obtain a near-infrared face image containing a complete face, and to stitch all the local face 3D structured light images together to obtain a 3D face image containing a complete face.

[0138] The present invention reuses an iris lens to collect iris images, 2D near-infrared face images and 3D face images, so that only two lenses, face lens and iris lens, are used on one device to collect visible light 2D face images, iris images, 2D near-infrared face images and 3D face images at the same time, and obtain 2D face information, iris information and 3D face information, thereby reducing the complexity and cost of the device and reducing the size of the device. In addition, the obtained 2D face information, iris information and 3D face information can be used for multi-modal biometric feature recognition and liveness detection, thereby improving the security and reliability of biometric feature recognition.

[0139] The rotating module 200 is as follows Fig.13 As shown, including:

[0140] The first detection unit 201 is used to perform face detection on the face image to obtain a first current position of a human eye in the face image.

[0141] The first calculation unit 202 is used to calculate a first difference between a first current position of a human eye and a first target position, and when an absolute value of the first difference is greater than a first preset threshold, a first angle difference is calculated based on the first difference.

[0142] The first rotating unit 203 is used to calculate a first angle that the carrier needs to rotate according to the first angle difference, and control the carrier to rotate according to the first angle.

[0143] The second detection unit 204 is used to obtain the front iris image collected by the iris lens, and perform human eye detection on the front iris image to obtain a second current position of the human eye in the front iris image.

[0144] The second calculation unit 205 is used to calculate a second difference between a second current position of the human eye and a second target position, and when an absolute value of the second difference is greater than a second preset threshold, a second angle difference is calculated based on the second difference.

[0145] The second rotation unit 206 is used to calculate a second angle that the stage needs to rotate according to the second angle difference, and control the stage to rotate to the first acquisition position according to the second angle.

[0146] Preferably, the first rotating unit further comprises:

[0147] The first angle difference is multiplied by a coefficient k for correction, where k=standard face size / face size in the face image.

[0148] The splicing module 500 is as follows Fig.14 As shown, including:

[0149] The first extraction unit 501 is used to extract feature points of the iris image and all local near-infrared face images, and count the feature points at the same position of the iris image and all local near-infrared face images.

[0150] The first alignment unit 502 is used to align the feature points at the same position of the iris image and all the local near-infrared face images to find the overlapping area.

[0151] The first fusion unit 503 is used to fuse the iris image and all local near-infrared face images based on the overlapping area to obtain the near-infrared face image.

[0152] Alternatively, the splicing module 500 is as follows Fig.15 As shown, including:

[0153] The second extraction unit 501 ′ is used to extract feature points of the face image, iris image and all local near-infrared face images.

[0154] The second alignment unit 502' is used to align the feature points of the iris image and all local near-infrared face images with the feature points of the face image at the same position based on the feature points of the face image.

[0155] The second fusion unit 503 ′ is used to fuse the aligned iris image and all the local near-infrared face images together to obtain the near-infrared face image.

[0156] To summarize, the present invention firstly utilizes a face lens to capture a face image and performs face detection. After locating the eye coordinates, a first angle is calculated based on the difference between the located eye coordinates and the first target position. After the stage drives the face lens and the iris lens to rotate by the first angle, the eye can quickly fall into the acquisition range of the iris lens.

[0157] Then, the iris lens is used to collect the user's front iris image, detect the iris image, locate the eye coordinates, and calculate the second angle based on the difference between the eye coordinates in the front iris image collected by the iris lens and the second target position. When the platform drives the face lens and the iris lens to rotate the second angle, the eye coordinates fall into the best collection position of the iris lens, and the iris lens is used to collect the iris image at this time; when the iris image collection is completed, the fill light is turned off and the structured light is turned on, and the iris lens is used to collect the first part of the 3D structured light image; then, the fill light is turned on again.

[0158] The stage drives the face lens and iris lens to rotate downward to collect a local near-infrared face image of the mouth area. After the local near-infrared face image of the mouth area is collected, the fill light goes out and the structured light turns on to collect the second part of the 3D structured light image.

[0159] The collected partial near-infrared face images of the mouth area are stitched together to obtain the entire near-infrared face image, and the first part of the 3D structured light image and the second part of the 3D structured light image are stitched together to obtain the entire 3D face image. Thus, the iris image and the 3D face image can be collected using one near-infrared lens, and the visible light 2D face image, iris image, 2D near-infrared face image and 3D face image can be obtained simultaneously.

[0160] The device provided in the embodiment of the present invention has the same implementation principle and technical effects as those in the aforementioned method embodiment 1. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiment 1. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the devices and units described above can all refer to the corresponding processes in the aforementioned method embodiment 1, and will not be repeated here.

[0161] Embodiment 3:

[0162] The method described in the above embodiment 1 provided by the present invention can realize the business logic through a computer program and record it on a storage medium, and the storage medium can be read and executed by a computer to achieve the effect of the solution described in embodiment 1 of this specification. Therefore, the present invention also provides a computer-readable storage medium for multimodal biometric feature collection, including a memory for storing processor-executable instructions, and when the instructions are executed by the processor, the steps of the multimodal biometric feature collection method including embodiment 1 are implemented.

[0163] The storage medium may include a physical device for storing information, which is usually a medium that digitizes the information and then stores it in an electrical, magnetic or optical manner. The storage medium may include: a device that stores information in an electrical energy manner, such as various memories, such as RAM, ROM, etc.; a device that stores information in a magnetic energy manner, such as a hard disk, a floppy disk, a magnetic tape, a magnetic core memory, a magnetic bubble memory, a USB flash drive; a device that stores information in an optical manner, such as a CD or a DVD. Of course, there are other readable storage media, such as quantum memory, graphene memory, etc.

[0164] The present invention reuses an iris lens to collect iris images, 2D near-infrared face images and 3D face images, so that only two lenses, face lens and iris lens, are used on one device to collect visible light 2D face images, iris images, 2D near-infrared face images and 3D face images at the same time, and obtain 2D face information, iris information and 3D face information, thereby reducing the complexity and cost of the device and reducing the size of the device. In addition, the obtained 2D face information, iris information and 3D face information can be used for multi-modal biometric feature recognition and liveness detection, thereby improving the security and reliability of biometric feature recognition.

[0165] The storage medium described above may also include other implementation methods according to the description of method embodiment 1. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment 1. For details, please refer to the description of the relevant method embodiment 1, and no further details will be given here.

[0166] Embodiment 4:

[0167] The present invention also provides a device for multimodal biometric collection, which may be a separate computer, or may include an actual operating device using one or more of the methods or one or more of the embodiments of this specification. The multimodal biometric collection device may include at least one processor and a memory storing computer executable instructions, and when the processor executes the instructions, the steps of the multimodal biometric collection method described in any one or more of Embodiment 1 are implemented.

[0168] The present invention reuses an iris lens to collect iris images, 2D near-infrared face images and 3D face images, so that only two lenses, face lens and iris lens, are used on one device to collect visible light 2D face images, iris images, 2D near-infrared face images and 3D face images at the same time, and obtain 2D face information, iris information and 3D face information, thereby reducing the complexity and cost of the device and reducing the size of the device. In addition, the obtained 2D face information, iris information and 3D face information can be used for multi-modal biometric feature recognition and liveness detection, thereby improving the security and reliability of biometric feature recognition.

[0169] The above-mentioned device can also include other implementation methods according to the description of method embodiment 1. The implementation principle and technical effects produced by this embodiment are the same as those of the aforementioned method embodiment 1. For details, please refer to the description of the relevant method embodiment 1, and no further description will be given here.

[0170] Finally, it should be noted that the above-described embodiments are only specific implementations of the present invention, which are used to illustrate the technical solutions of the present invention, rather than to limit them. The protection scope of the present invention is not limited thereto. Although the present invention is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed by the present invention, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. They should all be covered within the protection scope of the present invention. Therefore, the protection scope of the present invention shall be based on the protection scope of the claims.

Claims

1. A multimodal biometric feature collection method, characterized in that: The face lens and the iris lens are fixedly arranged on a rotatable carrier, and a fill light and a structured light are fixedly arranged on the carrier. The method comprises: Acquire a face image captured by the face lens when the platform is located at an initial position; Controlling the stage to rotate to a first acquisition position, wherein the first acquisition position makes the iris centered within the field of view of the iris lens; Control the fill light and structured light to be turned on in a time-sharing and mutually exclusive manner, and obtain the iris image and the partial face 3D structured light image collected by the iris lens in a time-sharing manner; Control the stage to rotate to at least one second acquisition position, and for each second acquisition position, control the fill light and the structured light to be turned on in a time-sharing and mutually exclusive manner, so as to obtain a local near-infrared face image and a local face 3D structured light image collected in a time-sharing manner by the iris lens; The iris image and all the local near-infrared face images are spliced ​​together to obtain a near-infrared face image containing a complete face, and all the local face 3D structured light images are spliced ​​together to obtain a 3D face image containing a complete face.

2. The multimodal biometric feature acquisition method according to claim 1, characterized in that: The controlling the stage to rotate to the first collection position comprises: Performing face detection on the face image to obtain a first current position of a human eye in the face image; Calculating a first difference between a first current position of a human eye and a first target position, and when an absolute value of the first difference is greater than a first preset threshold, calculating a first angle difference according to the first difference; Calculating a first angle that the stage needs to rotate according to the first angle difference, and controlling the stage to rotate according to the first angle; Acquire a front iris image captured by an iris lens, and perform human eye detection on the front iris image to obtain a second current position of the human eye in the front iris image; Calculating a second difference between a second current position of the human eye and a second target position, and when an absolute value of the second difference is greater than a second preset threshold, calculating a second angle difference according to the second difference; The second angle that the stage needs to rotate is calculated according to the second angle difference, and the stage is controlled to rotate to the first acquisition position according to the second angle.

3. The multimodal biometric feature acquisition method according to claim 2, characterized in that: The step of calculating the first angle that the stage needs to rotate according to the first angle difference further includes: The first angle difference is multiplied by a coefficient k for correction, where k=standard face size / face size in the face image.

4. The multimodal biometric feature acquisition method according to any one of claims 1 to 3, characterized in that: The step of splicing the iris image and all the local near-infrared face images together to obtain a near-infrared face image containing a complete face includes: Extracting feature points of the iris image and all local near-infrared face images, and counting the feature points at the same positions of the iris image and all local near-infrared face images; Aligning the feature points at the same position of the iris image and all local near-infrared face images to find the overlapping area; Based on the overlapping area, the iris image and all local near-infrared face images are fused to obtain the near-infrared face image.

5. The multimodal biometric feature acquisition method according to any one of claims 1 to 3, characterized in that: The step of splicing the iris image and all the local near-infrared face images together to obtain a near-infrared face image containing a complete face includes: Extracting feature points of the face image, iris image and all local near-infrared face images; Taking the feature points of the face image as a reference, align the feature points of the iris image and all local near-infrared face images with the feature points of the face image at the same position; The aligned iris image and all local near-infrared face images are fused together to obtain the near-infrared face image.

6. A multimodal biometric feature collection device, characterized in that: The face lens and the iris lens are fixedly arranged on a rotatable carrier, and a fill light and a structured light are fixedly arranged on the carrier. The device comprises: A first acquisition module, used for acquiring a face image captured by the face lens when the platform is located at an initial position; A rotation module, used for controlling the stage to rotate to a first acquisition position, wherein the first acquisition position makes the iris centered within the field of view of the iris lens; The second acquisition module is used to control the fill light and the structured light to be turned on in a time-sharing and mutually exclusive manner, and to acquire the iris image and the partial face 3D structured light image collected by the iris lens in a time-sharing manner; A third acquisition module is used to control the stage to rotate to at least one second acquisition position, and for each second acquisition position, control the fill light and the structured light to be turned on in a time-sharing and mutually exclusive manner, so as to acquire a local near-infrared face image and a local face 3D structured light image acquired in a time-sharing manner by the iris lens; The splicing module is used to splice the iris image and all the local near-infrared face images together to obtain a near-infrared face image containing a complete face, and to splice all the local face 3D structured light images together to obtain a 3D face image containing a complete face.

7. The multimodal biometric feature acquisition device according to claim 6, characterized in that: The rotation module comprises: A first detection unit, configured to perform face detection on the face image to obtain a first current position of a human eye in the face image; A first calculation unit is used to calculate a first difference between a first current position of a human eye and a first target position, and when an absolute value of the first difference is greater than a first preset threshold, a first angle difference is calculated according to the first difference; A first rotating unit, used for calculating a first angle that the carrier needs to rotate according to the first angle difference, and controlling the carrier to rotate according to the first angle; A second detection unit is used to obtain a front iris image collected by the iris lens, and perform human eye detection on the front iris image to obtain a second current position of the human eye in the front iris image; A second calculation unit is used to calculate a second difference between a second current position of the human eye and a second target position, and when an absolute value of the second difference is greater than a second preset threshold, a second angle difference is calculated according to the second difference; The second rotation unit is used to calculate a second angle that the stage needs to rotate according to the second angle difference, and control the stage to rotate to the first collection position according to the second angle.

8. The multimodal biometric feature acquisition device according to claim 6 or 7, characterized in that: The splicing module comprises: A first extraction unit, configured to extract feature points of the iris image and all local near-infrared face images, and to count feature points at the same positions of the iris image and all local near-infrared face images; A first alignment unit is used to align the feature points at the same position of the iris image and all local near-infrared face images to find the overlapping area; The first fusion unit is used to fuse the iris image and all local near-infrared face images based on the overlapping area.

9. A computer-readable storage medium for multimodal biometric acquisition, characterized in that: It includes a memory for storing processor executable instructions, and when the instructions are executed by the processor, the steps of the multimodal biometric feature collection method according to any one of claims 1 to 5 are implemented.

10. A device for multimodal biometric collection, characterized in that: The method comprises at least one processor and a memory storing computer executable instructions, wherein when the processor executes the instructions, the steps of the multimodal biometric feature collection method described in any one of claims 1 to 5 are implemented.

Images