Non-contact fingerprint image slap segmentation

Abstract

The invention relates to a method for segmenting palmprint images and generating accurate labeled individual fingerprints, said method comprising the following steps: - receiving an input image from a non-contact fingerprint reader under controlled lighting conditions; - computing the variance in the received image to estimate the palm region as a foreground palm mask in the input image; - identifying individual fingers by finding the boundaries of each finger; - verifying the number of fingers and geometric constraints; - computing the pose and orientation based on shape and geometric information; - identifying the valid fingertip region on each detected finger according to the pose, orientation, and geometric information; - outputting individual fingerprints.

Description

Technical Field

[0001] This invention relates to a method and apparatus for slap segmentation of contactless fingerprint images. Background Technology

[0002] More specifically, contactless fingerprint images are captured by contactless fingerprint readers. Contactless fingerprint readers have emerged in recent years. Unlike traditional livescan fingerprint readers that only acquire fingerprints from the touch device, contactless fingerprint readers capture images of all fingers / palms / hands presented to the device. To identify the location and valid area of ​​each individual fingerprint, the captured palm image needs to be automatically segmented.

[0003] Biometric data, such as fingerprints, plays an increasingly important role in fields such as data security, border control, law enforcement, finance, healthcare, and more. In Automated Fingerprint Identification Systems (AFIS), identification relies on the unique pattern of a fingerprint—the ridges and valleys on the surface of the fingertip. Traditional fingerprint images are obtained either from ink imprints on ten-print cards or by touch-based fingerprint capture devices, such as optical and capacitive readers. In recent years, with the increasing demand for faster fingerprint acquisition, registration, and matching, contactless fingerprint readers have been developed, which conveniently capture fingerprint images without any physical contact between the subject's fingers and the device. Despite the morphological changes, both contact and contactless fingerprint acquisition share a key step: how to accurately segment the desired individual fingerprint from the captured image, especially from a palm image involving multiple flat fingerprints captured simultaneously by placing the left / right palm of each of the four fingers (index, middle, ring, little, or both thumbs) on an ink card or fingerprint reader.

[0004] Considering the nature of ink cards and contact fingerprint readers, traditional contact palm segmentation methods primarily use shape, texture, orientation, and geometric constraints to identify each fingerprint.

[0005] Such methods for segmenting contact fingerprint images can be found in the following documents: US 7,072,496 B2, Craig I. Watson, “Slap Fingerprint Segmentation Evaluation II - Procedures and Results,” NIST Interagency / Internal Report (NISTIR) – 7553, 2009; Brad Ulery et al., “Slap Fingerprint Segmentation Evaluation 2004 Analysis Report,” NIST Interagency / Internal Report (NISTIR) – 7209, 2005; Sklansky, J., “Finding the Convex Hull of a Simple Polygon,” Pattern Recognition Letters, Vol. 1, No. 2, pp. 79-83, 1982; Christian Wolf et al., “Text Localization, Enhancement and Binarization in Multimedia Documents,” International Conference on Pattern Recognition, Vol. 4, pp. 1037-1040, 2002; Yong-Liang Zhang, Gang Xiao, Yan-Miao Li, Hong-Tao Wu, and Ya-Ping Huang, 2010, Slap Fingerprint Segmentation for Live-ScanDevices and Ten-Print Cards, In Proceedings of the 2010 20th International Conference on Pattern Recognition (ICPR '10).

[0006] Contactless fingerprint readers capture palm images using a high-resolution camera without any touching of the object in an open space, which is much faster and more convenient. However, traditional palm segmentation algorithms for contact fingerprint images fail in many cases due to modal variations of the object.

[0007] In fact, it is very common for fingers in palm images from contactless readers to not be clearly separated from each other. This is the most challenging scenario for traditional palm segmentation algorithms on touch-based devices, which are developed based on the strong assumption that fingerprints from different fingers are well separated from each other.

[0008] Furthermore, with or without guidance, the palm (the object to be imaged) can be positioned above the device in various poses, allowing the appearance of the palm in the captured image (such as rotation, orientation, and position) to be significantly altered.

[0009] Finally, compared to touch-based palm images, non-contact palm images capture the entire palm, which is problematic for traditional methods that rely on texture information, as the upper palm may show patterns similar to fingerprints.

[0010] Therefore, it is desirable to design a new method and apparatus for accurately identifying and segmenting each individual fingerprint from a palm image captured by a contactless fingerprint reader, which is expected to adapt to various hand shapes, hand postures and distances of the subject.

[0011] Therefore, further alternatives and advantageous solutions are expected in this field. Summary of the Invention

[0012] The present invention aims to propose a segmentation method for palm images acquired in a non-contact manner.

[0013] This invention is defined in its broadest sense as a method for segmenting a palm image and generating accurately marked individual fingerprints, the method comprising the following steps:

[0014] - Receive input images from a contactless fingerprint reader under controlled lighting conditions;

[0015] - Calculate the variance in the received image to estimate the palm region of the foreground palm mask in the input image;

[0016] - Identify individual fingers by finding the boundaries of each finger;

[0017] -Verify the number and geometric constraints of the fingers;

[0018] - Calculate pose and orientation based on shape and geometric information;

[0019] - Identify the effective fingertip area on each detected finger based on posture, orientation, and geometric information;

[0020] - Output a separate fingerprint.

[0021] This invention is a unique and efficient solution that combines the hardware and software capabilities of contactless fingerprint identification systems. It requires only two images to calculate variance and minimizes hardware settings to solve the problem.

[0022] Advantageously, in terms of hardware, the images acquired under controlled lighting conditions are images acquired with and without a flash.

[0023] The software portion of the method then receives two palm images of the same object, one with the flash on and the other with the flash off. The device is then able to transmit, process, and save the captured images and the processed individual fingerprint images. In terms of software, the proposed algorithm is capable of adaptively estimating the location of each individual fingerprint and then generating their location information.

[0024] According to an advantageous embodiment, the individual finger identification step includes, according to the equation An adaptive binarization sub-step calculates the dynamic range of both global and local dynamics and applies a foreground palm mask to the original lighting image, where α(x,y) is the local dynamic factor, β(x,y) is the global dynamic factor, and p and q are predefined parameters. This adaptive binarization sub-step outputs a mask for a single finger.

[0025] The use of such adaptive thresholds enables the automation of individual finger detection.

[0026] According to an advantageous feature of the invention, the step of calculating posture and orientation based on shape and geometric information includes a convexity check for detecting the fingertip point.

[0027] This is a simple way to determine the orientation of a slap.

[0028] According to a preferred embodiment of the invention, the step of calculating pose and orientation based on shape and geometric information includes a sub-step of defining the centerline of each individual finger mask, a sub-step of calculating the gradient along the direction of the finger centerline on the finger image, and a sub-step of detecting the first finger joint by finding a vertical line with the maximum gradient at a certain distance to the fingertip, wherein the fingertip mask is defined by a region extending between the first finger joint and the fingertip.

[0029] The use of gradients enables the automatic localization of the different geometric features of the captured finger.

[0030] The present invention also relates to a contactless fingerprint image processor connected to at least one contactless fingerprint reader with various lighting conditions, and adapted to acquire images of a palm / finger at a contactless location near the reader under different lighting conditions for acquiring a user's fingerprint. The processor is adapted to segment the palm image received from the contactless fingerprint reader under controlled lighting conditions and generate accurately labeled individual fingerprints. The processor is adapted to calculate the variance in the received image to estimate the palm region as a foreground palm mask in the input image, identify individual fingers by finding the boundaries of each finger, verify the number of fingers and geometric constraints, calculate pose and orientation based on shape and geometric information, identify the valid fingertip region on each detected finger according to the pose, orientation, and geometric information, and output individual fingerprints.

[0031] Such a processor enables the implementation of the method of the present invention to segment images such as palm images received from a contactless fingerprint reader.

[0032] Advantageously, the processor is adapted to operate according to the equation The dynamic range of both global and local dynamics is calculated to perform adaptive binarization of the foreground hand mask applied to the original lighting image, where α(x,y) is the local dynamic factor, β(x,y) is the global dynamic factor, and p and q are predefined parameters. This adaptive binarization sub-step outputs a mask for each finger.

[0033] Similarly advantageously, the processor is adapted to calculate pose and orientation based on shape and geometric information, thereby using convexity checks to detect the fingertip point.

[0034] According to a preferred embodiment, the processor is adapted to calculate pose and orientation based on shape and geometric information, including defining a centerline for each individual finger mask, calculating the gradient on the finger image along the direction of the finger centerline, and detecting a first finger joint by finding a vertical line with the maximum gradient at a certain distance to the fingertip. The fingertip mask is defined by a region extending between the first finger joint and the fingertip.

[0035] In order to achieve the foregoing and related objectives, one or more embodiments include features that are fully described below and specifically pointed out in the claims. Attached Figure Description

[0036] The following description and accompanying drawings illustrate certain illustrative aspects in detail and indicate only some of the various ways in which the principles of the embodiments can be employed. Other advantages and novel features will become apparent from the following detailed description when considered in conjunction with the accompanying drawings, and the disclosed embodiments are intended to include all such aspects and their equivalents.

[0037] · Figure 1A schematic diagram illustrating a contactless fingerprint segmentation system;

[0038] · Figure 2 A flowchart illustrating the image processing workflow is shown;

[0039] · Figure 3 The flowchart illustrating the foreground palm mask estimation is shown.

[0040] · Figure 4 An example of foreground mask estimation is shown;

[0041] · Figure 5 A flowchart illustrating individual finger identifiers is shown;

[0042] · Figure 6 The diagram illustrates the finger gap markings;

[0043] · Figure 7 The illustration shows individual finger markers;

[0044] · Figure 8 The flowchart illustrating fingerprint pose estimation and fingertip identification is shown.

[0045] · Figure 9 The illustration shows a single fingertip icon;

[0046] · Figure 10 A flowchart illustrating the final fingerprint generation process is shown; and

[0047] · Figure 11 An example of the final individual fingerprint segmentation is shown. Detailed Implementation

[0048] To gain a more complete understanding of the invention, it will now be described in detail with reference to the accompanying drawings. The detailed description will illustrate and describe what is considered to be the preferred embodiments of the invention. It should be understood, of course, that various modifications and variations in form or detail can be readily made without departing from the spirit of the invention. Therefore, the invention is not intended to be limited to the exact forms and details shown and described herein, nor to anything less than the entire invention disclosed herein and claimed below. The same elements have been designated by the same reference numerals in different drawings. For clarity, only those elements and steps useful for understanding the invention have been shown in the drawings and will be described therein.

[0049] This invention can be implemented using a software system or a hardware system. Figure 1This is a schematic diagram of the proposed contactless fingerprint segmentation system 1, which includes at least a controller 16, including but not limited to a CPU, FPGA, DSP, or GPU. The controller manages all other modules, namely image I / O 11 and 14, which are processed by the image processor 12 and stored in the image memory 13.

[0050] The group of I / O ports 11 and 14 includes, but is not limited to, USB 2 / 3, Firewire, Thunderbolt, SATA, DMA, Ethernet, or the Internet. These ports 11 and 14 acquire the input palm image from the contactless fingerprint reader 10 and send the output fingerprint 15 to other devices / systems.

[0051] The image processor 12 can be implemented in any programming language, including but not limited to C / C++, JAVA, Python, assembly, or JavaScript. According to the present invention, the image processor 12 or processing module identifies and extracts individual fingerprint ROIs 15 from a palm image.

[0052] Image storage 13 includes, but is not limited to, RAM, ROM, SSD, hard disk drive, or NAS. Image storage 13 or storage module stores intermediate results and the final output fingerprint.

[0053] Figure 2 This is a flowchart illustrating the proposed image processing workflow of the present invention, specifically how to segment a palm image and generate accurately labeled individual fingerprint ROIs. In the first step S0, a palm image is received from a contactless fingerprint reader 10 under controlled lighting conditions. In step S1, the palm region in the input image is initially estimated as a foreground palm mask by calculating the variance in the received image. Then, in step S2, individual fingers are identified by finding the boundaries of each finger, while verifying the number of fingers and geometric constraints. In the next step S3, for each finger, the pose and orientation are calculated based on shape and geometric information. In step S4, the valid fingertip region on each detected finger is identified based on the pose, orientation, and geometric information. The individual fingerprint ROI 15 is then output. The fingertip region is described as a rotated rectangular ROI, and both the segmented fingerprint image and the coordinates of the rectangular ROI are then saved to a storage device or sent to other systems.

[0054] Figure 3 As shown in the diagram Figure 2The flowchart describes the process for identifying the palm / foreground region as described in step S1. In step I0, the contactless fingerprint reader captures an image of the palm under controlled lighting conditions, such as using a flash. Under certain lighting conditions, the foreground palm region will exhibit greater lighting variations in different images compared to the background region. Based on these findings, the variance in the image is calculated in step I1, and adaptive thresholds are set for different image regions.

[0055] Then, pixels with a variance greater than a threshold are designated as the foreground palm-shaped region, and the other pixels are designated as the background region. The pixel values ​​of the palm-shaped mask... M(x,y) It can be determined by the following equation:

[0056] ,in I D ( x,y ) is the pixel variance. T ( x,y ) is for pixels ( x,y An adaptive threshold is applied. Therefore, an initial palm mask for the foreground palm region is obtained in step I2. However, due to uneven lighting conditions and a noisy background, the initial palm mask may contain many spurious estimates. Therefore, morphological operations are applied in step I3. This includes opening and closing to fill holes and removing spurious noisy areas on the initial mask to obtain an image such as the one extracted from the palm image with the flash on. Figure 4 The accurate palm-sized area mask SM is shown in the image.

[0057] Figure 5 This is a flowchart illustrating the process of identifying a single finger. One advantage of the contactless fingerprint reader 10 is that the palm can be placed over the reader without any physical touch of the device, which helps improve the speed and convenience of palm image acquisition. However, as a side effect, the contactless fingerprint reader 10 offers far fewer constraints on the position of the finger used for image acquisition. Therefore, the finger can rotate, tilt, or bend during capture, and furthermore, in a four-finger palm image, the fingers can be very close to each other.

[0058] Therefore, the traditional palm segmentation method (which assumes that the fingers are well separated) will fail in most cases in contactless fingerprint systems. Thus, the most challenging part of identifying valid fingerprint ROIs is identifying each individual fingerprint.

[0059] The single-finger identification method of the present invention includes a first step in which two images are received, one being an original image IF, and the other being as described above. Figure 3 The obtained foreground mask SM.

[0060] Figure 6 The bottom image FS shows the gaps between the fingers that need to be determined from the two images, especially when the fingers are adjacent to each other.

[0061] In the first step F1, an adaptive binarization method is applied to the original image IF, constrained by the foreground mask. In this method, the binarization threshold is determined by calculating the dynamic range of both global and local dynamics, as shown in the following equation: ,in α ( x,y ) is a local dynamic factor for each pixel (x,y), which is the average value within a sliding window centered on pixel (x,y), where x and y are the coordinates of the pixel. β ( x,y () is a global dynamic factor calculated based on changes across the entire image. p and q It is a constant positive value, where p It equals the maximum standard deviation of the image, which is 128 for an 8-bit grayscale image, and q This is equal to a preset deviation within the range of [0.1, 0.6] based on different applications. This binarization allows for the definition of lines separating the fingers, such as... Figure 6 The bottom image FS is shown.

[0062] Then, as Figure 7 As illustrated in the image, to identify each individual finger, the convex hull surrounding the entire palm is first located in step F2, which helps in detecting the fingertip. Then, in step F3, the convexity inside the convex hull is examined to locate the finger base / joint. Typically, convexity defects are first calculated, i.e., areas not belonging to the palm but located inside the convex hull. In this case, it refers to the area between two fingers, as well as the areas on either side of the palm. Therefore, as illustrated in image TD, points in the convex defects with the maximum distance from the corresponding edge of the convex hull (i.e., the finger base / joint) are detected. Finally, as illustrated in image FD, each individual finger is ultimately identified by combining the positions of the fingertip and the finger base, and a corresponding mask FM is generated for each of them in step F4.

[0063] After identifying each individual finger in step F4, the position of the fingertip needs to be accurately detected.

[0064] Figure 8 This is a flowchart illustrating the process of finger pose estimation and fingertip segmentation according to the present invention. Figure 9 The illustration shows pose estimation and fingertip marking.

[0065] The finger pose is estimated by finding the centerline of the finger mask in step T1, such as... Figure 9As illustrated schematically on a single finger mask. In some special cases where the fingers are not straight, the average angle of the center line is estimated.

[0066] Based on the results of step T1, the gradient along the finger centerline on the finger image FI is calculated. Then, in step T2 of finger pose estimation, the first finger joint can be detected by finding the vertical line with the maximum gradient at a certain distance to the fingertip. This enables the determination of the fingertip mask TM.

[0067] Figure 10 This is a flowchart illustrating the process used to generate the final individual fingerprint image.

[0068] In the first step P1, input images are received, one is a palm fingerprint image IF, and the other is a set of individual fingertip masks TM, and a separate fingerprint image is generated by combining the palm image IF and the fingertip masks TM.

[0069] In step P2, post-processing is performed, including at least image smoothing and denoising operations applied to the individual fingerprint images to remove random noise. Advantageously, further morphological transformations are applied to the obtained fingerprint images to remove structural noise and smooth the edges of the fingerprints. The final individual fingerprint FIF is then generated, as follows. Figure 11 As shown above.

[0070] In the above detailed description, reference has been made to the accompanying drawings, which illustrate specific embodiments in which the invention may be practiced. These embodiments have been described in sufficient detail to enable those skilled in the art to practice the invention. Therefore, the above detailed description is not intended to be limiting, and the scope of the invention is defined only by the appended claims as properly interpreted and the full scope of their equivalents.

Claims

1. A method for segmenting a palm image and generating an accurately labeled individual fingerprint, the method comprising the following steps: - Receive input images from the contactless fingerprint reader with and without flash; - Calculate the pixel variance in the received image, and threshold the pixel variance to estimate the palm region as a foreground palm mask in the input image. - Identify individual fingers by finding the boundaries of each finger; -Verify the number and geometric constraints of the fingers; - Calculate pose and orientation based on shape and geometric information; - Identify the effective fingertip area on each detected finger based on posture, orientation, and geometric information; - Output a separate fingerprint; The individual finger identification step includes identifying the finger according to the equation. An adaptive binarization sub-step calculates the dynamic range of both global and local dynamics and applies a foreground palm mask to the original lighting image, where α(x,y) is the local dynamic factor, β(x,y) is the global dynamic factor, and p and q are predefined parameters. This adaptive binarization sub-step outputs a mask for a single finger.

2. The method for segmenting a palm image according to claim 1, wherein the step of calculating pose and orientation based on shape and geometric information includes a convexity check for detecting the fingertip point.

3. The method for segmenting a palm image according to claim 1, wherein the step of calculating pose and orientation based on shape and geometric information includes: The sub-steps define the centerline of each individual finger mask, calculate the gradient along the direction of the finger centerline on the finger image, and detect the first finger joint by finding the vertical line with the maximum gradient at a certain distance to the fingertip. The fingertip mask is defined by the region extending between the first finger joint and the fingertip.

4. A contactless fingerprint image processor, connected to at least one contactless fingerprint reader under various lighting conditions, and adapted to acquire images of a palm / finger at a contactless location near the reader, with and without a flash, for acquiring a user's fingerprint. The processor is adapted to segment the palm image received from the contactless fingerprint reader under controlled lighting conditions and generate accurately labeled individual fingerprints. The processor is adapted to calculate the pixel variance in the received image, threshold the pixel variance to estimate the palm region as a foreground palm mask in the input image, identify individual fingers by finding the boundaries of each finger, verify the number of fingers and geometric constraints, calculate pose and orientation based on shape and geometric information, identify the valid fingertip region on each detected finger according to the pose, orientation, and geometric information, and output an individual fingerprint; wherein... The contactless fingerprint image processor is adapted to acquire fingerprints according to the equation. The dynamic range of both global and local dynamics is calculated to perform adaptive binarization of the foreground hand mask applied to the original lighting image, where α(x,y) is the local dynamic factor, β(x,y) is the global dynamic factor, and p and q are predefined parameters. This adaptive binarization sub-step outputs a mask for each finger.

5. The non-contact fingerprint image processor according to claim 4, wherein the processor is adapted to calculate pose and orientation based on shape and geometric information, thereby using convexity inspection to detect the fingertip point of the finger.

6. The non-contact fingerprint image processor of claim 4, wherein the processor is adapted to calculate pose and orientation based on shape and geometric information, including defining a centerline for each individual finger mask, calculating a gradient on the finger image along the direction of the finger centerline, detecting a first finger joint by finding a vertical line with the maximum gradient at a distance to the fingertip, the fingertip mask being defined by a region extending between the first finger joint and the fingertip.

Images