System and method for indexing human fingerprint images

Abstract

Systems and methods for indexing human fingerprint images. The invention relates to a method for indexing a human fingerprint image. The method of the present invention comprises the steps of: defining a plurality of categories, each category corresponding to a local pattern of a fingerprint; constructing a ground truth value by allowing a machine learning model to determine an inference probability for each category for the acquired fingerprint image; inputting an original fingerprint image; assigning, by a machine learning model, a multi-dimensional vector to the original fingerprint image according to an inference probability determined for each category based on a ground truth value; generating an index key from the multi-dimensional vector of the original fingerprint; and indexing the fingerprint index key, the indexing including a multi-stage check, the multi-stage check including both an exclusive classification check and a continuous classification check.

Description

Technical Field

[0001] The present disclosure relates to a method of indexing human fingerprint images, and to an automatic fingerprint identification system, wherein fingerprint images are indexed according to the method. Background Art

[0002] Fingerprints are widely used biometric features for personal identification due to their uniqueness, persistence and user acceptability. Many automatic fingerprint identification systems (AFIS) have been deployed worldwide since their emergence in the 1980s. The size of the fingerprint database involved in AFIS continues to grow.

[0003] Representative examples of civilian applications include: the OBIM (Office of Biometric Identity Management) program, which holds over 200 million unique identities and processes over 300,000 biometric transactions per day; and India's Aadhar project, which is now the world's largest biometric deployment with the enrolment of over 1 billion tenprints of individuals that are considered unique.

[0004] Identification of a person requires comparing one or more fingerprints with all the fingerprints in such a huge database. Both high identification accuracy and high search efficiency are critical. NIST (National Institute of Standards and Technology) FpVTE2012 (Fingerprint Vendor Technology Evaluation 2012) is an evaluation of large-scale 1-to-many fingerprint identification algorithms using a registration sample size of millions. It has different test scenarios by using one, two, four, eight or ten fingerprints for searching. From their report, it can be seen that for one finger search, the most accurate submission D (from Cogent) achieved a FNIR (false negative identification rate) of 1.97% for the left index finger and 1.9% for the right index finger for a registration set of 100,000 subjects. But for two fingers search, the best FNIR is 0.27% for a registration set of 1.6 million subjects. For ten fingers search, the best FNIR is 0.09% for a registration set of 3 million subjects. This is why most AFIS systems use two or more fingerprints for identification, even though more storage and computing resources are required. From the report of NIST MINEX III (Evaluation of Large 1-to-1 Fingerprint Verification), it can be seen that the most accurate submissions have a matching time of about 1ms or 2ms for a single comparison. When trying to search a huge database through such a kind of 1-to-1 matching, the time consumption is challenging. However, another observation in the FPVTE2012 report is that in the most accurate submissions, ten-finger search is faster than two-finger search, which is because of the deployment of fingerprint indexing.

[0005] Fingerprint indexing is a fast filtering stage of large-scale fingerprint identification systems, which quickly cuts off most of the candidates and maintains a small portion for further accurate but slow inspection to produce a final decision. However, an efficient fingerprint indexing algorithm should ensure that the true mate of the query does exist in the candidate list. Penetration rate (PR), miss rate (MR) and indexing speed are usually used to measure indexing performance. Penetration rate is the proportion of the passing list to the entire candidate list, and miss rate is the proportion of missing pairs in the indexing stage. The optimal fingerprint index aims to minimize both PR (for speed or resources) and MR (for accuracy) by indexing at an extremely fast speed.

[0006] The traditional method for shortening the candidate list is by implementing fingerprint classification, and more specifically, exclusive classification. This technique divides the database into a fixed number of categories during the identification phase, such as Arch, Tented Arch, Left Loop, Right Loop, and Whorl, and the searched fingerprint is compared only with fingerprints belonging to the same category. This method is usually fast, but due to the small number of categories and the uneven distribution between categories, it is not possible to greatly reduce the number of candidates. Some existing AFIS still use exclusive classification for ten-finger fingerprint to ten-finger fingerprint matching, because the sequential ten-finger pattern brings much more discrimination, and the candidate list can be shortened to 12% (penetration) of the entire reference database with a low missing rate (less than 0.1%). Unlike general fingerprint classification, fingerprint indexing (also called continuous classification) represents fingerprints as some stable and usually fixed-length features for indexing.

[0007] Finally, many indexing techniques have been proposed in the past two decades. Depending on the features that have been used, they can be roughly grouped into two categories: based on “global features” or based on “local features”. Global features define the global pattern of ridges, and typical methods use the following: direction fields such as in the method disclosed in the document entitled “Indexing Fingerprint Databases Based on Multiple Features” by Johan de Boer et al. on November 29, 2011 and in the document entitled “Fingerprint classification by directional image partitioning” by R. Cappelli et al. (Transactions on Pattern Analysis and Machine Intelligence (TPAMI), Vol. 21, No. 5, May 1999); Finger Codes such as in the document entitled “Indexing Fingerprint Databases Based on Multiple Features” cited above; Textures such as in the method disclosed in the document entitled “A fingerprint orientation model based on 2D Fourier expansion (FOMFE) and its application to singular-point detection and fingerprint indexing” by Yi Wang et al. (TPAMI, Vol. 29, No. 4, April 2007). Local features are typically: minutiae and their neighborhoods, such as in the document titled “Fingerprint indexing and matching: An integrated approach” by Kai Cao et al. (IEEE International Joint Conference on Biometrics (IJCB)) in 2017; minutiae and ridge flow curvature, such as in patent document US6,181,807B; minutiae and neighborhoods, such as in patent document US6,941,003B[7]; or local areas of minutiae grouped by gradient histograms, such as in patent document US8,625,861B. Summary of the invention

[0008] The present disclosure intends to combine the advantages of exclusive classification (such as high-level description, fast speed, short-length vectors) with the high discrimination of continuous classification. In the present disclosure, using category information as ground-truth, a deep learning-based method is proposed to determine the vector for each ten-fingerprint, and the penetration rate is reduced to less than about 5% with a low missing rate (less than about 0.1%), and a matching speed of more than about 10 million object comparisons per second is produced when using a single thread.

[0009] According to a first aspect, the present invention relates to a method for indexing a human fingerprint image, comprising the following steps:

[0010] defining a plurality of categories, each category corresponding to a local pattern of a fingerprint;

[0011] The ground truth is constructed by allowing the machine learning model to determine the inferred probability of each class for the acquired fingerprint image;

[0012] Input the original fingerprint image;

[0013] Assigning, by the machine learning model, a multi-dimensional vector to the raw fingerprint image according to an inferred probability determined for each class based on the ground truth;

[0014] Generating an indexing key from the multi-dimensional vector of raw fingerprints; and

[0015] Fingerprint index keys are indexed, the indexing including a multi-stage check including both an exclusive classification check and a sequential classification check.

[0016] Preferably, - the machine learning model is a convolutional neural network; - the number of defined categories is included between 4 and 16; - the categories include at least four categories corresponding to local patterns: bow / tent bow, left winnowing net, right winnowing net and bucket shape; - the categories include at least one unknown category; - for the ten fingerprints of the object, the length of the index key is less than 250 bytes; - for the implementation of the exclusive classification check, the number of unpaired fingerprints exceeding a threshold is determined for each category to eliminate the number of unpaired fingerprints for each category exceeding the threshold; - the continuous classification check includes determining the Manhattan distance for each of the vectors; - the continuous classification stage includes determining the cosine distance for each of the vectors; and - the continuous classification stage includes determining the fusion distance for each of the vectors, and the fusion distance is a weighted sum of the Manhattan distance and the cosine distance.

[0017] According to a second aspect, the present disclosure relates to a fingerprint identification system, comprising a database, wherein fingerprints are indexed in the database according to the above method. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Other features and aspects of the present disclosure will become apparent from the following description and the accompanying drawings, in which:

[0019] Figure 1 An exemplary fingerprint identification system according to the present disclosure is presented;

[0020] Figure 2A , 2B , 2C, 2D and 2E are exemplary original images corresponding to predefined fingerprint categories of arch / tent arch, left dustpan, right dustpan, bucket and unknown implemented according to the method of the present disclosure;

[0021] Figure 3 is an exemplary schematic representation depicting the steps of a method according to the present disclosure allowing the use of a convolutional neural network to generate a multidimensional vector and, starting from a raw image of a fingerprint, generating a multidimensional vector allowing the construction of a ground truth;

[0022] Figure 4 is an exemplary scheme showing steps leading to obtaining a fusion vector according to the method of the present disclosure;

[0023] Figure 5 is an example of a table showing how to generate index keys according to the method of the present disclosure; and

[0024] Figure 6 Represents a multi-stage fingerprint indexing according to the method of the present disclosure. DETAILED DESCRIPTION

[0025] Figure 1A fingerprint identification system 101 for indexing human fingerprints according to the present disclosure is shown. In an example, the fingerprint identification system may be or form part of an automatic fingerprint identification system (AFIS). Such a system includes a retrieval system (one or more) 104 that stores and retrieves features of fingerprint images. For example, the retrieval system 104 may include a storage device 108, which may be one or more memory / storage devices such as a hard drive, a cloud drive, a server, etc., and may include a database containing fingerprints indexed according to the methods described in the disclosure herein. The retrieval system 104 may also include one or more application processors 109. For example, the retrieval system 104 may include circuits such as, but not limited to, one or more single-core or multi-core processors. The processor (one or more) may include any combination of general-purpose processors and special-purpose processors (e.g., graphics processors, application processors, etc.). The processor may be coupled to or may include a memory / storage device and may be configured to execute instructions stored in the memory / storage device to enable various applications or operating systems to run on the retrieval system. The fingerprint identification system may include additional elements, such as, for example, memory / storage, display, camera, sensor, or input / output (I / O) interface. In other embodiments, the described components may be included in more than one device (e.g., the circuit may be separately included in more than one device).

[0026] In one aspect of the present disclosure, the fingerprint identification system 101 may optionally include multiple workstations, such as an input station 112. The input station 112 may be further connected to a scanner, such as a flatbed scanner, or a ten-print live scanner and a digital camera. The scanner or camera is used to scan fingerprints or capture latent fingerprints, which can be loaded into a microprocessor. The loaded fingerprints are segmented, and a quality is assigned to each fingerprint. The fingerprints can be further enhanced and processed. Matching features are also extracted, loaded into the microprocessor, and sent to a distributed matcher controller. Based on the matching features, a filtered file database is loaded from a data storage and retrieval system to perform an initial match of searches and file records. The matching results are transmitted back to the distributed matcher controller for analysis. Index keys can be generated by the following method.

[0027] The present disclosure is intended to generate short vectors of indices for fingerprints to accelerate fingerprint identification by fingerprint identification systems. Unlike most existing methods that use features extracted from finger images (such as direction fields, minutiae, or singularities) to derive index keys, the present disclosure allows the index keys to be determined directly from the original fingerprint images. According to the present disclosure, a "ground truth" is constructed by assigning a vector to each fingerprint according to its category, where the ground truth is a set of acquired fingerprint images that have been matched with the category assignments. The category assignments can be "real" or "correct" category assignments for the fingerprint images. For example, the ground truth can be verified, such as by human intervention. Thus, if Figure 2A , 2B , 2C, 2D and 2E, several categories are predefined, such as the following five categories: bow / tent bow ( Figure 2A ), left-skirt shape ( Figure 2B ), right basket shape ( Figure 2C ), bucket-shaped ( Figure 2D ) and unknown ( Figure 2E ). Unknown labels are for fingerprints that are difficult to identify due to extremely low quality, as well as those ambiguous cases. Fewer categories may be used, such as only several of the five previously named categories may be used, or a different set of categories may be generated. It is also contemplated that more than five categories may be used. For example, additional categories may include any of a variety of whorls, such as a plain whorl, a twin-loop whorl, a central pocket loop whorl, and an accidental whorl. Categories may be defined in different ways (such as according to the needs or required specifications of a developer or researcher), including from four to eight categories, and fewer or more categories are considered as long as the descriptions are consistent.

[0028] Figure 3 The assigned multidimensional vector is shown in , in this case a five-dimensional vector. Each dimensional vector represents the inferred probability of belonging to the corresponding category. For example, in Figure 3, the fingerprint is definitely of the arcuate type, so for the ground truth, the first element is 1. A machine learning model can be used to assign the multidimensional vector to the image. For example, the machine learning model may have trained a multiscale model to represent the global information of the fingerprint. In an example, the machine learning model is implemented as a neural network (NN) model that is trained and / or configured to detect predefined categories within a fingerprint image. As discussed herein, a neural network (NN) is a computational model that uses a collection of connected nodes to process input data based on machine learning techniques. Because a neural network can be represented by connecting different operations together, a neural network is referred to as a network. A model of a NN (e.g., a feedforward neural network) can be represented as a graph that represents how operations are connected together from an input layer through one or more hidden layers and ultimately connected to an output layer, wherein each layer includes one or more nodes, and wherein different layers perform different types of operations on corresponding inputs.

[0029] In one or more implementations, the machine learning model is implemented as a convolutional neural network (CNN). As discussed herein, CNN refers to a specific type of neural network, but uses different types of layers consisting of nodes present in three dimensions, where the dimensions can change between layers. In a CNN, a node in a layer may only be connected to a subset of nodes in a previous layer. The final output layer can be fully connected and sized according to the number of classifiers. CNN can include various combinations of the following types of layers and in some instances include various multiples of each of the following types of layers, and various orders of the following types of layers: input layer, convolutional layer, pooling layer, rectified linear unit layer (ReLU) and fully connected layer. Part of the operation performed by the convolutional neural network includes using a set of filters (or kernels) that iterate on the input data based on one or more parameters.

[0030] Figure 4 An embodiment of ground truth construction is depicted where two models with the same structure but different input sizes are trained. The model structure can be predefined with filter layers, however, the parameters can be continuously updated with training images. Larger sized images may require more computation than smaller images. Multi-scale processing and fusion of vectors can produce complementation for the final performance. One has input size W1×H1 and the other has input size W2×H2. Both use CNN and output nd vectors. Resnet-18 convolutional neural network is an example of a network structure. The final index key is the fusion of the two vectors. Figure 5The fusion vector described in detail aims to obtain complementary information from two vectors and derive some features to speed up indexing, such as the class labels and normalization factors of the vectors.

[0031] Indexing is a pre-filtering stage before the accurate inspection stage. In the present disclosure, fingerprint indexing itself is a multi-stage process. Figure 6 As shown in, in the indexing stage, objects are selected from the registration database and passed to a multi-stage check (pass stage 1 to pass stage n). The vector generated for each fingerprint can be used for exclusive classification (selecting the category with the highest probability) and continuous classification. For ten fingerprint applications, the stage using exclusive classification will count the number of unmatched fingerprints for each candidate object and make a decision. The stage using continuous classification will calculate the distance between two objects and their vectors. The distance will be compared with a predefined threshold to make a decision. The indexing stage may include:

[0032] 1) Exclusive classification stage, used to remove, for example, the number of unpaired fingerprints (by category) exceeding a threshold numThres. Unpaired means that the corresponding categories of the corresponding fingerprints have different categories, that is, for ten fingerprints (i.e., 2 or more fingerprints per person / object), two fingerprints are unpaired, which means that their corresponding categories (such as the category with the highest probability in the vector) are different. If the answer is "yes" ( Figure 6 ), the comparison between the probe object and the gallery object ends with a similarity score of "zero", which means that there is no match. Otherwise (the answer is "no" - Figure 6 N in , and pass it to the next stage.

[0033] 2) A continuous classification stage using, for example, the Manhattan distance d. For nd vectors X and Y, the Manhattan distance is defined as:

[0034]

[0035] If the distance is below the threshold mdThres, it is passed to the next stage, otherwise a similarity score of "zero" is returned.

[0036] 3) Continuous classification stage using cosine distance. Cosine distance = 1 - cosine similarity, and cosine similarity is defined as:

[0037]

[0038] If the cosine distance is below cdThres, it is passed to the next stage, otherwise a similarity score of "zero" is returned.

[0039] 4) A continuous classification stage for checking the fusion distance, which is a weighted sum of the Manhattan distance and the cosine distance. If the distance is below fdThres, the candidate is sent to the next stage, otherwise a similarity score of "zero" is returned. If the candidate passes all the above stages, it is passed to further accurate checking stages (such as accurate matching based on minutiae, which uses two sets of minutiae points (which are several feature points for a fingerprint) to perform point matching between two fingerprints), otherwise the candidate is considered a mismatch.

[0040] In one aspect of the present disclosure, five predefined categories are used for model training and inference and four-stage indexing, and for ten fingerprints of a person, the index key extraction time using a CPU (2.3GHZ) can be about 300ms, and the index key extraction time using a GPU (T4) can be about 64ms. The indexing speed can be about 10 million comparisons per second using a single thread. For the database NIST14, with a reference database of 1.06 million enrolled objects, the embodiment can achieve a penetration rate of about 4.86% with a zero miss rate.

[0041] Therefore, the method according to the present disclosure is an end-to-end method using the original fingerprint image as input and using the index key as output, thereby avoiding complex and error-propagating image preprocessing, orientation\singularity\detail point extraction, etc. The method is robust to noise, transformation, low quality and distortion, and it is easy to construct a scalable ground truth.

Claims

1. A method for indexing a human fingerprint image, comprising the following steps: defining a plurality of categories, each category corresponding to a local pattern of a fingerprint; The ground truth is constructed by allowing the machine learning model to determine the inferred probability for each class for the acquired fingerprint image; Input the original fingerprint image; Assigning, by the machine learning model, a multi-dimensional vector to the raw fingerprint image according to an inferred probability determined for each class based on the ground truth; generating an index key from the multidimensional vector of the original fingerprint; as well as The fingerprint index key is indexed, the indexing comprising a multi-stage check including both an exclusive classification check and a sequential classification check.

2. The method of claim 1, wherein the machine learning model is a convolutional neural network.

3. Method according to claim 1 or 2, wherein the number of defined categories is comprised between 4 and 16.

4. The method according to one of claims 1 to 3, wherein the categories include at least four categories corresponding to the local patterns: bow / tent bow, left winnowing net, right winnowing net, and bucket.

5. The method according to one of claims 1-4, wherein the categories include at least one unknown category.

6. The method according to one of the preceding claims, wherein for a ten fingerprint of a subject, the length of the index key is below 250 bytes.

7. The method according to claim 1, wherein for the implementation of the exclusive classification check, a number of unpaired fingerprints exceeding a threshold is determined for each class in order to reject the number of unpaired fingerprints for each class exceeding the threshold.

8. The method according to one of the preceding claims, wherein the successive classification checks comprise determining a Manhattan distance for each of the vectors.

9. Method according to one of the preceding claims, wherein the successive classification stages comprise determining a cosine distance for each of the vectors.

10. A method according to any one of claims 8 or 9, wherein the successive classification stages comprise determining a fused distance for each of the vectors, the fused distance being a weighted sum of a Manhattan distance and a cosine distance.

11. A fingerprint identification system (101) comprising a database in which fingerprints are indexed according to the method of one of the preceding claims.

Images