A Large-Scale Fingerprint Retrieval Method Based on Point Cloud Registration

Abstract

The present invention discloses a large-scale fingerprint retrieval method based on point cloud registration, which can efficiently and accurately retrieve the input fingerprints to be retrieved and obtain the fingerprint library sample set most likely to have an "identical" matching relationship. The overall solution can be divided into three levels: a preliminary screening level, a fine registration level, and a robust optimization level. The three levels are interdependent, processed serially, screened layer by layer, and none of them can be missing, jointly constituting a complete fingerprint retrieval algorithm. After inputting the fingerprints to be retrieved, the preliminary screening level uses global features to complete a rapid large-scale preliminary screening. The fine registration level uses local registration to achieve further screening and provides the matching relationship between minutiae points for the robust optimization level. The robust optimization level judges the correctness of the minutiae point matching relationship according to geometric consistency test and the maximum clique algorithm, and outputs the retrieval result using the comprehensive matching score.

Description

Technical Field

[0001] The present invention relates to the technical field of fingerprint retrieval, and particularly relates to a large-scale fingerprint retrieval method based on point cloud registration. Background Art

[0002] Due to the rapid development of artificial intelligence technology, biometric recognition has gradually become popular in identity recognition applications. Compared with traditional identity recognition methods such as documents, passwords, access control cards, etc., biometric recognition has the advantages of greater convenience, anti-counterfeiting, and not being easily lost. And fingerprints, due to their uniqueness and persistence, have become one of the most widely used biometric features in the current biometric recognition field. However, the application of fingerprint recognition technology also encounters some problems:

[0003] One is the time efficiency problem of fingerprint retrieval and recognition. From a technical solution perspective, in applications such as criminal investigation fingerprint automatic recognition systems, large-scale fingerprint attendance systems, and access control systems, it is necessary to compare the input query fingerprint with the fingerprints already registered in the fingerprint database one by one until the best-matching registered fingerprint is found or the entire fingerprint database is searched and a conclusion of no corresponding registered fingerprint is given. However, as the application fields of automatic fingerprint recognition technology continue to expand, the scale of fingerprint databases is also constantly expanding. The fingerprint database of Chinese residents' identities has reached the level of hundreds of millions of people. This will result in an overly long time for the process of comparing the fingerprint to be queried with the fingerprints in the fingerprint database one by one, making it inapplicable in practice.

[0004] The second is that the fingerprint images of the same finger entered twice may be different, resulting in fingerprint retrieval failure. The entire fingerprint recognition process can be divided into two links: feature extraction and comparison. The feature extraction link will extract the fingerprint features used for fingerprint recognition, and the most common of which are "minutiae" features. However, during the fingerprint input process, the results of multiple inputs of the same fingerprint may be different. The possible reasons include that the finger skin is relatively soft, and the fingerprint image collected by the pressing method will undergo a certain degree of irregular elastic deformation, causing the fingerprint image to deform. In addition, factors such as temporary peeling and wrinkles of the finger may cause changes such as missing fingerprint textures. These will all affect the extraction of features such as "minutiae" during the fingerprint recognition process, resulting in recognition errors and fingerprint retrieval failures.

[0005] The existing minutiae-based fingerprint retrieval technologies can be mainly divided into two categories according to the basic ideas. One is the learning-based method, which uses neural networks or deep learning to find the similarity between two fingerprints. This mainly relies on the training of a large number of true value data to obtain appropriate network parameters, thereby improving the prediction performance of the network. The other is the feature-based method, which constructs an index of the fingerprint database using the features of the point cloud. For the fingerprint to be retrieved, the feature index is used for comparison to determine a relatively similar fingerprint sequence and obtain the retrieval result.

[0006] Both of the above two methods have their own drawbacks:

[0007] For the learning-based method, a large number of training samples need to be obtained in advance, which is rather laborious. Moreover, the network is not interpretable, and its generalization performance is difficult to control. When there is a large range of noise in the fingerprint minutiae, the matching accuracy drops.

[0008] For the feature-based method, the fingerprint features of minutiae can be extracted offline, which speeds up the retrieval. However, since the feature-based method no longer retains the original minutiae data of the fingerprint and loses a large amount of information, the retrieval accuracy drops significantly.

[0009] Currently, there is no solution that can improve the fingerprint retrieval speed while ensuring the matching accuracy. Summary of the Invention

[0010] In view of this, the present invention provides a large-scale fingerprint retrieval method based on point cloud registration, which can regard fingerprint minutiae as two-dimensional point clouds with directions, and use the point cloud registration method for fingerprint retrieval, realizing the improvement of the fingerprint retrieval speed while ensuring the matching accuracy.

[0011] To achieve the above object, the technical solution of the present invention includes the following steps:

[0012] Step 1: Before online retrieval, the fingerprint samples in the fingerprint database are input into the preliminary screening layer. The preliminary screening layer inputs the fingerprint samples in the fingerprint database into the DAA global feature descriptor extraction module to extract the DAA features of the fingerprint samples, and stores them as sample features in the offline fingerprint feature database. After the retrieval starts, the fingerprint to be retrieved is input into the preliminary screening layer. The DAA global feature descriptor extraction module extracts the DAA features of the current fingerprint to be retrieved, compares them with the sample features in the offline fingerprint feature database, obtains the similarity scores between the DAA features of the current fingerprint to be retrieved and all sample DAA features, sorts the sample features in descending order of similarity scores, and selects the fingerprint samples corresponding to the top N sample features as the output of the preliminary screening layer and sends them to the fine registration layer.

[0013] Step 2: The fine registration layer performs random relative translation on the fingerprint to be retrieved and the selected fingerprint samples, finds the optimal initial state by calculating the registration score after translation, then uses the ICP algorithm for iterative optimization to determine the transformation relationship between the fingerprint to be retrieved and all fingerprint samples, calculates the corresponding objective function value using the registration result of ICP, and determines the matching relationship between minutiae through the nearest neighbor search and matching method; the objective function value and the matching relationship between minutiae are sent to the robust optimization layer.

[0014] Step 3: The robust optimization layer uses geometric consistency checking for the matching relationships between minutiae points, then determines the quality of the matching relationships between minutiae points according to the maximum clique algorithm, and filters out incorrect minutiae point matching pairs; calculates the final comprehensive matching score of the fingerprint sample based on the proportion of correct minutiae point matching pairs and the objective function value given by the fine matching layer; performs sorting using the comprehensive matching score to achieve screening at any filtering level and obtain the final screening result.

[0015] Further, the DAA global feature descriptor extraction module is used to extract distance-angle-area (DAA) features; the distance is used to describe the distance information between any two minutiae points; with the minimum distance and the maximum distance as the boundaries, this range is split into n segments to count all the information of distance D; for each obtained distance, the count value of its corresponding segment is incremented by one, and finally it is normalized to obtain the histogram of distance information.

[0016] The angle is used to describe the included angle between a minutiae point and the line connecting two other minutiae points, and the range is set in [0, π), which is replaced by a cosine function that is monotonic in this range, and the final value range is clamped in (-1, 1]; the final value range is split into n segments, all the included angles are counted and added in the corresponding segments, and finally it is normalized to obtain the histogram of angle information.

[0017] The area is used to describe the area of the triangle constructed by any three minutiae points, which is obtained by preliminary estimation. The area A of the triangle has a certain range. The range of the area is divided into n segments, all the areas are counted and added in the corresponding segments, and it is normalized to obtain the histogram of area information.

[0018] The histograms of distance information, angle information, and area information are combined to obtain the DAA features of each fingerprint sample.

[0019] Further, compare the sample features in the offline fingerprint feature library to obtain the similarity scores between the DAA features of the currently to-be-retrieved fingerprint and all sample DAA features. Specifically:

[0020] The Manhattan distance is used as the index to measure the difference of the DAA feature descriptor, and the calculation formula of the similarity score is as follows:

[0021]

[0022] where DAA1 is the DAA feature of the fingerprint sample, represents the data in the p-th column of the DAA feature of the fingerprint sample, S is the reference maximum score; DAA2 is the DAA feature of the currently to-be-retrieved fingerprint.

[0023] Compare the DAA features of the to-be-retrieved fingerprint with all the fingerprint sample DAA features in the library one by one to calculate the similarity.

[0024] Further, the fine registration layer performs random relative translation on the fingerprint to be retrieved and the selected fingerprint samples, and finds the optimal initial state by calculating the registration score after translation. The specific steps are as follows:

[0025] From the fingerprint samples screened by the preliminary screening layer, select the fingerprint samples with the number of minutiae exceeding the set threshold A as reference fingerprints, and select the fingerprint samples with the number of minutiae lower than the set threshold B as matching fingerprints, where threshold B < threshold A.

[0026] Adopt a position random translation strategy to randomly translate the positions of the minutiae set of the matching fingerprint along the up, down, left, and right four directions within a certain area range with a given step size, calculate the cost of the nearest neighbor minutiae matching pair between the matching fingerprint and the reference fingerprint, and select the position with the lowest cost as the initial relative registration position of the fingerprint.

[0027] Further, in step two, use the ICP algorithm for iterative optimization to determine the transformation relationship between the fingerprint to be retrieved and all fingerprint samples. Use the registration result of ICP to calculate the corresponding objective function value, and determine the matching relationship between minutiae through the nearest neighbor search and matching method. The specific steps are as follows:

[0028] Let the minutiae set after filtering out outliers in the matching fingerprint be The minutiae set after filtering out outliers in the reference fingerprint be Define the centroids of the two sets of points P and Q as p and q respectively:

[0029] Then construct the simplified objective function as:

[0030]

[0031] where R is the rotation matrix and T is the translation matrix.

[0032] Based on the simplified objective function, the ICP solution steps are as follows:

[0033] Step a. Calculate the centroid positions of the two sets of points, and then calculate the centroid-removed coordinates p′ i , q′ i :

[0034] p′ i = p i - p, q′ i = q i - q

[0035] Step b. Calculate the optimized rotation matrix R according to the following optimization problem * :

[0036]

[0037]

[0038] Define the matrix W as:

[0039] Perform SVD decomposition on W to obtain

[0040] W = U∑V T

[0041] where ∑ is a diagonal matrix composed of singular values, and the diagonal elements are arranged from large to small, while U and V are diagonal matrices; when W is full rank, R = UV T .

[0042] Step c. According to the optimized rotation matrix R * Calculate the optimized translation matrix T * : T * = q - R * p.

[0043] If the determinant of R * is negative at this time, then take -R * as the optimal value.

[0044] Use the obtained R * and T * to perform pose transformation on P. If the error is greater than the set error threshold, then perform iteration until the number of iterations reaches the iteration threshold or the error is less than the error threshold, obtain the nearest neighbor matching result of the matching fingerprint and the reference fingerprint minutiae, and finally the matching relationship between minutiae within the threshold range and the objective function value can be output to the robust optimization layer for further retrieval.

[0045] Advantageous effects:

[0046] 1. The present invention provides a large-scale fingerprint retrieval method based on point cloud registration, regarding the minutiae of fingerprints as relatively sparse discrete two-dimensional point clouds, thereby realizing the application of a lot of knowledge based on point clouds in the field of fingerprint retrieval, such as global feature descriptors of point clouds, ICP algorithms, etc., achieving the effect of improving the fingerprint retrieval speed while ensuring the matching accuracy.

[0047] 2. The present invention proposes a brand-new feature descriptor: reducing the dimension of ESF (an advanced global feature descriptor of three-dimensional point clouds) to two dimensions, thereby adapting to the feature description of fingerprint minutiae. By comprehensively considering distance, angle, and area, it has high robustness to the translation, rotation, and scaling of fingerprint minutiae.

[0048] 3. The present invention improves the classical ICP algorithm: aiming at the situation that it is difficult for the classical ICP algorithm to register fingerprints with low overlap and high noise points, through finding the optimal initial iteration value and filtering outlier detail points, the fine registration of fingerprints with large differences is realized, and the extracted matching point pairs are more accurate.

[0049] 4. The present invention has a robust optimization layer: in the case of initially finding matching points, a second screening is carried out. Using geometric consistency and the maximum clique algorithm, the incorrect matching points are filtered out, so that the transformation relationship between fingerprints is more accurate. And scoring is carried out based on this to ensure that correct fingerprint samples are screened out. Description of the Drawings

[0050] Figure 1 is the overall algorithm framework diagram of a large-scale fingerprint retrieval method based on point cloud registration provided by the present invention;

[0051] Figure 2 is the flowchart of the preliminary screening layer;

[0052] Figure 3 is the flowchart of the fine registration layer;

[0053] Figure 4 is the schematic diagram of the input data of the robust optimization layer;

[0054] Figure 5 is the flowchart of the robust optimization layer;

[0055] Figure 6 is the schematic diagram of the matching point pairs;

[0056] Figure 7 is the edge consistency graph. Detailed Embodiment

[0057] The following combines the drawings and gives embodiments to describe the present invention in detail.

[0058] The present invention proposes a hierarchical robust large-scale fingerprint retrieval algorithm based on feature descriptors and geometric consistency, which can efficiently and accurately retrieve the input fingerprints to be retrieved and obtain the fingerprint library sample set most likely to have the "same" matching relationship. The algorithm can be generally divided into three levels: the preliminary screening layer, the fine registration layer and the robust optimization layer. The three levels are interdependent, processed serially, screened layer by layer, and none of them can be missing, jointly forming a complete fingerprint retrieval algorithm. The overall framework of the algorithm is as Figure 1As shown, after inputting the fingerprint to be retrieved, the preliminary screening layer uses global features to complete a quick and large-scale preliminary screening. The fine registration layer uses local registration to achieve further screening and provides the matching relationship between minutiae points for the robust optimization layer. The robust optimization layer determines the correctness of the minutiae point matching relationship based on geometric consistency checking and the maximum clique algorithm, and outputs the retrieval result using the comprehensive matching score.

[0059] Figure 1 Fig. shows the overall algorithm framework of a large-scale fingerprint retrieval method based on point cloud registration provided by the present invention, which includes the following three levels in the order of data flow:

[0060] Preliminary screening layer: Before online retrieval, the fingerprint samples in the fingerprint database are input into the DAA global feature descriptor extraction module to extract the DAA features of the fingerprint samples. The DAA feature refers to the distance-angle-area feature, thereby obtaining an offline fingerprint feature database with a simpler storage format. After starting the retrieval, the fingerprint to be retrieved is input into the DAA global feature descriptor extraction module to obtain the features of the current fingerprint to be retrieved. These features are used to compare with the sample features in the offline fingerprint feature database, and then the similarity scores of all samples in the database are obtained through simple numerical operations. Finally, after sorting, the fingerprint sample IDs with higher scores are sent to the fine matching layer. The fingerprint samples themselves are stored in the form of minutiae points.

[0061] Fine matching layer: Input the screening result of the preliminary screening layer and perform improved ICP matching. First, randomly perform relative translation on the fingerprint to be retrieved and the fingerprint samples preliminarily screened in the fingerprint database, and find the optimal initial state by calculating the registration score after translation. This is mainly for the case where the translation amount of samples with the same matching relationship is large. Then use the ICP algorithm for iterative optimization to further determine the transformation relationship between the fingerprint to be retrieved and all fingerprint samples, which is used to handle the case where the rotation amount is large. Using the registration result of ICP, calculate the corresponding objective function value, determine the matching relationship between minutiae points through nearest neighbor search and matching, and at the same time the objective function value is used to complete further screening.

[0062] Robust optimization layer: Use geometric consistency checking for the input minutiae point matching relationship, and then judge the quality of the minutiae point matching relationship according to the maximum clique algorithm, and filter out the wrong minutiae point matching pairs (the majority). According to the proportion of correct matching point pairs and the objective function value given by the fine matching layer, calculate the final comprehensive matching score of the sample. Use this score for sorting to achieve screening at any filtering level and obtain the final screening result. In fact, the optimal result obtained by screening is basically the fingerprint sample with the same matching relationship.

[0063] The first layer - preliminary screening layer

[0064] Figure 2 The algorithm flow of the preliminary screening layer in the present invention is shown. The function of this layer is to utilize the advantage of fast query of the offline library to preliminarily filter out those fingerprint samples that are least likely to have the "same" relationship, so as to improve the time efficiency as much as possible.

[0065] Two-dimensional point cloud global feature descriptor DAA

[0066] To efficiently implement fingerprint retrieval, it is first necessary to preliminarily screen the fingerprints in the database using global features. Fingerprint minutiae can be regarded as a two-dimensional point cloud according to coordinates (the minutiae directions will vary due to rotation, so this link does not consider it). The feature descriptors of the point cloud can be roughly divided into two categories: global feature descriptors and local feature descriptors. Among them, local feature descriptors are generally used for online matching of point clouds, suitable for online matching of fingerprints, and occupy a large storage space; while global feature descriptors represent the global information of fingerprints, are convenient to store, and are suitable for offline matching of fingerprints. Therefore, global feature descriptors are selected for this layer.

[0067] There are many global feature descriptors that can be used for point clouds, such as VFH, CVFH, ESF, GFPFH, etc. However, these algorithms are all based on three-dimensional point clouds and do not focus on feature descriptors for two-dimensional point clouds. Based on ESF, this layer performs dimensionality reduction processing on it and designs a global feature descriptor DAA (Distance-Angle-Area) suitable for two-dimensional point clouds to express the features of all minutiae and statistically presents them in the form of a combined histogram of 3×64 columns. Since the average number of minutiae extracted from each fingerprint image is about 20 - 40, which is relatively small, DAA will traverse all minutiae to ensure the accuracy of the extracted features to the greatest extent. The specific meaning of DAA in the present invention is as follows:

[0068] a. Distance

[0069] The distance describes the distance information between any two minutiae. Taking the minimum distance and the maximum distance as boundaries, this range is split into 64 segments to statistically count all the distance D information. For each obtained distance, the count value of its corresponding segment is incremented by one, and finally it is normalized to obtain the histogram of distance information.

[0070] b. Angle

[0071] The angle is defined as the included angle between one minutia and the line connecting two other minutiae, and the range is set in [0, π). For the convenience of statistics, the monotonic cosine function in this range can be used instead. Therefore, the final value range is clamped in (-1, 1]. Similarly, it is split into 64 segments, with a resolution of 0.032 for each segment. All included angles are statistically counted and added in the corresponding segments, and finally normalized.

[0072] c. Area

[0073] Due to the stable structure of triangles, they are often used in fingerprint retrieval tasks. Given this, DAA also uses the area of triangles constructed by any three minutiae as feature information. It is preliminarily estimated that the area of triangles has a certain range. It is divided into 64 segments and statistically analyzed using the same method as above.

[0074] The above three pieces of feature information all have strong translational invariance and rotational invariance, and are very suitable for the feature extraction of fingerprint minutiae data. Moreover, angle A 1 has robustness to length, and A 2 has robustness to length and angle. It can still be well processed even when there are elastic deformations in fingerprints. In addition, the form of histogram is used when statistically analyzing information. This can not only maintain the robustness of feature information, but also more comprehensively reflect the information of fingerprint minutiae compared with the method using index values. Therefore, DAA has great potential in extracting fingerprint minutiae features.

[0075] Combining the histograms obtained from the above three features can obtain the DAA features of each fingerprint sample.

[0076] Construction of Offline Fingerprint Feature Database

[0077] To achieve fast fingerprint retrieval, global features can be used to initially filter out those least likely fingerprint samples. The offline fingerprint feature database can well meet this requirement. The global feature sub-DAA is used to construct all fingerprint samples in the database. Instead of storing the sample point information of each fingerprint, the combined histogram of DAA is used for representation.

[0078] By using DAA as a feature descriptor to preprocess each fingerprint, their features are obtained. Combining these features yields the offline fingerprint feature database. The preprocessing steps are completely offline and are a one-time operation, which may take more time. However, it is precisely the relatively high time cost of this link that reduces the difficulty of the initial screening of online fingerprints and can almost be completed in real time.

[0079] Initial Screening of Online Fingerprint Features

[0080] After completing the database construction, it can be used as a comparison baseline to find the similar set of the fingerprint to be retrieved. This link is completed online. For the fingerprint to be retrieved sent in, first use DAA to extract its features. The extracted features are compared one by one with the offline feature database, which involves the problem of similarity measurement.

[0081] The present invention uses the Manhattan Distance as an index to measure the difference of DAA feature descriptors. Therefore, the similarity calculation formula is as follows:

[0082]

[0083] where represents the data of the p-th column of the fingerprint DAA with ID = 1, and S is the reference maximum score. In this way, the fingerprint to be retrieved is compared one by one with all the fingerprint features in the library, and the similarity between them is calculated. The higher the value, the greater the probability of belonging to the same fingerprint. All the scores are sorted, and the TopN (taking the top N) algorithm is used to screen them to filter out those fingerprints that are least likely to match.

[0084] The second layer - fine registration layer

[0085] Figure 3 shows the algorithm flow of the fine registration layer. The function of this layer is to perform the nearest neighbor matching of fingerprint minutiae based on the improved ICP according to the screening results of the preliminary screening layer, determine the matching relationship between minutiae and the objective function value to complete further screening, and output it to the next layer for retrieval.

[0086] Optimization of the initial relative registration position of fingerprints

[0087] After inputting the screened and matched fingerprint samples processed by the preliminary screening layer and the fingerprint to be retrieved, it is necessary to register the fingerprint to be retrieved with the screened and matched fingerprint samples, find the matching relationship between fingerprint minutiae, and then determine whether they are the "same" fingerprint. However, since the number of minutiae extracted from each fingerprint is different, and the number of fingerprint minutiae with the "same" relationship is also different. Therefore, determining the registration direction is beneficial to improving the registration accuracy. This layer uses the fingerprint with more minutiae as the reference fingerprint and the fingerprint with fewer minutiae as the matching fingerprint for registration, which can reduce the registration time and improve the registration accuracy.

[0088] At the same time, due to the rotation and translation relationships between fingerprint images, and the rotation and translation also exist in the corresponding extracted minutiae features. If the initial relative registration position error of fingerprint minutiae features is large, it will additionally increase the registration time and algorithm complexity. Therefore, optimizing the initial relative registration position of fingerprints to make it have a small cost is crucial. This layer adopts a position random translation strategy, randomly translates the position of the matching fingerprint minutiae set in the up, down, left, and right four directions within a certain area range with a given step size, calculates the cost of the nearest neighbor minutiae matching pair between the matching fingerprint and the reference fingerprint, and selects the position with the lowest cost as the initial relative registration position of the fingerprint. The specific steps are as follows:

[0089] First, randomly translate the positions of the matching fingerprint minutiae set, and calculate the least - square distance from each matching fingerprint minutia after translation to its nearest - neighbor reference fingerprint minutia, that is

[0090]

[0091] where \(M\) is the matching fingerprint minutiae set, \(Q\) is the reference fingerprint minutiae set, \(m\) i is the \(i\) - th matching fingerprint minutia, \(q\) i is the point in the reference fingerprint minutiae set that is closest to \(m\) in the i Euclidean distance in the plane, \(T\) ξ is the random translation matrix, \(n\) M is the number of matching fingerprint minutiae, \(n\) Q is the number of reference fingerprint minutiae, \(D(M,Q,T\) ξ ) is the least - square distance from each matching fingerprint minutia to its nearest reference fingerprint minutia after the matching fingerprint minutiae set is randomly translated by \(T\) ξ .

[0092] Meanwhile, according to the Euclidean distance between the nearest - neighbor minutiae, count the number of minutia pairs within a certain distance range. Take the number of minutia pairs within a certain distance range and the least - square distance of the nearest - neighbor minutiae as the cost of the nearest - neighbor minutia matching pair. The fewer the number of minutia pairs within a certain distance range, the lower the cost. If the number of minutia pairs within a certain distance range is the same, the smaller the least - square distance of the nearest - neighbor minutiae, the lower the cost.

[0093] Within a certain regional range, make \(\xi\) random translations of the matching fingerprint minutiae set with a given step size and without repetition, and select the translation matrix with the lowest cost as the translation matrix \(T\) for the initial registration position of the matching fingerprint init , and the initial registration position of the reference fingerprint remains unchanged:

[0094]

[0095] Before the optimization of the registration position, the relationships of each minutia may be chaotic, but after the optimization of the registration position, the matching relationship between fingerprint minutiae can be enhanced, and the cost of the nearest - neighbor minutia matching pair between the matching fingerprint and the reference fingerprint can be effectively reduced.

[0096] Nearest - neighbor search and matching of fingerprint minutiae

[0097] The above steps roughly optimize the translation relationship between fingerprint images, while the optimization of the rotation and fine translation relationships between fingerprint images will be completed in this step. First, the above optimization of the initial relative registration position of fingerprints only considers most of the minutiae points in the matching fingerprints. There may also be outlier minutiae points in the matching fingerprints. Filtering out the outlier minutiae points in the matching fingerprints is beneficial to the correct matching of subsequent minutiae points. The outlier minutiae point filtering strategy adopted is: delete the minutiae points in the matching fingerprints where the distance between the nearest neighbor minutiae point pairs in the optimized initial relative registration position of fingerprints is greater than a certain threshold, and only retain the matching fingerprint minutiae points within the threshold range. At the same time, before filtering out the outlier minutiae points, if the number of minutiae point pairs within a certain distance range is too small or the least squares distance of the nearest neighbor minutiae points is too large, then filter out the fingerprint to be matched and do not participate in the subsequent outlier minutiae point filtering, nearest neighbor matching and search of fingerprint minutiae points, reducing the time complexity of this layer.

[0098] Next, the two-dimensional ICP (Iterative Closest Point) method is used to implement the nearest neighbor matching and search of the minutiae points of the reference fingerprint and the matching fingerprint. The steps are as follows:

[0099] Let the set of minutiae points after filtering out the outlier minutiae points in the matching fingerprint be Then the objective function is constructed as

[0100]

[0101] where R is the rotation matrix and T is the translation matrix, jointly constituting an Euclidean transformation.

[0102] Define the centroids of the two sets of points P and Q as:

[0103]

[0104] Subsequently, the following processing is performed on Equation (4):

[0105]

[0106] Since the sum of (q i - q - R(p i - p)) is zero, the objective function can be simplified to

[0107]

[0108] Therefore, based on the simplified objective function of Equation (7), the ICP solution steps are:

[0109] a. Calculate the centroid positions of the two sets of points, and then calculate the centroid-removed coordinates of each point:

[0110] p′ i = p i-p, q' i = q i -q (8)

[0111] b. Calculate the rotation matrix according to the following optimization problem:

[0112]

[0113] c. Calculate T according to R in the above formula:

[0114] T * = q - Rp (10)

[0115] For the solution of the rotation matrix R, expanding formula (9) gives

[0116]

[0117] Among them, has nothing to do with R, R T If R is 0, the actually optimized objective function is

[0118]

[0119] Define the matrix W as:

[0120]

[0121] Perform SVD decomposition on W to obtain

[0122] W = U∑V T (14)

[0123] Among them, ∑ is a diagonal matrix composed of singular values, and the diagonal elements are arranged from large to small, while U and V are diagonal matrices. When W is full rank, R is

[0124] R = UV T (15)

[0125] After solving R, T can be solved according to formula (10). If the determinant of R is negative at this time, take -R as the optimal value.

[0126] Use the obtained R and T to perform pose transformation on P. If the error is greater than the threshold, perform iteration until the number of iterations reaches the threshold or the error is less than the threshold, obtain the nearest neighbor matching result of the matching fingerprint and the reference fingerprint minutiae, and finally the matching relationship between minutiae within the threshold range and the objective function value can be output to the robust optimization layer for further retrieval.

[0127] The third layer: Robust optimization layer

[0128] Figure 4Shows a schematic diagram of the input data of the robust optimization layer. The input of the robust optimization layer is the output of the nearest neighbor matching of the previous layer (fine registration layer), that is, the retrieval results of each ID fingerprint after being filtered by the previous layer. Each retrieval result of a certain ID fingerprint consists of several matching point pairs of minutiae. The schematic diagram of the input of this layer is shown as follows. The function of this layer is to further filter the fingerprint sample retrieval results output by the fine registration layer and ensure high retrieval accuracy, so as to ensure a high penetration rate of fingerprint retrieval at different filtering levels. The flow chart of this layer is as Figure 5 shown.

[0129] Geometric consistency

[0130] The minimum data unit input to this layer is several matching point pairs of minutiae of two fingerprints. This layer will start from the geometric consistency of the matching point pairs of minutiae to judge the probability of correct matching of two fingerprints. Then, sort several probability values according to their magnitudes, and select the fingerprints corresponding to the TopN probability values according to the filtering level requirements as the final output results.

[0131] Suppose the matching pairs of the minutiae corresponding to u1, u2, and u3 are correct, while the matching pair of the minutiae corresponding to u4 is incorrect. Then the following relationships can be given, where |a'-c'| represents the length of the line segment connecting points a' and c', and the other expressions have similar meanings.

[0132] |a'-c'|≈|a-c| (16)

[0133] |b'-c'|≈|b-c| (17)

[0134] |a'-b'|≈|a-b| (18)

[0135] This is because for the line segments connecting any two points in a two-dimensional point set, any rotation and translation of the point set in the plane will not change the lengths of these line segments. Therefore, if there are correct matching point pairs between two point sets, these matching point pairs must satisfy geometric relationships similar to those in equations (16) to (18). For u4, although the probability that the geometric relationship represented by equation (19) holds for incorrect matching point pairs is not zero, the probability value is very small.

[0136] |d'-c'|≈|d-c| (19)

[0137] Generally speaking, taking u1 and u2 as examples, if the matching point pairs corresponding to u1 and u2 satisfy the geometric relationship in equation (16), it is said that there is "geometric consistency" between u1 and u2; conversely, since the matching point pairs corresponding to u1 and u4 do not satisfy the geometric relationship similar to that in equation (16), it is said that there is no "geometric consistency" between u1 and u4.

[0138] The above-mentioned "geometric consistency" also applies to the direction of minutiae points. This is because the direction of minutiae points is related to the texture of finger fingerprints. Even when the same finger is used to input fingerprints twice with translation and rotation, the direction of the same minutiae point extracted twice changes little. Assume that the direction of minutiae point a is defined as θ a , and the directions of other minutiae points are defined similarly. The following geometric relationships between the directions of minutiae points can be obtained:

[0139] |θ a' -θ c' |≈|θ a -θ c | (20)

[0140] |θ b' -θ c' |≈|θ b -θ c | (21)

[0141] |θ a' -θ b' |≈|θ a -θ b | (22)

[0142] Considering that there may be dust on the finger and the finger may be deformed easily when inputting fingerprints due to its softness, etc., resulting in the direction or position of a small number of minutiae points being different from the actual direction or actual position of the minutiae points in the fingerprint. Therefore, it is not enough to only consider whether the matching pairs of two minutiae points have "geometric consistency", but it is necessary to comprehensively consider the "geometric consistency" among multiple pairs of matching points.

[0143] The "geometric consistency" among multiple pairs of matching points is defined as follows. Taking u1, u2, and u3 as examples, when the matching pairs corresponding to u1, u2, and u3 pairwise all have the relationship in Equation (16), it is said that u1, u2, and u3 satisfy "geometric consistency". The "geometric consistency" among multiple pairs of matching points plays an important role in this layer. For two fingerprints with the "same" relationship, if there are no large differences in fingerprints and minutiae points between them, then more correct matching pairs can be constructed between the minutiae points of the two in the fine matching layer, and there should be a relatively large number of matching pairs that have the relationship of "geometric consistency" with each other; conversely, if two fingerprints do not have the "same" relationship, although there is a small probability of constructing matching pairs between minutiae points that satisfy the relationship in Equation (16) in the fine matching layer, the number of matching pairs with the relationship of "geometric consistency" with each other will be very small. Thus, it can better distinguish whether two fingerprints have the "same" relationship.

[0144] Maximum clique algorithm

[0145] Suppose that given the matching point pairs of all minutiae of two fingerprints, it is necessary to solve for the maximum number of multiple matching point pairs that satisfy the "geometric consistency" relationship among multiple matching point pairs. It is illustrated in the form of a schematic diagram represented by a small number of points and edges, as Figure 6 shown. Figure 6 In Figure 6 , the circles represent the minutiae in the two fingerprints, denoted by a, b, c and a', b', c', d' respectively, and θ a represents the direction of minutia a, and the mathematical representation of the directions of other minutiae is similar; the line segments are the connections after the matching of minutiae, denoted by u1, u2, u3, u4, u5 respectively, and are called edges. Whether there is a "geometric consistency" relationship between each edge is judged by the following formula (23). For example, when a', b' and a, b satisfy formula 23, it is said that the edges u1, u2 have a "geometric consistency" relationship. In formula (23), and σ are non-zero thresholds, and the meaning of their non-zero is: to ensure that when there are differences between two fingerprints pressed by the same finger, the correctly matched fingerprint minutia pairs can still be recognized.

[0146]

[0147] It is obtained through the discrimination of formula (23) that Figure 6 the matching pairs represented by the edges u1, u2, u4 respectively have a "geometric consistency" relationship with each other, and the matching pairs represented by the edges u3, u5 respectively have a "geometric consistency" relationship, that is, there are "geometric consistency" relationships among different multiple matching pairs in the above figure. Such "geometric consistency" relationships among different multiple matching pairs cannot hold simultaneously, so there must be wrongly matched minutia pairs.

[0148] Therefore, a "consistency graph" related to the edges u1, u2, u3, u4, u5 can be constructed to facilitate the subsequent solution by the maximum clique algorithm, as Figure 7 shown. Among them, the blue circles u1, u2, u3, u4, u5 correspond to Figure 6 the 5 edges in Figure 6 , and the black line segments indicate that there is a "geometric consistency" relationship between two edges. For example, there are line segments connected between u1 and u2, u2 and u4, u1 and u4 respectively, indicating that there is a "geometric consistency" relationship between these edges pairwise. If there is no line segment connected between two of the blue circles u1, u2, u3, u4, u5, it means that there is no "geometric consistency" relationship between these two edges. For example, there is no line segment connected between u2 and u3, u3 and u4. Therefore, Figure 7 there is Figure 6 a corresponding relationship related to the "geometric consistency". Figure 7 is the consistency graph of the edges. Figure 7There are multiple pairs of edges with "geometric consistency" relationships, which form two subsets of edges, namely the two subsets of u1, u2, u4 and u3, u5. And it is impossible for the "geometric consistency" relationships of the two subsets to both hold, otherwise there should be only one subset. Therefore, it is necessary to find the subset with the most edges that satisfies the "geometric consistency" relationship through an algorithm and remove the incorrect subset with fewer edges.

[0149] First, assign weights to each line segment to make it a weighted undirected graph, then construct the adjacency matrix of the weighted undirected graph, and then use the maximum clique algorithm to solve this matrix to determine whether each edge is correct or not.

[0150] Assign a weight of 1 to the line segments to form a 5×5 matrix M as shown in Equation (24). The rows and columns of the matrix represent u1, u2, u4, u3, u5 in sequence. When there is a line segment connection between edges ui and uj, the corresponding matrix element M(i,j) = 1 and M(j,i) = 1; conversely, when there is no line segment connection between edges ui and uj, the corresponding matrix element M(i,j) = 0 and M(j,i) = 0. The diagonal elements of the matrix M(i,i) = 1. It can be seen that this matrix is a symmetric matrix.

[0151]

[0152] So far, the minutiae point diagrams with matching relationships in the two fingerprints have been transformed into Figure 7 and the adjacency matrix in Equation (24). Next, the subset of the most consistent edges can be found by solving the problem represented by Equation (25).

[0153]

[0154] Among them, the optimization variable u is a binary vector of length n (the value of each element can only be 0 or 1). When the edge corresponding to an element in u has consistency, this element is 1, otherwise it is 0. Since u is binary and is subject to the constraint that when M(i,j) = 0, then uiuj = 0, the consistent edges in the final output should not have ui and uj appear simultaneously.

[0155] Since the elements in the M matrix are 1 or 0 and it is a symmetric matrix, the problem represented by Equation (25) can be transformed into the classical maximum clique problem as shown in Equation (26):

[0156]

[0157] In the example, the final output will be u = [1,1,0,1,0] T , indicating that the solution is Figure 7 the subset of edges u1, u2, u4 with consistency in it, while u3, u5 are discarded. This result also means Figure 6Among them, a and a', b and b', c and c' are correctly matched minutiae point pairs, while b and c', c and d' are not correctly matched minutiae point pairs. Therefore, it can be calculated that Figure 6 The proportion of correctly matched minutiae point pairs among all matched minutiae point pairs in

[0158]

[0159] Scoring Function and TopN Sorting

[0160] Next, score and sort the fingerprint retrieval results of the same fingerprint ID in the fingerprint database, so as to output the corresponding retrieval results according to the sorting results and different filtering levels required by the competition questions, and calculate the fingerprint penetration rate under different filtering levels. The objective function value of the nearest neighbor iteration in each fingerprint retrieval result is output at the fine matching layer.

[0161] Define the scoring function score(i), and its scoring calculation for the fingerprint retrieval result with ID i is as follows:

[0162]

[0163] The above formula indicates that the smaller the average cost of each matched minutiae point pair and the larger the proportion of correctly matched minutiae point pairs, the smaller the score, and the greater the possibility that the fingerprint retrieval result is a "same finger" result. When performing TopN sorting, sort the scores of each fingerprint retrieval result from largest to smallest. According to the requirements of the filtering level, select a certain number of fingerprint retrieval results from the end of the sequence as the output.

[0164] In summary, the above is only a preferred embodiment of the present invention and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A large-scale fingerprint retrieval method based on point cloud registration, characterized in that, It includes the following steps: Step 1: Before online retrieval, the fingerprint samples in the fingerprint database are input into the preliminary screening layer. The preliminary screening layer inputs the fingerprint samples in the fingerprint database into the DAA global feature descriptor extraction module to extract the DAA features of the fingerprint samples, which are stored as sample features in the offline fingerprint feature database. After the retrieval starts, the fingerprint to be retrieved is input into the preliminary screening layer. The DAA global feature descriptor extraction module extracts the DAA features of the current fingerprint to be retrieved, compares them with the sample features in the offline fingerprint feature database, obtains the similarity scores between the DAA features of the current fingerprint to be retrieved and all sample DAA features, sorts the sample features in descending order of similarity scores, and selects the fingerprint samples corresponding to the top N sample features as the output of the preliminary screening layer and sends them to the fine registration layer; Step 2: The fine registration layer performs random relative translation on the fingerprint to be retrieved and the selected fingerprint samples, finds the optimal initial state by calculating the registration score after translation, and then uses the ICP algorithm for iterative optimization to determine the transformation relationship between the fingerprint to be retrieved and all fingerprint samples. Using the registration result of ICP, the corresponding objective function value is calculated, and the matching relationship between minutiae points is determined through the nearest neighbor search and matching method; The objective function value and the matching relationship between minutiae points are sent to the robust optimization layer; Step 3: The robust optimization layer uses geometric consistency checking for the matching relationship between minutiae points, and then judges the quality of the matching relationship between minutiae points according to the maximum clique algorithm, filtering out incorrect minutiae point matching pairs; Calculate the final comprehensive matching score of the fingerprint sample according to the proportion of correct minutiae point matching pairs and the objective function value given by the fine matching layer; Use the comprehensive matching score for sorting to achieve screening at any filtering level and obtain the final screening result.

2. The large-scale fingerprint retrieval method based on point cloud registration according to claim 1, characterized in that, The DAA global feature descriptor extraction module is used to extract distance-angle-area DAA features; The distance is used to describe the distance information between any two minutiae points; with the minimum distance and the maximum distance as the boundaries, this range is split into n segments to count all the distance D information; for each obtained distance, the count value of its corresponding segment is incremented by one, and finally it is normalized to obtain the histogram of distance information; The angle is used to describe the included angle between a minutiae point and the line connecting it with two other minutiae points, and the range is set in [0,π), which is replaced by a cosine function that is monotonic in this range, and the final value range is clamped in (-1,1]; The final value range is split into n segments, all included angles are counted and added in the corresponding segments, and finally it is normalized to obtain the histogram of angle information; The area is used to describe the area of the triangle formed by any three minutiae points, which is estimated preliminarily. The area A of the triangle has a certain range. The range of the area is divided into n segments, all areas are counted and added in the corresponding segments, and it is normalized to obtain the histogram of area information; Combine the histograms of distance information, angle information, and area information to obtain the DAA features of each fingerprint sample.

3. A large-scale fingerprint retrieval method based on point cloud registration according to claim 1 or 2, characterized in that, Compare the sample features in the offline fingerprint feature library to obtain the similarity scores between the DAA features of the currently to-be-retrieved fingerprint and all sample DAA features. Specifically: Use the Manhattan distance as an index to measure the difference of DAA feature descriptors. The calculation formula for the similarity score is as follows: where DAA1 is the DAA feature of the fingerprint sample, representing the data of the p-th column of the DAA feature of the fingerprint sample, S is the reference maximum score; DAA2 is the DAA feature of the currently to-be-retrieved fingerprint; Compare the DAA features of the to-be-retrieved fingerprint with all fingerprint samples in the library one by one to calculate the similarity.

4. A large-scale fingerprint retrieval method based on point cloud registration according to claim 1 or 2, characterized in that, The fine registration layer performs random relative translation on the to-be-retrieved fingerprint and the selected fingerprint samples, and finds the optimal initial state by calculating the registration score after translation. The specific steps are as follows: From the fingerprint samples selected by the preliminary screening layer, select the fingerprint samples with the number of minutiae exceeding the set threshold A as reference fingerprints, and select the fingerprint samples with the number of minutiae lower than the set threshold B as matching fingerprints; threshold B < threshold A; Adopt a position random translation strategy to randomly translate the positions of the matching fingerprint minutia set in the up, down, left, and right four directions within a certain area range with a given step size, calculate the cost of the nearest neighbor minutia matching pair between the matching fingerprint and the reference fingerprint, and select the position with the lowest cost as the initial relative registration position of the fingerprint.

5. The large-scale fingerprint retrieval method based on point cloud registration according to claim 4, characterized in that, In the second step, use the ICP algorithm for iterative optimization to determine the transformation relationship between the to-be-retrieved fingerprint and all fingerprint samples. Use the registration result of ICP to calculate the corresponding objective function value, and determine the matching relationship between minutiae through the nearest neighbor search and matching method. The specific steps are as follows: Let the set of minutiae points after filtering outlier minutiae points in the matching fingerprint be Let the set of minutiae points after filtering outlier minutiae points in the reference fingerprint be Define the centroids of two sets of points P and Q as p and q respectively: Then construct a simplified objective function as: where R is the rotation matrix and T is the translation matrix; Based on the simplified objective function, the ICP solution steps are: Step a. Calculate the centroid positions of two sets of points, and then calculate the centroid - removed coordinates p′ i , q′ i : p′ i = p i - p, q′ i = q i - q Step b. Calculate the optimized rotation matrix R according to the following optimization problem * :[[]]END]] Define the matrix W as: Perform SVD decomposition on W to obtain W = U∑V T Among them, ∑ is a diagonal matrix composed of singular values, and the diagonal elements are arranged from large to small, while U and V are diagonal matrices; when W is full rank, R is R = UV T Step c. Calculate the optimized translation matrix T according to the optimized rotation matrix R * : T * * = q - R * p;​ If the determinant of R * is negative at this time, then take -R * as the optimal value; Using the obtained R * and T * Perform pose transformation on P. If the error is greater than the set error threshold, perform iteration until the number of iterations reaches the iteration threshold or the error is less than the error threshold, obtain the nearest neighbor matching result of the matching fingerprint and the reference fingerprint minutiae, and finally output the matching relationship between minutiae within the threshold range and the objective function value to the robust optimization layer for further retrieval.

Images