A method and system for fingerprint two-dimensional pose estimation based on dense estimation

Abstract

The application discloses a kind of based on dense estimation's fingerprint two-dimensional attitude estimation method and system, the method includes: given the fingerprint image collected on various fingerprint collection equipment, the 2D attitude of fingerprint is predicted, including fingerprint center position and orientation, this attitude information can be used in the fingerprint positioning in image, and can assist subsequent fingerprint identification task, improve the efficiency and accuracy of identification algorithm.2D attitude estimation problem of fingerprint is decomposed to single pixel, after fingerprint image is passed through specific dense attitude estimation model, the estimation result on single pixel is output, finally, the result of dense estimation is integrated in fingerprint effective area, and the 2D fingerprint attitude estimation is obtained.The estimation system of fingerprint 2D attitude of the application can effectively improve the generalization ability of fingerprint 2D attitude estimation algorithm under multiple image modalities by the way of dense estimation.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of fingerprint recognition, and in particular to a fingerprint two-dimensional pose estimation method and system based on dense estimation. BACKGROUND

[0002] Due to the characteristics of easy acquisition and invariability, fingerprint recognition has been an important identity recognition technology, and has been widely applied in company attendance, entry and exit management, secure payment and criminal investigation. The existing fingerprint data is generally two-dimensional fingerprints collected by contact, such as rolling fingerprints, flat fingerprints, and on-site fingerprints with complex background noise commonly used in criminal investigation. By extracting features from the collected fingerprint data, it can be used for fingerprint retrieval, fingerprint matching and other subsequent fingerprint recognition tasks, so as to realize identity recognition. Before extracting fingerprint features, if the fingerprint can be aligned first, the use efficiency of fingerprint features in subsequent fingerprint recognition tasks can be effectively improved, such as quickly excluding feature matching calculation in different positions and quickly rejecting false matching. For on-site fingerprint data with complex background noise, due to the limited effective fingerprint area that can be detected, through fingerprint alignment operation, the search space of feature matching can be effectively reduced, and the efficiency and accuracy of the recognition algorithm can be improved.

[0003] There are many related algorithms for fingerprint 2D pose estimation based on fingerprint images, and the estimation objects include the center position and overall orientation of the fingerprint. These methods generally have the following limitations and deficiencies: some algorithms estimate the 2D pose of the fingerprint based on the entire fingerprint image, and can only cope with the case where the fingerprint area is relatively complete and ideal. When the background noise is complex or the fingerprint area is incomplete, the performance cannot be good. Some algorithms are based on hand-designed features, and estimate the center position of the fingerprint by voting point by point or image block by image block. The generalization ability and computational efficiency of the algorithm are limited, and the estimation of the fingerprint orientation is still based on the entire image, and the algorithm performance is limited. Therefore, a better fingerprint pose estimation method is needed to overcome the above defects. SUMMARY

[0004] The present application aims to at least solve one of the technical problems in the related art to some extent.

[0005] To this end, the present application proposes a fingerprint two-dimensional pose estimation method based on dense estimation. The method of the present application is divided into two stages in the whole, which are dense pose estimation based on fingerprint image and fingerprint overall pose integration based on dense pose estimation. In the first stage, the 2D pose estimation task of the fingerprint is assigned to a single pixel point, and the dense estimation model pre-trained in the offline period is used to perform pixel-level dense pose estimation based on the fingerprint image. In the second stage, based on the aforementioned pixel-level dense estimation map, the overall 2D pose estimation of the fingerprint is integrated. The first stage is the basic step of the present application, which changes the 2D pose estimation task of the fingerprint into a dense pose estimation task on the pixel point, and improves the generalization ability of the algorithm when facing complex background noise and effective area defects; the second stage integrates the pixel-level dense estimation result into the final 2D pose estimation result. The dense pose estimation model proposed in the present application needs to be trained in the offline period. The algorithm needs a large number of fingerprint images and corresponding fingerprint 2D pose true values, and uses the method of supervised learning to optimize the parameters of the dense estimation model.

[0006] Another object of the present application is to propose a fingerprint two-dimensional pose estimation system based on dense estimation.

[0007] To achieve the above object, the present application proposes a fingerprint two-dimensional pose estimation method based on dense estimation, which comprises:

[0008] Obtaining a fingerprint image to be predicted;

[0009] Inputting the fingerprint image to be predicted into a trained dense estimation model to perform pixel-level dense estimation and obtain a dense estimation result;

[0010] Integrating fingerprint center estimation results and fingerprint orientation estimation results based on the dense estimation result to obtain a fingerprint pose estimation result of the fingerprint image to be predicted according to the integration result.

[0011] In addition, the fingerprint two-dimensional pose estimation method based on dense estimation according to the above embodiments of the present application can also have the following additional technical features:

[0012] Further, training the dense estimation model comprises:

[0013] Obtaining a sample fingerprint image and labeling the sample fingerprint image to generate a training data set;

[0014] Assigning a fingerprint pose estimation task to a single pixel point of the image of the training data set using a preset pose decomposition scheme to obtain a pose decomposition result;

[0015] Training the dense estimation model using the pose decomposition result to generate a trained dense estimation model.

[0016] Further, the pose decomposition scheme further comprises:

[0017] decomposing the image of the training data set into a plurality of image blocks to estimate on the plurality of image blocks to obtain the pose decomposition result.

[0018] Further, the dense estimation model comprises a deep convolutional network model based on supervised learning; and training the dense estimation model based on the pose decomposition result to generate a trained dense estimation model comprises:

[0019] obtaining a large number of sample fingerprint images and corresponding fingerprint pose ground truths labeled;

[0020] training and optimizing the deep convolutional network model based on the large number of sample fingerprint images and the fingerprint pose ground truths to obtain optimized network model parameters based on a loss calculation result of a model loss function.

[0021] Further, the integrating the fingerprint center estimation result and the fingerprint orientation estimation result based on the dense estimation result to obtain the fingerprint pose estimation result of the to-be-predicted fingerprint image comprises:

[0022] obtaining the fingerprint center estimation result and the fingerprint orientation estimation result through fingerprint center calculation and fingerprint orientation calculation respectively based on the dense estimation result;

[0023] integrating the fingerprint center estimation result and the fingerprint orientation estimation result to obtain the fingerprint pose estimation result of the to-be-predicted fingerprint image based on a fingerprint overall pose integration result.

[0024] To achieve the above object, another aspect of the present application provides a fingerprint two-dimensional pose estimation system based on dense estimation, comprising:

[0025] an image acquisition module configured to acquire a to-be-predicted fingerprint image;

[0026] a dense estimation module configured to input the to-be-predicted fingerprint image into a trained dense estimation model to perform pixel point level dense estimation to obtain a dense estimation result;

[0027] a pose estimation module configured to integrate a fingerprint center estimation result and a fingerprint orientation estimation result based on the dense estimation result to obtain a fingerprint pose estimation result of the to-be-predicted fingerprint image according to an integration result.

[0028] The fingerprint two-dimensional posture estimation method and system based on dense estimation can automatically estimate the fingerprint 2D posture in the fingerprint image, and can effectively improve the efficiency and accuracy of the fingerprint recognition system.

[0029] Additional aspects and advantages of the present application will be described in the description that follows, and will become apparent from the description, or can be learned from practice of the present application. BRIEF DESCRIPTION OF DRAWINGS

[0030] The above and / or additional aspects and advantages of the present application will become apparent and be more readily understood through consideration of the following description, taken in conjunction with the accompanying drawings, in which:

[0031] Figure 1 is a flowchart of the fingerprint two-dimensional posture estimation method based on dense estimation according to an embodiment of the present application;

[0032] Figure 2 is a logic diagram of the fingerprint two-dimensional posture estimation method based on dense estimation according to an embodiment of the present application;

[0033] Figure 3 is a finger 2D posture definition diagram according to an embodiment of the present application;

[0034] Figure 4 is a dense estimation vector diagram according to an embodiment of the present application;

[0035] Figure 5 is a prediction network structure diagram of dense estimation according to an embodiment of the present application;

[0036] Figure 6 is a fingerprint 2D posture estimation result diagram on different modal fingerprint images according to an embodiment of the present application;

[0037] Figure 7 is a structure diagram of the fingerprint two-dimensional posture estimation system based on dense estimation according to an embodiment of the present application. DETAILED DESCRIPTION

[0038] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The technical solutions in the embodiments of the present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0039] In order to enable persons skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by persons skilled in the art without creative labor should belong to the protection scope of the present application.

[0040] A method and system for fingerprint 2D pose estimation based on dense estimation according to embodiments of the present application are described below with reference to the accompanying drawings.

[0041] Figure 1 is a flowchart of the method for fingerprint 2D pose estimation based on dense estimation according to embodiments of the present application.

[0042] As shown in Figure 1 , the method comprises but is not limited to the following steps:

[0043] S1, obtaining a fingerprint image to be predicted;

[0044] S2, inputting the fingerprint image to be predicted into a trained dense estimation model to perform dense estimation at a pixel point level to obtain a dense estimation result;

[0045] S3, integrating a fingerprint center estimation result and a fingerprint orientation estimation result based on the dense estimation result to obtain a fingerprint pose estimation result of the fingerprint image to be predicted according to the integrated result.

[0046] Specifically, the fingerprint 2D pose estimation system based on dense estimation according to the present application is divided into two stages, namely, dense pose estimation based on a fingerprint image and fingerprint overall pose integration based on dense pose estimation. The dense estimation based on a fingerprint image stage is divided into two steps, namely, fingerprint pose dense decomposition and dense estimation based on a fingerprint image. In the pose decomposition step, the fingerprint 2D pose estimation task needs to be allocated to a single pixel point; in the dense estimation step, pixel point level dense estimation is performed based on the fingerprint image according to the pose decomposition result obtained in the previous step. The fingerprint pose integration based on dense estimation stage is divided into two steps, namely, fingerprint center calculation and fingerprint orientation calculation, and the final fingerprint 2D pose estimation result is obtained by integrating the two steps respectively based on the pixel point level dense estimation result. As shown in Figure 2 , it is a logic flowchart of the present application.

[0047] It can be understood that the present application can be widely applied to various modal fingerprint images, such as rolled fingerprints, flat fingerprints, live fingerprints, etc., and can also be applied to palm prints, footprints, finger vein images, face images, and other skin textures. Preferably, the embodiments of the present application take the fingerprint 2D pose estimation of a certain fingerprint image as an example to explain the specific method of the present application. First, the symbols appearing in the embodiments of the present application are defined as follows:

[0048] Symbol: x, y: fingerprint center position, θ: fingerprint orientation, ΔGx, ΔGy: x and y direction coordinates of each pixel point in the standard coordinate system of the fingerprint, ΔCx, ΔCy: x and y direction offsets of each pixel point from the fingerprint center, N x , N y: number of classification bins in x and y direction, t: interval boundary when equal interval division, k: classification bin index, delta k : width of the k-th classification bin, g(): interval probability normalization, (i,j): pixel point with index (i,j) in output image, S: fingerprint region image, A: fingerprint orientation attention image, Z: trigonometric function normalization parameter.

[0049] As an embodiment, the first stage is a dense estimation based on a fingerprint image. The dense estimation based on the fingerprint image in the application is divided into two steps, which are fingerprint pose dense decomposition and dense estimation based on the fingerprint image. By assigning the fingerprint 2D pose estimation task to a single pixel point, the pixel point level dense estimation based on the fingerprint image can effectively reduce the influence of complex background noise and fingerprint region defects on the fingerprint 2D pose estimation.

[0050] The pose decomposition step in the application can use many schemes, for example, decomposing into directly estimating the fingerprint center position and orientation at each pixel point, and also can be decomposed into estimating on many small image blocks. Figure 3 As shown in the figure, the finger 2D pose definition graph of the application is preferred to be decomposed into an offset pointing to the fingerprint center. Two vectors are estimated at each pixel point, one is a vector (Delta Gx, Delta Gy) pointing from the current pixel point to the fingerprint center position in the image coordinate system, and the other is the coordinate (Delta Cx, Delta Cy) of the current pixel point in the standard coordinate system of the fingerprint, where the standard coordinate system of the fingerprint refers to the image coordinate system after the fingerprint pose alignment. For each pixel point, there are many schemes for predicting the above vector coordinates, for example, directly regressing the coordinate values or predicting the classification bins. Here, taking the prediction of the classification bins as an example, the possible value range of the vector coordinates is divided into N x and N y classification bins along the x and y directions. There are many schemes for dividing the classification bins, and here, taking the non-equal interval bins as an example, the following is a calculation formula for determining the non-equal interval bin boundary:

[0051]

[0052] It can be known that, Figure 4 It is shown that two kinds of vectors need to be predicted by a pixel point under this kind of pose decomposition scheme. In addition to the two kinds of vectors described above, in order to avoid the influence of the non-fingerprint region and improve the prediction performance of the fingerprint orientation, two kinds of probability images are also estimated at each pixel point, which are a fingerprint region probability image S and a fingerprint orientation attention image A.

[0053] The dense pose estimation algorithm (model) in the application can use various schemes. Here, a deep convolutional network using supervised learning is taken as an example. In the offline period, a large number of fingerprint images and corresponding fingerprint pose true values (obtained by manual labeling or automatic methods) are used to train the deep convolutional network to obtain optimized network parameters. In the online period, the trained dense pose estimation model is used to perform pixel-level dense estimation on the fingerprint image. Figure 5 A deep convolutional network structure for dense estimation is shown.

[0054] The second stage: fingerprint pose integration based on dense estimation. The fingerprint pose integration based on dense estimation in the application is divided into two steps, namely fingerprint center estimation and fingerprint orientation estimation. According to the dense estimation result obtained in the previous stage, the center position and orientation of the fingerprint can be integrated to obtain the final fingerprint 2D pose estimation.

[0055] Preferably, the fingerprint pose integration of the application is related to the pose decomposition scheme in the previous stage, and the decomposition is taken as an example of the offset to the fingerprint center. At this time, the center position estimation of the fingerprint can be integrated according to the following formula:

[0056]

[0057]

[0058] wherein, and are normalized classification interval probabilities, and the following normalization is taken as an example:

[0059] g(p ijk )= ijk / k

[0060] And the estimation of the fingerprint orientation can be integrated according to the following formula:

[0061]

[0062]

[0063]

[0064] wherein, and are obtained by fusing multiple classification intervals at each pixel point, and a weighted average is taken as an example:

[0065]

[0066]

[0067] Figure 6 The 2D pose estimation results on the fingerprint images of different modalities are shown.

[0068] The fingerprint 2D pose estimation method based on dense estimation according to the embodiment of the application can effectively improve the generalization ability of the fingerprint 2D pose estimation algorithm under various image modalities through the dense estimation manner.

[0069] To achieve the above-mentioned embodiments, as Figure 7 shown, the embodiment further provides a fingerprint 2D pose estimation system 10 based on dense estimation, which comprises an image acquisition module 100, a dense estimation module 200 and a pose estimation module 300.

[0070] The image acquisition module 100 is configured to acquire a fingerprint image to be predicted.

[0071] The dense estimation module 200 is configured to input the fingerprint image to be predicted into a trained dense estimation model to perform dense estimation at the pixel point level to obtain a dense estimation result.

[0072] The pose estimation module 300 is configured to integrate a fingerprint center estimation result and a fingerprint orientation estimation result based on the dense estimation result to obtain a fingerprint pose estimation result of the fingerprint image to be predicted according to the integrated result.

[0073] Further, the system 10 further comprises a model training module, comprising:

[0074] The sample acquisition subunit is configured to acquire sample fingerprint images and label the sample fingerprint images to generate a training data set.

[0075] The pose decomposition subunit is configured to decompose the fingerprint pose estimation task into a single pixel point of an image of the training data set by using a preset pose decomposition scheme to obtain a pose decomposition result.

[0076] The model training subunit is configured to train the dense estimation model by using the pose decomposition result to generate a trained dense estimation model.

[0077] Further, the pose decomposition subunit is further configured to:

[0078] decompose the image of the training data set into a plurality of image blocks to perform estimation on the plurality of image blocks to obtain the pose decomposition result.

[0079] Further, the dense estimation model comprises a deep convolutional network model based on supervised learning; and the model training subunit is further configured to:

[0080] acquire a large number of sample fingerprint images and corresponding fingerprint pose true values labeled.

[0081] The deep convolutional network model is trained and optimized by using the large number of sample fingerprint images and the fingerprint posture true values, to obtain optimized network model parameters based on loss calculation results of a model loss function.

[0082] Further, the posture estimation module 300 is further configured to:

[0083] The fingerprint center estimation result and the fingerprint orientation estimation result are obtained through fingerprint center calculation and fingerprint orientation calculation respectively according to the dense estimation result;

[0084] The fingerprint overall posture is integrated based on the fingerprint center estimation result and the fingerprint orientation estimation result, to obtain the fingerprint posture estimation result of the to-be-predicted fingerprint image based on the fingerprint overall posture integration result.

[0085] The fingerprint two-dimensional posture estimation system based on dense estimation according to the embodiment of the present application can effectively improve the generalization ability of the fingerprint 2D posture estimation algorithm under various image modalities through the dense estimation manner.

[0086] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" and the like means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any appropriate manner in any one or more embodiments or examples. In addition, the person skilled in the art can combine and combine the different embodiments or examples described in the present specification and the features of the different embodiments or examples without contradiction.

[0087] In addition, the terms "first", "second" are only for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of indicated technical features. Therefore, the features defined with "first", "second" can explicitly or implicitly include at least one of the features. In the description of the present application, the meaning of "a plurality of" is at least two, for example, two, three, etc., unless otherwise specifically limited.

Claims

1. A fingerprint two-dimensional pose estimation method based on dense estimation, characterized in that, Includes the following steps: Obtain the fingerprint image to be predicted, and extract the fingerprint center position and fingerprint orientation from the fingerprint image; The fingerprint image to be predicted is input into the trained dense estimation model to perform dense pose estimation based on the fingerprint image and fingerprint pose integration based on the dense pose estimation. The dense pose estimation includes dense decomposition of fingerprint pose and dense estimation based on fingerprint images; the fingerprint pose integration includes fingerprint center calculation and fingerprint orientation calculation. Specifically, the fingerprint 2D pose estimation task is assigned to a single pixel for dense decomposition of fingerprint pose, and pixel-level dense estimation is performed based on the dense decomposition result of fingerprint pose. The density estimation includes a vector pointing from the current pixel to the center of the fingerprint in the image coordinate system and the coordinates of the current pixel in the fingerprint standard coordinate system, which is the image coordinate system aligned with the fingerprint pose. Based on the dense estimation results, fingerprint center estimation and fingerprint orientation estimation are performed separately, and the fingerprint center position and orientation are integrated to obtain the fingerprint two-dimensional pose estimation result of the fingerprint image to be predicted.

2. The method according to claim 1, characterized in that, Training the dense estimation model includes: Obtain sample fingerprint images and annotate the sample fingerprint images to generate a training dataset; The fingerprint pose estimation task is assigned to a single pixel of the image in the training dataset using a preset pose decomposition scheme to obtain the pose decomposition result. The dense estimation model is trained using the attitude decomposition results to generate a trained dense estimation model.

3. The method according to claim 2, characterized in that, The attitude decomposition scheme further includes: The images in the training dataset are decomposed into multiple image patches, and the pose decomposition result is obtained by estimating on the multiple image patches.

4. The method according to claim 3, characterized in that, The dense estimation model includes a deep convolutional network model based on supervised learning; training the dense estimation model using the pose decomposition results to generate a trained dense estimation model includes: Acquire a large number of sample fingerprint images and the corresponding ground truth values ​​of fingerprint pose obtained from the annotations; The deep convolutional network model is trained and optimized using the large number of sample fingerprint images and the ground truth fingerprint pose values, and the optimized network model parameters are obtained based on the loss calculation results of the model loss function.

5. The method according to claim 4, characterized in that, The integration of fingerprint center estimation and fingerprint orientation estimation results obtained from dense estimation results, to obtain fingerprint pose estimation results for the fingerprint image to be predicted based on the integration results, includes: Based on the density estimation results, fingerprint center estimation results and fingerprint orientation estimation results are obtained respectively through fingerprint center calculation and fingerprint orientation calculation. The fingerprint center estimation result and the fingerprint orientation estimation result are integrated into the overall fingerprint pose to obtain the fingerprint pose estimation result of the fingerprint image to be predicted based on the overall fingerprint pose integration result.

6. A fingerprint two-dimensional pose estimation system based on dense estimation, characterized in that, Includes the following steps: The image acquisition module is used to acquire the fingerprint image to be predicted; The dense estimation module is used to input the fingerprint image to be predicted into the trained dense estimation model to perform dense pose estimation based on the fingerprint image and fingerprint pose integration based on the dense pose estimation. The dense pose estimation includes dense decomposition of fingerprint pose and dense estimation based on fingerprint images; the fingerprint pose integration includes fingerprint center calculation and fingerprint orientation calculation. Specifically, the fingerprint 2D pose estimation task is assigned to a single pixel for dense decomposition of fingerprint pose, and pixel-level dense estimation is performed based on the dense decomposition result of fingerprint pose. The density estimation includes a vector pointing from the current pixel to the center of the fingerprint in the image coordinate system and the coordinates of the current pixel in the fingerprint standard coordinate system, which is the image coordinate system aligned with the fingerprint pose. The pose estimation module is used to perform fingerprint center estimation and fingerprint orientation estimation based on the dense estimation results, integrate them to obtain the fingerprint center position and orientation, and obtain the fingerprint two-dimensional pose estimation result of the fingerprint image to be predicted based on the integration result.

7. The system according to claim 6, characterized in that, The system also includes a model training module, comprising: The sample acquisition subunit is used to acquire sample fingerprint images and annotate the sample fingerprint images to generate a training dataset. The pose decomposition subunit is used to assign the fingerprint pose estimation task to a single pixel of the image in the training dataset using a preset pose decomposition scheme to obtain the pose decomposition result. The model training subunit is used to train the dense estimation model using the pose decomposition results to generate a trained dense estimation model.

8. The system according to claim 7, characterized in that, The attitude decomposition subunit is also used for: The images in the training dataset are decomposed into multiple image patches, and the pose decomposition result is obtained by estimating on the multiple image patches.

9. The system according to claim 8, characterized in that, The dense estimation model includes a deep convolutional network model based on supervised learning; the model training subunit is further used for: Acquire a large number of sample fingerprint images and the corresponding ground truth values ​​of fingerprint pose obtained from the annotations; The deep convolutional network model is trained and optimized using the large number of sample fingerprint images and the ground truth fingerprint pose values, and the optimized network model parameters are obtained based on the loss calculation results of the model loss function.

10. The system according to claim 9, characterized in that, The attitude estimation module is also used for: Based on the density estimation results, fingerprint center estimation results and fingerprint orientation estimation results are obtained respectively through fingerprint center calculation and fingerprint orientation calculation. The fingerprint center estimation result and the fingerprint orientation estimation result are integrated into the overall fingerprint pose to obtain the fingerprint pose estimation result of the fingerprint image to be predicted based on the overall fingerprint pose integration result.

Images