A cross-library fingerprint recognition method based on deep learning

Abstract

The present invention discloses a cross-library fingerprint recognition method based on deep learning, comprising: dividing a first fingerprint image set and a second fingerprint image set into a target image set and a source image set; determining a pseudo target image set based on the target image set and the source image set; performing enhancement processing on the images in the target image set and the pseudo target image set to obtain a first enhanced fingerprint image and a second enhanced fingerprint image respectively; obtaining a first feature vector and a second feature vector based on the first enhanced fingerprint image and the second enhanced fingerprint image, and obtaining a first detail point feature vector and a second detail point feature vector; determining a texture matching score based on the first feature vector and the second feature vector, and determining a detail point matching score based on the first detail point feature vector and the second detail point feature vector; determining a final matching score; and calculating an equal error rate according to the final matching score to complete fingerprint recognition. The present invention can improve the accuracy of fingerprint cross-library recognition.

Description

Technical Field

[0001] The present invention belongs to the technical field of image processing, and in particular relates to a cross-library fingerprint recognition method based on deep learning. Background Art

[0002] In the prior art, fingerprint recognition is usually performed using one collector. However, with the diversification of collectors, multiple collectors may be used to collect fingerprints to complete identity registration and authentication. There are image differences between fingerprint images collected by different collectors due to different imaging principles and / or systematic deformation. Such image differences will seriously affect the accuracy of fingerprint recognition and even make fingerprint recognition impossible. Summary of the invention

[0003] In order to solve the above problems existing in the prior art, the present invention provides a cross-library fingerprint recognition method based on deep learning. The technical problem to be solved by the present invention is achieved by the following technical solutions:

[0004] A cross-library fingerprint recognition method based on deep learning, the method comprising: step 1: according to a preset image quality classification rule, a first fingerprint image set and a second fingerprint image set collected in pairs by a first fingerprint collector and a second fingerprint collector for the same finger are divided into a target image set and a source image set, wherein the image quality of the target image set is higher than the image quality of the source image set, and the images in the target image set and the source image set are matched in pairs for the same finger; step 2: based on the target image set and the source image set, a pseudo target image set is determined; wherein the pseudo target image set includes a first pseudo image set and a second pseudo image set; step 3: by using a fast fingerprint enhancement algorithm, the target image set and the pseudo target image set are The images in the target image set are enhanced to obtain a first enhanced fingerprint image and a second enhanced fingerprint image respectively; Step 4: Based on the first enhanced fingerprint image and the second enhanced fingerprint image, a first feature vector and a second feature vector are obtained, as well as a first detail point feature vector and a second detail point feature vector are obtained; Step 5: A texture matching score is determined based on the first feature vector and the second feature vector, and a detail point matching score is determined based on the first detail point feature vector and the second detail point feature vector; Step 6: A final matching score is determined based on the texture matching score and the detail point matching score; Step 7: An equal error rate is calculated based on the final matching score to complete fingerprint recognition.

[0005] In one embodiment of the present invention, before step 1, the method further comprises: cropping the fingerprint images in the first fingerprint image set and the second fingerprint image set according to a preset fixed size.

[0006] In one embodiment of the present invention, step 2 includes: step 2-1: dividing the target image set into a first target image set and a second target image set according to a preset division rule; step 2-2: dividing the source image set into a first source image set and a second source image set according to a preset division rule; step 2-3: inputting the images in the first target image set and the first source image set into an initial style transfer network in pairs to train the initial style transfer network and obtain first model parameters; step 2-4: loading the first model parameters into the initial style transfer network to obtain a first target style transfer network; step 2-5: inputting the images in the second target image set and the second source image set into the first target style transfer network in pairs to test the source images in the second source image set. The method comprises the following steps: performing style transfer according to the target images in the second target image set to obtain a second pseudo image set; step 2-6: inputting the images in the second target image set and the second source image set into the initial style transfer network in pairs to train the initial style transfer network and obtain second model parameters; step 2-7: loading the second model parameters into the initial style transfer network to obtain a second target style transfer network; step 2-8: inputting the images in the first target image set and the first source image set into the second target style transfer network in pairs to test the style transfer of the source images in the first source image set according to the target images in the first target image set to obtain a first pseudo image set; step 2-9: determining the first pseudo image set and the second pseudo image set as pseudo target image sets.

[0007] Beneficial effects of the present invention:

[0008] 1. The present invention can perform cross-library fingerprint recognition based on deep learning. Compared with the solution of manually designing feature descriptors in traditional methods, the present invention does not require manually designed feature descriptors, which can significantly improve the efficiency, cost and accuracy of fingerprint recognition;

[0009] 2. The present invention can reduce the difference between images captured by different acquisition devices and improve the fingerprint recognition accuracy by dividing the target image set and the source image set into a first target image set, a second target image set, a first source image set and a second source image set, respectively obtaining a first pseudo image set and a second pseudo image set;

[0010] 3. The present invention can improve the accuracy of fingerprint cross-library recognition without correcting image deformation and only using image enhancement algorithm.

[0011] The present invention will be further described in detail below with reference to the accompanying drawings and embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 It is a schematic diagram of a cross-library fingerprint recognition method based on deep learning provided by an embodiment of the present invention;

[0013] Figure 2 It is a flowchart of a cross-library fingerprint recognition method based on deep learning provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0014] The present invention is further described in detail below with reference to specific embodiments, but the embodiments of the present invention are not limited thereto.

[0015] Embodiment 1

[0016] See also Figure 1 , Figure 1 Schematic diagram of a cross-library fingerprint recognition method based on deep learning provided by an embodiment of the present invention, the method comprising:

[0017] Step 1: According to the preset image quality classification rules, the first fingerprint image set and the second fingerprint image set collected in pairs by the first fingerprint collector and the second fingerprint collector for the same finger are divided into a target image set and a source image set, wherein the image quality of the target image set is higher than the image quality of the source image set, and the images in the target image set and the source image set are matched in pairs for the same finger.

[0018] The preset image quality classification rule is set by those skilled in the art according to business needs, and the present invention does not impose any limitation thereto. For example, the image quality classification rule is: images with high image clarity are classified as target images, and images with low image clarity are classified as source images, based on image clarity, by human eyes.

[0019] The first fingerprint collector and the second fingerprint collector can collect fingerprint images of the same finger in pairs. For example: the first fingerprint collector collects fingerprint image A from finger 1, and the second fingerprint collector collects fingerprint image a from finger 1; the first fingerprint collector collects fingerprint image B from finger 2, and the second fingerprint collector collects fingerprint image b from finger 2; the first fingerprint collector collects fingerprint image C from finger 3, and the second fingerprint collector collects fingerprint image c from finger 3. The first fingerprint image set includes fingerprint images A, B, and C, and the second fingerprint image set includes fingerprint images a, b, and c. Since the quality of the fingerprint image collected by the first fingerprint collector is higher than that of the fingerprint image collected by the second fingerprint collector, the first fingerprint image set is divided into a target image set, and the second fingerprint image set is divided into a source image set. That is, the images in the target image set and the source image set are matched in pairs for the same finger.

[0020] Optionally, before step 1, the method further includes: cropping the fingerprint images in the first fingerprint image set and the second fingerprint image set according to a preset fixed size.

[0021] Since the image sizes collected by each collector may be different, for example, the present invention can crop all fingerprint images into a square ROI area (region of interest) with a preset fixed size, where the fixed size is set to 256.

[0022] Step 2: Based on the target image set and the source image set, determine a pseudo target image set; wherein the pseudo target image set includes a first pseudo image set and a second pseudo image set.

[0023] Optionally, the step 2 includes:

[0024] Step 2-1: Divide the target image set into a first target image set and a second target image set according to a preset division rule.

[0025] Step 2-2: Divide the source image set into a first source image set and a second source image set according to a preset division rule.

[0026] The preset division rule is set by those skilled in the art according to business needs, and the present invention does not limit it. For example, the preset division rule is: dividing the image set into the first half image set and the second half image set.

[0027] Step 2-3: inputting the images in the first target image set and the first source image set into an initial style transfer network in pairs to train the initial style transfer network and obtain first model parameters;

[0028] The present invention obtains model parameters by training a part of the target image set and the source image set, and performs style transfer test on another part of the target image set and the source image set based on the target style transfer network loaded with the model parameters to obtain a pseudo image, which can improve the similarity between the source image and the target image, thereby improving the accuracy and precision of subsequent fingerprint image recognition.

[0029] Step 2-4: Load the first model parameters into the initial style transfer network to obtain a first target style transfer network.

[0030] Step 2-5: testing the images in the second target image set and the second source image set by inputting them into the first target style transfer network, so as to perform style transfer on the source images in the second source image set according to the target images in the second target image set, thereby obtaining a second pseudo image set;

[0031] Step 2-6: Input the images in the second target image set and the second source image set into the initial style transfer network in pairs to train the initial style transfer network to obtain second model parameters.

[0032] Step 2-7: Load the second model parameters into the initial style transfer network to obtain a second target style transfer network.

[0033] Step 2-8: Input the images in the first target image set and the first source image set into the second target style transfer network for testing, so as to perform style transfer on the source images in the first source image set according to the target images in the first target image set, and obtain a first pseudo image set.

[0034] Step 2-9: Determine the first dummy image set and the second dummy image set as dummy target image sets.

[0035] Optionally, the style transfer network is a CycleGAN network.

[0036] The present invention is based on the CycleGAN network and can ignore the correction of nonlinear deformation to complete the fingerprint image pixel alignment.

[0037] Step 3: Enhance the images in the target image set and the pseudo target image set using a fast fingerprint enhancement algorithm to obtain a first enhanced fingerprint image and a second enhanced fingerprint image, respectively.

[0038] Optionally, step 3 includes:

[0039] Step 3-1: Using a fast fingerprint enhancement algorithm, respectively obtain the fingerprint direction field of the target image in the target image set.

[0040] Step 3-2: Estimate the local ridge direction and frequency of the target image based on the fingerprint direction field.

[0041] Step 3-3: Based on the local ridge direction and frequency, the images in the target image set and the pseudo target image set are enhanced to obtain a first enhanced fingerprint image and a second enhanced fingerprint image, respectively.

[0042] The present invention can improve the clarity of the ridges in the fingerprint pattern through a fast fingerprint enhancement algorithm.

[0043] The main steps of the fast fingerprint enhancement algorithm include: 1) normalization: normalize the input fingerprint image to have a pre-set mean and variance; 2) local direction estimation: estimate the direction image from the normalized input fingerprint image; 3) calculate the frequency image based on the normalized input fingerprint image and the estimated direction image; 4) regional mask estimation: divide each block in the normalized input fingerprint image into a recoverable block and an irrecoverable block to obtain a regional mask; 5) apply a set of Gabor filters with parameters set to the local ridge direction and ridge frequency to the ridge and valley pixels in the normalized input fingerprint image to obtain an enhanced fingerprint image. The fast fingerprint enhancement algorithm can improve the clarity of the ridge structure in the recoverable area of ​​the fingerprint image and remove the irrecoverable area. After the fast fingerprint enhancement algorithm, it is convenient to extract distinctive deep texture features in the subsequent process.

[0044] Step 4: Based on the first enhanced fingerprint image and the second enhanced fingerprint image, obtain a first feature vector and a second feature vector, and obtain a first minutiae point feature vector and a second minutiae point feature vector.

[0045] Optionally, step 4 includes:

[0046] Step 4-1: Input the first enhanced fingerprint image and the second enhanced fingerprint image into the feature extraction network respectively to obtain the first feature vector and the second feature vector respectively.

[0047] Step 4-2: Based on the first enhanced fingerprint image and the second enhanced fingerprint image, a first minutiae feature vector and a second minutiae feature vector are extracted by using a preset fingerprint minutiae extraction algorithm.

[0048] Optionally, the feature extraction network is a ResNeSt neural network.

[0049] The present invention can extract fixed-length feature representation of fingerprint texture through ResNeSt neural network, and obtain domain alignment features after domain adaptive correction.

[0050] For example, the feature extraction network is a ResNeSt 50 network. Specifically, the ResNeSt block adopts the idea of ​​grouping and slicing. The input is first divided into k parts inside the block, and then each k part is divided into r parts. The present invention preferably sets k to be 1 and r to be 2. The SENet and SKNet attention ideas are added to enable the network to adaptively adjust the size of the receptive field to obtain information from different receptive fields, effectively improving the feature extraction ability of the network. The network model has 50 layers, because a too deep network is not required for fingerprint images, and the network model is shown in Table 1. During training, the CurricularFaceloss loss function of metric learning is combined to adaptively adjust the relative importance of easy and difficult samples at different training stages, thereby guiding network training and making the network's predicted value closer to the label, so that the fingerprint feature extraction has higher accuracy.

[0051] Table 1 ResNeSt 50 network model structure parameters

[0052]

[0053] The preset fingerprint minutiae extraction algorithm is set by those skilled in the art according to business needs. For example, the preset fingerprint minutiae extraction algorithm uses the commercial software Verifinger SDK12.1.

[0054] Step 5: Determine a texture matching score based on the first feature vector and the second feature vector, and determine a detail point matching score based on the first detail point feature vector and the second detail point feature vector.

[0055] Optionally, step 5 includes:

[0056] Step 5-1: Based on the first feature vector and the second feature vector, a texture matching score is calculated through a cross matching protocol.

[0057] See also Figure 2 , Figure 2 is a schematic flow chart of a cross-library fingerprint recognition method based on deep learning provided by an embodiment of the present invention. Figure 2 Medium X T represents the target image, X S represents the source image, X S→T represents the pseudo target image, S→T represents the style transfer of the source image according to the target image, G T→S represents the generator from the target domain to the source domain, G S→T represents the generator from the source domain to the target domain, D T , Ds represent the discriminators of the target domain and source domain respectively.

[0058] Step 5-2: Based on the first detail point feature vector and the second detail point feature vector, a detail point matching score is calculated through a cross matching protocol.

[0059] Step 6: Determine a final matching score based on the texture matching score and the detail point matching score.

[0060] The present invention can take into account that fingerprint images have structural features that are completely different from those of human faces, and thus combines global features with fine-grained features, that is, fuses texture matching scores and detail point matching scores to obtain a final matching score.

[0061] Optionally, step 6 includes:

[0062] According to the texture matching score and the detail point matching score, the final matching score is determined, which is expressed as: s = w t s t +w m s m ;

[0063] Among them, s t represents the texture matching score, w t Indicates the proportion of texture matching score, s m represents the detail point matching score, w m It represents the proportion of texture matching score, and s represents the final matching score.

[0064] Through a large number of experimental verifications, the present invention preferably t =0.7, w m =0.3.

[0065] Step 7: Calculate the equal error rate (EER) based on the final matching score to complete fingerprint recognition.

[0066] The present invention can first use the CycleGAN network to perform style transfer on the fingerprint image, ignore the correction of nonlinear deformation to complete the fingerprint image pixel alignment, then use the fast image enhancement algorithm to improve the clarity of the fingerprint ridges, and then use the ResNeSt 50 network to extract the fixed-length feature representation of the fingerprint texture, obtain the domain alignment feature after domain adaptive correction, match to obtain the texture matching score, and finally fuse the texture and detail point matching scores at the decision level to obtain the final matching score, also known as the final similarity score, and calculate the equal error rate, thereby completing cross-library fingerprint recognition.

[0067] In summary, the beneficial effects of the present invention are:

[0068] 1. The present invention can perform cross-library fingerprint recognition based on deep learning. Compared with the solution of manually designing feature descriptors in traditional methods, the present invention does not require manually designed feature descriptors, which can significantly improve the efficiency, cost and accuracy of fingerprint recognition;

[0069] 2. The present invention can reduce the difference between images captured by different acquisition devices and improve the fingerprint recognition accuracy by dividing the target image set and the source image set into a first target image set, a second target image set, a first source image set and a second source image set, respectively obtaining a first pseudo image set and a second pseudo image set;

[0070] 3. The present invention can improve the accuracy of fingerprint cross-library recognition without correcting image deformation and only using image enhancement algorithm.

[0071] The above contents are further detailed descriptions of the present invention in combination with specific preferred embodiments, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the protection scope of the present invention.

Claims

1. A cross-library fingerprint recognition method based on deep learning, characterized in that: The method comprises: Step 1: According to a preset image quality classification rule, the first fingerprint image set and the second fingerprint image set collected in pairs by the first fingerprint collector and the second fingerprint collector for the same finger are divided into a target image set and a source image set, wherein the image quality of the target image set is higher than the image quality of the source image set, and the images in the target image set and the source image set are matched in pairs for the same finger; Step 2: determining a pseudo target image set based on the target image set and the source image set; wherein the pseudo target image set includes a first pseudo image set and a second pseudo image set; Step 3: Enhance the images in the target image set and the pseudo target image set by using a fast fingerprint enhancement algorithm to obtain a first enhanced fingerprint image and a second enhanced fingerprint image respectively; Step 4: Based on the first enhanced fingerprint image and the second enhanced fingerprint image, obtain a first feature vector and a second feature vector, and obtain a first minutiae feature vector and a second minutiae feature vector; Step 5: determining a texture matching score based on the first feature vector and the second feature vector, and determining a detail point matching score based on the first detail point feature vector and the second detail point feature vector; Step 6: Determine a final matching score according to the texture matching score and the detail point matching score; Step 7: Calculate the error rate based on the final matching score to complete fingerprint recognition The step 2 comprises: Step 2-1: Divide the target image set into a first target image set and a second target image set according to a preset division rule; Step 2-2: dividing the source image set into a first source image set and a second source image set according to a preset division rule; Step 2-3: inputting the images in the first target image set and the first source image set into an initial style transfer network in pairs to train the initial style transfer network and obtain first model parameters; Step 2-4: Loading the first model parameters into the initial style transfer network to obtain a first target style transfer network; Step 2-5: testing the images in the second target image set and the second source image set by inputting them into the first target style transfer network, so as to perform style transfer on the source images in the second source image set according to the target images in the second target image set, thereby obtaining a second pseudo image set; Step 2-6: inputting the images in the second target image set and the second source image set into the initial style transfer network in pairs to train the initial style transfer network and obtain second model parameters; Step 2-7: Loading the second model parameters into the initial style transfer network to obtain a second target style transfer network; Step 2-8: input the images in the first target image set and the first source image set into the second target style transfer network for testing, so as to perform style transfer on the source images in the first source image set according to the target images in the first target image set, thereby obtaining a first pseudo image set; Step 2-9: Determine the first dummy image set and the second dummy image set as dummy target image sets.

2. The method according to claim 1, characterized in that Before step 1, the method further comprises: The fingerprint images in the first fingerprint image set and the second fingerprint image set are cropped according to a preset fixed size.

3. The method according to claim 1, characterized in that The initial style transfer network is a CycleGAN network.

4. The method according to claim 1, characterized in that The step 3 comprises: Step 3-1: using a fast fingerprint enhancement algorithm, respectively obtaining the fingerprint direction field of the target image in the target image set; Step 3-2: Estimate the local ridge direction and frequency of the target image according to the fingerprint direction field; Step 3-3: Based on the local ridge direction and frequency, the images in the target image set and the pseudo target image set are enhanced to obtain a first enhanced fingerprint image and a second enhanced fingerprint image, respectively.

5. The method according to claim 1, characterized in that The step 4 comprises: Step 4-1: inputting the first enhanced fingerprint image and the second enhanced fingerprint image into a feature extraction network respectively to obtain a first feature vector and a second feature vector respectively; the feature extraction network is a ResNeSt neural network; Step 4-2: Based on the first enhanced fingerprint image and the second enhanced fingerprint image, a first minutiae feature vector and a second minutiae feature vector are extracted by using a preset fingerprint minutiae extraction algorithm.

6. The method according to claim 1, characterized in that The step 5 comprises: Step 5-1: Based on the first feature vector and the second feature vector, a texture matching score is calculated by a cross matching protocol; Step 5-2: Based on the first detail point feature vector and the second detail point feature vector, a detail point matching score is calculated through a cross matching protocol.

7. The method according to claim 1, characterized in that The step 6 comprises: According to the texture matching score and the detail point matching score, the final matching score is determined, which is expressed as: s = w t s t +w m s m ; Among them, s t represents the texture matching score, w t Indicates the proportion of texture matching score, s m represents the detail point matching score, w m It represents the proportion of texture matching score, and s represents the final matching score.

Images