A fingerprint recognition method based on the fusion of global features and local minutiae features

Abstract

The present invention discloses a fingerprint recognition method based on the fusion of global features and local minutiae features. The present invention comprises the following steps: (1) batch image input and preprocessing; (2) feature extraction using a CvT model network; (3) obtaining minutiae information of the fingerprint image using a CNN convolutional neural network; (4) performing minutiae screening on the obtained minutiae score map by setting a threshold, and finally obtaining minutiae information; (5) performing a test, and performing image similarity evaluation on the trained student network using the obtained global feature map and minutiae information. The present invention aims to achieve high-precision fingerprint recognition by realizing the complementarity of global and local fingerprint information through the fusion of fingerprint global features and minutiae local features.

Description

Technical Field

[0001] The present invention relates to the field of computer vision technology, in particular to feature extraction and fusion technology in deep learning, and specifically proposes a fingerprint recognition method based on the fusion of global features and local detail point features. Background Art

[0002] With the increasing informatization of modern society, fingerprint recognition, as an efficient, reliable, and secure biometric technology, has gained widespread application. The core of fingerprint recognition systems lies in fingerprint feature extraction and matching. Traditional methods for fingerprint image processing typically rely on local minutiae features for recognition. However, using only local minutiae features can be affected by noise and distortion, resulting in reduced fingerprint recognition accuracy.

[0003] In order to improve the accuracy and robustness of fingerprint recognition, many scholars have proposed a method that combines global features with local minutiae features in recent years. This method can use a global feature extractor to extract global information from the fingerprint image, and a local minutiae feature extractor to extract minutiae information from the fingerprint image. The two types of information are then combined to obtain richer and more accurate fingerprint features.

[0004] However, current global feature extractors often require a large amount of computing resources and training data, while local minutiae feature extractors may be affected by noise and distortion. To address these issues, this patent proposes a fingerprint recognition method based on the fusion of global features and local minutiae features. This method combines convolutional neural networks and minutiae detection algorithms, achieving higher computational efficiency and robustness while ensuring high accuracy. Summary of the Invention

[0005] The present invention proposes a fingerprint recognition method based on the fusion of global features and local minutiae features, aiming to achieve high-precision fingerprint recognition by realizing the complementarity of fingerprint global information and local information through the fusion of fingerprint global features and local minutiae features.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions:

[0007] (1) Batch image input and preprocessing: Use the Gabor filter algorithm to perform noise reduction and image enhancement on batch fingerprint images, then perform cropping and normalization preprocessing, and divide the preprocessed image dataset into training set and test set;

[0008] (2) Using the CvT model network for feature extraction: the fingerprint images in the preprocessed training set are input into a weight-shared encoder network to obtain high-dimensional feature maps of batch fingerprint images, and the corresponding high-dimensional feature maps are used as the fingerprint global features of the corresponding fingerprint images;

[0009] (3) Use CNN convolutional neural network to obtain the minutiae information of the fingerprint image: the input is the fingerprint global feature map of the fingerprint image, and after multiple layers of convolution and pooling, a minutiae score map is obtained;

[0010] (4) Screening the minutiae points on the obtained minutiae score map by setting a threshold, and finally obtaining the minutiae point information;

[0011] (5) Perform the test and evaluate the image similarity of the trained student network using the obtained global feature map and detail point information.

[0012] Furthermore, the steps of preprocessing and data enhancement of the input image in step 1 are as follows:

[0013] 1-1. Determine the fingerprint image dataset and preprocess the fingerprint images, including cropping the central area to 160*160 pixels, removing background noise, and performing edge detection.

[0014] 1-2. Perform convolution operations on the preprocessed fingerprint image using the real and imaginary filter kernels to obtain a set of filtered response images;

[0015] 1-3. Perform weighted summing of the filtered response images to obtain the final filtered image, and process the filtered image for the next step of network training.

[0016] Furthermore, the steps for obtaining the fingerprint global feature map in step 2 are specifically:

[0017] 2-1 Use batch fingerprint images to train the pre-trained CvT model; the CvT model is pre-trained using a face dataset;

[0018] 2-2 The last layer output of the CvT model is used as the global feature map of the batch image to be extracted, whose dimension is 512 and is expressed as Where c, h, and w represent the number of channels, height, and width of the feature map, respectively.

[0019] Furthermore, the steps of extracting minutiae information of the fingerprint image in step 3 are specifically as follows:

[0020] 3-1 Use CNN convolutional neural network to estimate the minutiae information of fingerprint images. The global feature map of fingerprint extracted by CvT model is used as input. After 4 layers of convolution and pooling operations, a minutiae score map is output, that is, a 160*160*1 minutiae vector. The minutiae score map indicates the possibility of having a minutiae feature at the pixel point (x, y). The value range of each pixel point is [0, 1], which indicates the probability of the minutiae in the 160*160 pixel map.

[0021] The 3-2 threshold is set to T, which is used to screen the final set of candidate detail points for comparison on the obtained detail point score map.

[0022] Furthermore, the convolutional network structure of the detail point score map extraction module is as follows:

[0023] The convolutional network structure consists of four layers. The first layer is a 2D convolution layer with a convolution kernel size of [3, 3, 3, 16]; the second layer is a 2D convolution layer and a pooling layer with a convolution kernel size of [3, 3, 16, 32]; the third layer is a 2D convolution layer and a pooling layer with a convolution kernel size of [3, 3, 32, 32]; the fourth layer is a 2D convolution layer and an activation layer with a convolution kernel size of [1, 1, 32, 1], and the activation layer uses a Sigmoid activation function.

[0024] Furthermore, the steps of the image similarity evaluation method in step 6 are specifically:

[0025] 6-1 Extract global feature maps and minutiae score maps from the fingerprint images in the training set, average the global feature maps and minutiae score maps of each fingerprint image to obtain an average vector and an average score map, extract the minutiae information set from the average score map based on the threshold, and store the obtained average vector and minutiae information set as reference feature vectors;

[0026] 6-2 Extract global feature maps and detail point sets from fingerprint images in the test set;

[0027] 6-3 Compare the global feature map extracted from the test set with all stored reference feature vectors by cosine similarity, and extract the similarity result accary1;

[0028] 6-4 Calculate the Euclidean distance between the detail point information set extracted from the test set and the stored detail point information set to obtain a matching accuracy accary2, and use result = max (accary1, accary2) as the accuracy of the final image evaluation.

[0029] After testing, the advantages of this invention over previous fingerprint recognition methods are:

[0030] 1. Use the currently popular and advanced Convolutional Vision Transformer to achieve the task of fingerprint recognition;

[0031] 2. Using neural networks to extract fingerprint directional features and minutiae features, replacing traditional methods, can effectively improve the accuracy of fingerprint recognition;

[0032] 3. In the final testing phase, the global fingerprint features and detail point features were fused and evaluated, effectively achieving the complementarity of global and local fingerprint information.

[0033] In summary, the present invention uses a convolutional neural network as a global feature extractor, effectively extracting global information from fingerprint images. Furthermore, we employ a minutiae detection algorithm to extract local minutiae features. Through methods such as feature screening and fusion, we can effectively reduce the impact of noise and distortion on fingerprint recognition, thereby improving the accuracy and robustness of fingerprint recognition. Furthermore, the present invention achieves high computational efficiency and robustness while maintaining high accuracy, and thus has promising application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Flowchart of execution of the method of the present invention. DETAILED DESCRIPTION

[0035] The specific embodiments of the present invention will be further described below with reference to the accompanying drawings.

[0036] like Figure 1 As shown, a fingerprint recognition method based on the fusion of global features and local minutiae features includes the following steps:

[0037] (1) Batch image input and preprocessing: Use the Gabor filter algorithm to perform noise reduction and image enhancement on batch fingerprint images, then perform cropping and normalization preprocessing, and divide the preprocessed image dataset into training set and test set;

[0038] (2) Using the CvT model network for feature extraction: the fingerprint images in the preprocessed training set are input into a weight-shared encoder network to obtain high-dimensional feature maps of batch fingerprint images, and the corresponding high-dimensional feature maps are used as the fingerprint global feature maps of the corresponding fingerprint images.

[0039] (3) Use CNN convolutional neural network to obtain the minutiae information of fingerprint images: The input is the fingerprint global feature map of the fingerprint image. After multiple layers of convolution and pooling, a minutiae score map is obtained. The minutiae score map indicates the possibility of having a minutiae feature at the pixel point (x, y).

[0040] (4) A threshold is set to filter the detail points in the obtained detail point score map. Finally, three types of information of the detail points are obtained, namely the x-coordinate and the y-coordinate, which are used for subsequent similarity comparison.

[0041] (5) Perform the test and evaluate the image similarity of the trained student network using the obtained global feature map vector and detail point information.

[0042] The steps for preprocessing and data enhancement of the input image in step 1 are as follows:

[0043] 1-1. Determine a fingerprint image dataset and preprocess the fingerprint images, including cropping the central region to 160*160 pixels, removing background noise, and performing edge detection.

[0044] 1-2. Perform convolution operations on the preprocessed fingerprint image using real and imaginary filter kernels to obtain a set of filtered response images.

[0045] 1-3. Perform weighted summing of the filtered response images to obtain the final filtered image, and process the filtered image for the next step of network training.

[0046] The steps to obtain the global fingerprint features in step 2 are as follows:

[0047] 2-1 Use batch fingerprint images to train the pre-trained CvT model; the CvT model is pre-trained using a face dataset.

[0048] 2-2 The last layer output of the CvT model is used as the global feature of the batch image to be extracted, whose dimension is 512 and is expressed as Where c, h, and w represent the number of channels, height, and width of the feature map, respectively.

[0049] The steps for extracting minutiae from the fingerprint image in step 3 are as follows:

[0050] 3-1 uses a convolutional neural network (CNN) to estimate the minutiae information of a fingerprint image. The global fingerprint feature map extracted by CvT is used as input. After four layers of convolution and pooling, it outputs a minutiae score map, a 160*160*1 minutiae vector. The minutiae score map indicates the probability of a minutiae feature at a pixel (x, y). Each pixel has a value in the range [0, 1], indicating the likelihood of a minutiae in this 160*160 pixel image. The convolutional network structure of the minutiae score map extraction module is shown in Table 1.

[0051] The 3-2 threshold is set to 0.5, which is used to screen the final set of candidate detail points for comparison on the obtained detail point score map.

[0052] Table 1 Model structure of detail point score map extraction module

[0053]

[0054] Note: Input shape is (160, 160, 3); conv2d represents 2D convolution, kernel = [ksize, ksize, Cin, Cout], where ksize represents the convolution kernel size, cin represents the input channel, and cout represents the output channel.

[0055] The steps of the image similarity evaluation method in step 6 are specifically:

[0056] 6-1 Extract feature maps and minutiae score maps from the fingerprint images in the training set, average the feature maps and minutiae score maps of each fingerprint image to obtain an average vector and an average score map, extract the minutiae set from the average score map based on the threshold, and store the obtained average vector and minutiae set as reference feature vectors;

[0057] 6-2 Extract feature maps and detail point sets from fingerprint images in the test set;

[0058] 6-3 Compare the feature map extracted from the test set with all stored reference feature vectors by cosine similarity, and extract the similarity result accary1;

[0059] 6-4 Calculate the Euclidean distance between each pair of minutiae points extracted from the test set and the stored minutiae points to obtain a matching accuracy accary2. Result = max(accary1, accary2) is used as the accuracy of the final image evaluation. The fingerprint category with the highest similarity result is selected as the predicted category.

[0060] The present invention mainly proposes a fingerprint recognition method based on the fusion of global features and local minutiae features, which improves the accuracy and robustness of the algorithm model.

[0061] The present invention uses the CvT (Convolutional Vision Transformer) model pre-trained on the face dataset to perform batch fingerprint image training. Compared with the traditional CNN model, the CvT model can significantly reduce the computational complexity while maintaining high accuracy and has higher computational efficiency. The output of the last layer of the model is used as the feature map of the batch image to be extracted, which has a dimension of 512 and is expressed as Where c, h, and w represent the number of channels, height, and width of the feature map, respectively;

[0062] Experimental test results

[0063] We collected the following five datasets: CT-1000 (10 fingers from both hands of passerby A, totaling 1000 fingerprint images), LSY-1000 (10 fingers from both hands of passerby B, totaling 1000 fingerprint images), CT+LSY-2000 (20 fingers from both hands of passersby A and B, totaling 2000 fingerprint images), 10-index finger (10 index fingers, approximately 500 fingerprint images), and 30-index finger (30 index fingers, approximately 1200 fingerprint images). We also validated our algorithm on the first fingerprint database of the FVC2008 competition dataset (testing 46 categories). The test results for these datasets are shown in Table 2 below:

[0064] Table 2 Test results of different dataset constructions

[0065] Dataset Test finger Training set:test set Accuracy (%) ct-1000 10 fingers on both hands 800:200 99.5 lsy-1000 10 fingers on both hands 800:200 99.5 ct+lsy-2000 10 fingers on both hands 1600:400 100 10—Index finger Index fingers of both hands 100:465 99.5 30—index finger Index fingers of both hands 300:900 99 fvc2008 46 human fingers 368:46 100

[0066] The input images (three channels) of different sizes were tested, and the test results are shown in Table 3 below:

[0067] Table 3 Test results of different image resolutions

[0068]

[0069]

[0070] The experimental results show that under the condition of standardized fingerprint entry, ideal recognition effect can be achieved. The final accuracy of the model is affected by the size of the image input to the model. When the image size is 448*448, it can reach 100%. As the image size decreases, the accuracy of the algorithm model will also decrease. The model is designed for fingerprint images of size 160*160, and the final accuracy reaches 93%.

Claims

1. A fingerprint recognition method based on the fusion of global features and local minutiae features, characterized in that The following steps are involved: (1) Batch image input and preprocessing: Use the Gabor filter algorithm to perform noise reduction and image enhancement on batch fingerprint images, then perform cropping and normalization preprocessing, and divide the preprocessed image dataset into training set and test set; (2) Using the CvT model network for feature extraction: the fingerprint images in the preprocessed training set are input into a weight-shared encoder network to obtain high-dimensional feature maps of batch fingerprint images, and the corresponding high-dimensional feature maps are used as the fingerprint global features of the corresponding fingerprint images; (3) Use CNN convolutional neural network to obtain minutiae information of fingerprint images: the input is the fingerprint global feature map of the fingerprint image, and after multiple layers of convolution and pooling, a minutiae score map is obtained; (4) The obtained detail point score map is filtered by setting a threshold, and finally the detail point information is obtained. The specific implementation is as follows: 4-1. Use a CNN convolutional neural network to estimate the minutiae information of a fingerprint image. The global fingerprint feature map extracted by the CvT model is used as input. After four layers of convolution and pooling, a minutiae score map is output, i.e., a 160*160*1 minutiae vector. The minutiae score map indicates the probability of a minutiae feature at a pixel (x, y). The value of each pixel ranges from [0 to 1], indicating the probability of a minutiae feature in the 160*160 pixel map. 4-2. The threshold is set to T, which is used to screen the candidate minutiae points for comparison on the obtained minutiae score map; (5) Perform the test and evaluate the image similarity of the trained student network using the obtained global feature map and detail point information.

2. The fingerprint recognition method based on the fusion of global features and local minutiae features according to claim 1 is characterized in that: The steps for preprocessing and data enhancement of the input image in step 1 are as follows: 1-1. Determine the fingerprint image dataset and preprocess the fingerprint images, including cropping the central area to 160*160 pixels, removing background noise, and performing edge detection. 1-2. Perform convolution operations on the preprocessed fingerprint image using the real and imaginary filter kernels to obtain a set of filtered response images; 1-3. Perform weighted summing of the filtered response images to obtain the final filtered image, and process the filtered image for the next step of network training.

3. The fingerprint recognition method based on the fusion of global features and local minutiae features according to claim 1 or 2, characterized in that: The steps to obtain the fingerprint global feature map in step 2 are as follows: 2-1. Use batch fingerprint images to train the pre-trained CvT model; the CvT model is pre-trained using a face dataset; 2-2. The last layer output of the CvT model is used as the global feature map of the batch image to be extracted, whose dimension is 512 and is expressed as Where c, h, and w represent the number of channels, height, and width of the feature map, respectively.

4. The fingerprint recognition method based on the fusion of global features and local minutiae features according to claim 3 is characterized in that The convolutional network structure of the detail point score map extraction module is as follows: The convolutional network structure consists of four layers. The first layer is a 2D convolution layer with a convolution kernel size of [3, 3, 3, 16]; the second layer is a 2D convolution layer and a pooling layer with a convolution kernel size of [3, 3, 16, 32]; the third layer is a 2D convolution layer and a pooling layer with a convolution kernel size of [3, 3, 32, 32]; the fourth layer is a 2D convolution layer and an activation layer with a convolution kernel size of [1, 1, 32, 1], and the activation layer uses a Sigmoid activation function.

5. The fingerprint recognition method based on the fusion of global features and local minutiae features according to claim 4 is characterized in that The steps of the image similarity evaluation method in step 6 are specifically: 6-1. Extract global feature maps and minutiae score maps from the fingerprint images in the training set. Average the global feature maps and minutiae score maps of each fingerprint image to obtain an average vector and an average score map. Extract the minutiae information set from the average score map based on the threshold value, and store the obtained average vector and minutiae information set as the reference feature vector. 6-2. Extract global feature maps and minutiae points from the fingerprint images in the test set; 6-3. Compare the global feature map extracted from the test set with all stored reference feature vectors using cosine similarity, and extract the similarity result accary1; 6-4. Calculate the Euclidean distance between the detail point information set extracted from the test set and the stored detail point information set to obtain a matching accuracy accary2, and use result = max(accary1, accary2) as the accuracy of the final image evaluation.

Images