Fingerprint vein recognition method based on ResNet and DenseNet

Abstract

The vein recognition method based on ResNet and DenseNet adopts ResNet and DenseNet to process vein images of the same vein picture under different modalities respectively. ResNet processes original vein images, and DenseNet processes vein images after ROI (Region Of Interest, ROI) region positioning. The two networks are independent of each other and perform feature extraction respectively. Finally, an adaptive threshold fusion method is used to fuse the output results of the two networks to obtain the final recognition result. The method can obtain more rich feature information, which is beneficial to improving the recognition accuracy; the effectiveness of the two networks can be accurately evaluated, and the final result can be comprehensively improved on this basis, which is beneficial to improving the accuracy of the final result; and relatively less training data can achieve good recognition performance.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of biometric recognition, and particularly relates to a finger vein recognition method based on ResNet and DenseNet, which is used for realizing high-precision finger vein recognition. BACKGROUND

[0002] Vein recognition is a new technology in the field of biometric recognition, which uses the vein blood vessel image under the human skin for identity recognition. Due to its high security, high anti-counterfeiting and other advantages, it is often applied to banks, prisons, hospitals and other scenes. Common vein recognition methods include finger vein recognition, palm vein recognition, back of hand vein recognition, etc. Among them, finger vein recognition is the current research and application hotspot due to its small device size and relatively low cost.

[0003] Finger vein recognition usually includes four links of collection, preprocessing, feature extraction and feature matching. Among them, the two more critical links are preprocessing and feature extraction. The traditional finger vein recognition methods include Gabor filter, Local Binary Patterns (LBP), Support-Vector Machine (SVM), etc., but the recognition accuracy of these methods is not high enough. In recent years, with the continuous development of artificial intelligence and deep learning technology, convolutional neural networks have been gradually applied to the field of finger vein recognition and have achieved good results. However, most of the existing methods use a single convolutional neural network for feature extraction, mainly analyzing the single feature of the image, which not only limits the algorithm accuracy, but also requires a large amount of training data to train the network to obtain an acceptable recognition accuracy. SUMMARY

[0004] In view of the above limitations, the application provides a finger vein recognition method based on ResNet and DenseNet. The method uses two different neural networks to process finger vein images in different modalities, and designs a self-adaptive threshold fusion method for feature fusion. Compared with the method of analyzing only a single feature, the application extracts two kinds of feature information of the same vein image for analysis and fusion recognition, which not only has higher accuracy, but also reduces the requirement for data volume.

[0005] The finger vein recognition method based on ResNet and DenseNet includes the following steps:

[0006] Step 1, input the finger vein image, and save the original image as a data set B;

[0007] Step 2, ROI positioning; the input finger vein image is processed using the Canny algorithm, then ROI positioning and cutting are performed, and the cut image is saved as dataset A;

[0008] Step 3, double network feature extraction and training; dataset A and dataset B are input into DenseNet and ResNet networks respectively for feature extraction, and the two networks are independently trained;

[0009] Step 4, feature fusion using an adaptive threshold fusion method; first, set two threshold vectors and initialize them; then perform the following steps:

[0010] Step 4.1, multiply the output results of the two networks with their corresponding threshold vectors, then add them to complete the fusion between the networks, and calculate the loss value;

[0011] Step 4.2, calculate the gradient according to the loss value obtained in the previous step, perform back propagation, and complete the dynamic update of the threshold vectors;

[0012] Step 5, feature matching and output; map the fusion results of the previous step to the probability of each class through the Softmax function to complete feature matching;

[0013] Step 6, repeat steps 3 to 5; when all data in the training set have been trained once, an Epoch iteration is completed, and the Epoch value is set according to the needs; save the model parameters with the best results during training;

[0014] Step 7: after completing the iteration, load the optimal network parameters, input the vein image to be identified, and obtain the matching result; finally, output the label with the maximum probability in the matching result, which is the identity information corresponding to the image.

[0015] Further, in step 2, the following sub-steps are included:

[0016] Step 2.1, use the Canny algorithm to detect the edges of the finger to obtain a binary image of the finger edges; in the image, there are only two kinds of pixel points with pixel values of 0 and 255, wherein the pixel value of 0 indicates that the pixel point does not contain edge information, and the pixel value of 255 indicates that the pixel point contains edge information;

[0017] Step 2.2, for the obtained finger edge image, starting from the center position of the first column, search pixel by pixel upwards, and the first pixel value of 255 is detected as the pixel point of the upper edge, the position is recorded and the next column is searched; repeat the above operation to find all the pixel points of the upper edge of the finger; then use the retrieval method to obtain all the pixel points of the lower edge of the finger; the upper edge position E TOP and the lower edge position E BOT of the ROI image are determined by formulas (1) and (2):

[0018] E TOP = H / 2 - min({lt i}), i = 1, 2, 3, …, W-1 (1)

[0019] E BOT = H / 2 + min({lb i}), i = 1, 2, 3, …, W-1 (2)

[0020] Wherein, H and W refer to the height value and width value of the vein image, i represents the index value, min() represents the minimum value of the set, lt i and lb i respectively represent the distance of the upper edge and lower edge pixel points of the i-th column from the center line.

[0021] Step 2.3, according to the obtained E TOP and E BOT , the original image is intercepted to obtain the ROI image, which is saved as data set A.

[0022] Further, in step 3, the ResNet neural network is used to process data set B, which is composed of residual block structure stacking; the DenseNet neural network is used to process data set A, and the feature information is extracted to the maximum extent by using dense network connection and by taking the output of the front layer network as an additional input of the rear layer network.

[0023] Further, in step 4, X and X' are respectively the output vectors of ResNet and DenseNet networks, X = (x1, x2, …, x t ) T , X' = (x'1, x'2, …, x' t ) T , t is the dimension of the vector; Q and Q' are threshold vectors, Q = (q1, q2, …, q i , …, q t ) T , Q' = (q'1, q'2, …, q' i , …, q' t ) Twherein each item q i , q' i has an initial value of 0.5, 1≤i≤t; Y is a feature vector obtained after adaptive threshold fusion, Y=(y1, y2, …, y t ) T ; Channel is the number of channels of the output vector.

[0024] Further, in step 4.1, the network fusion method is shown as formula (3):

[0025] Y=X⊙Q+X'⊙Q' (3)

[0026] wherein, ⊙ represents multiplication of corresponding elements of two vectors.

[0027] Further, in step 4.2, the update method is shown as formula (4).

[0028]

[0029] wherein, G, G' are gradient vectors calculated, and L is a learning rate of the updated threshold vector.

[0030] Advantages of the present application:

[0031] (1) The present application uses two networks in the feature extraction link, extracts features from different modal vein images, and can obtain more abundant feature information, which is beneficial to improve the recognition accuracy.

[0032] (2) The present application uses an adaptive threshold fusion method for the output results of ResNet and DenseNet networks, rather than simply adding the results. This method can accurately evaluate the effectiveness of the two networks and make a comprehensive evaluation on this basis, which is beneficial to improve the accuracy of the final result.

[0033] (3) Most of the existing methods only analyze single features of images, and the present application extracts two kinds of feature information from a vein image for analysis, which is more sufficient in mining original image data. Therefore, the present method can use relatively less training data to achieve good recognition performance. BRIEF DESCRIPTION OF DRAWINGS

[0034] Figure 1 is a specific implementation flowchart of the finger vein recognition method in the embodiment of the present application.

[0035] Figure 2 is a ROI positioning method schematic diagram in the embodiment of the present application.

[0036] Figure 3 is an adaptive threshold setting method schematic diagram in the embodiment of the present application. DETAILED DESCRIPTION

[0037] The technical solutions of the present application are further described in detail below in combination with the drawings of the specification.

[0038] The specific implementation process of the method is shown in Figure 1 .

[0039] Step 1: input the finger vein image, and save the original image as dataset B.

[0040] Step 2: ROI positioning.

[0041] This step uses the Canny algorithm to process the input finger vein image, then performs ROI positioning and cutting, and saves the cut image as dataset A. The specific process is as follows.

[0042] (1) Use the Canny algorithm to perform finger edge detection to obtain the edge image of the finger. It is a binary image, and only two kinds of pixel points with pixel values of 0 and 255 exist in the image. Among them, the pixel value of 0 indicates that the pixel point does not contain edge information, and the pixel value of 255 indicates that the pixel point contains edge information.

[0043] (2) Starting from the center position of the first column, search pixel by pixel upwards for the obtained finger edge image, and the first pixel value of 255 is detected as the pixel point of the upper edge, record its position and search the next column. Repeat the above operation to find all the pixel points of the upper edge of the finger. Then use a similar method to obtain all the pixel points of the lower edge of the finger. Next, the upper edge position E TOP and the lower edge position E BOT of the ROI image are determined by formulas (1) and (2). The principle of the method is shown in Figure 2 .

[0044] E TOP = H / 2 - min({lt i}), i = 1, 2, 3, …, W-1 (1)

[0045] E BOT = H / 2 + min({lb i}), i = 1, 2, 3, …, W-1 (2)

[0046] Where H and W are the height and width values of the finger vein image, i represents the index value, min() represents the minimum value of the set, lt i and lb i represent the distance of the upper and lower edge pixel points of the finger in the ith column from the center line, respectively.

[0047] (3) According to the obtained E TOP , EBOT The original image is cropped to obtain the ROI image, which is then saved as dataset A.

[0048] Step 3: Dual network feature extraction and training.

[0049] This step uses a dual-network approach for feature extraction, with two networks processing vein images of different modalities, namely dataset A and dataset B. The standard ResNet152 neural network is used to process the original vein images (dataset B). This network consists of a large number of stacked residual blocks. Due to its network characteristics, it can solve the gradient vanishing problem caused by deep networks. The standard DenseNet121 neural network is used to process the ROI images (dataset A). The DenseNet neural network utilizes dense network connections, using the output of the previous layer as additional input to the next layer to maximize feature extraction. In this method, the two networks are trained and feature extraction are performed independently.

[0050] Step 4: Use an adaptive threshold fusion method to perform feature fusion.

[0051] Step 3 uses two different neural networks for feature extraction, and this step fuses their results. An adaptive threshold feature fusion method is employed to achieve optimal fusion and improve recognition accuracy. The process for setting the adaptive threshold is as follows: Figure 3 As shown.

[0052] Where X and X' are the output vectors of the ResNet and DenseNet networks, respectively, and X = (x1, x2, ..., x...). t ) T X' = ​​(x'1, x'2, ..., x') t ) T t is the dimension of the vector. Q and Q' are threshold vectors, constructed similarly to X and X', respectively, Q = (q1, q2, ..., q i ,…,q t ) T Q' = (q'1, q'2, ..., q') i ,…,q' t ) T Each of the terms q i ,q' i The initial values ​​of all are 0.5, 1≤i≤t; Y is the feature vector obtained after adaptive threshold fusion, Y=(y1,y2,…,y t ) T Channel refers to the number of channels in the output vector.

[0053] Step 4.1 The two network output results are multiplied by their corresponding threshold vectors first, and then added to complete the fusion between networks and calculate the loss value Loss. The network fusion method is shown in equation (3).

[0054] Y = X 0 Q + X' 0 Q' (3)

[0055] Where, 0 represents the multiplication of corresponding elements of two vectors.

[0056] Step 4.2 The loss value Loss obtained by 4.1 calls the standard function (such as the Backward function in Pytorch) to calculate the gradient vector G, G'. Then, the threshold vector is dynamically updated through backpropagation, and the update method is shown in equation (4).

[0057]

[0058] Where, G, G' are the calculated gradient vectors, and L is the learning rate of updating the threshold vector.

[0059] Step 5: Feature matching and output.

[0060] The fusion result Y of the previous step is mapped to the probability of each class through the Softmax function, completing the feature matching;

[0061] Step 6: Repeat steps 3 to 5. When all data in the training set have been trained once, it means that an Epoch iteration is completed. In this embodiment, it is set to stop repeating after 50 Epochs of training iteration, and the model parameters with the best results in the training process are saved.

[0062] Step 7: After the above steps are completed, load the optimal network parameters, input the vein image to be recognized, and the matching result can be obtained. Finally, the label with the maximum probability in the matching result is output, which is the identity information corresponding to the image.

[0063] The application provides a finger vein recognition algorithm based on ResNet and DenseNet joint recognition. The application carries out experiments on the Hong Kong Polytechnic University public dataset, first divides the training set and the test set in a 4:2 manner, and the experimental results show that the single ResNet152 network recognition accuracy result is 98.72%, and the single DenseNet121 network recognition accuracy result is 96.96%. The application extracts two kinds of feature information of the same vein image, uses ResNet152 and DenseNet121 networks for analysis and fusion recognition, and the recognition accuracy reaches 99.52%, which is increased by 0.81% and 2.64% than using a single network respectively. Secondly, when the training set and the test set are divided in a 2:4 manner, that is, the training set is only half of the original, the recognition accuracy of the method proposed by the application reaches 97.92%, which is better than the recognition accuracy of the DenseNet121 network in the 4:2 manner, and is close to the recognition accuracy of the single ResNet152 network. In summary, the method proposed by the application can improve the recognition accuracy, the result is better than that of a single network, and can reduce the demand for training data to a certain extent. The method is real and effective.

[0064] The above merely describes preferred embodiments of the application, and the protection scope of the application is not limited to the above embodiments. Any equivalent modification or change made by those skilled in the art according to the disclosed content of the application shall be included in the protection scope of the claims.

Claims

1. A finger vein recognition method based on ResNet and DenseNet, characterized in that: The steps of the method are as follows: Step 1, input the finger vein image, and save the original image as dataset B; Step 2, ROI positioning; use Canny algorithm to process the input finger vein image, then perform ROI positioning and cutting, and save the cut image as dataset A; The step 2 specifically comprises the following steps: Step 2.1, use Canny algorithm to detect the edge of the finger, and obtain the binary image of the finger edge; there are only two kinds of pixel points with pixel values of 0 and 255 in the image, wherein the pixel value of 0 indicates that the pixel point does not contain edge information, and the pixel value of 255 indicates that the pixel point contains edge information; Step 2.2, for the obtained finger edge image, starting from the center position of the first column, search pixel by pixel upwards, and when the first pixel value of 255 is detected, it is the pixel point of the upper edge, record its position and search the next column; repeat the above operation to find all the pixel points of the upper edge of the finger; then use the search method to obtain all the pixel points of the lower edge of the finger; the upper edge position E TOP and the lower edge position E BOT of the ROI image are determined by formulas (1) and (2) E TOP = H / 2 - min({lt i}), i = 1, 2, 3, …, W - 1 (1) E BOT = H / 2 + min({lb i}), i = 1,2,3,…,W-1 (2) where H and W denote the height value and the width value of the vein image, i denotes an index value, min() denotes the minimum value of a set, lt i , lb i denote the distance of the upper edge pixel and the lower edge pixel of the i-th column of the finger from the middle line, respectively. Step 2.3, according to the obtained E TOP , E BOT , the original image is intercepted to obtain an ROI image, which is saved as dataset A; Step 3, double network feature extraction and training; dataset A and dataset B are input into DenseNet and ResNet network for feature extraction, and the two networks are trained independently; Step 4, using adaptive threshold fusion method, feature fusion; first set two threshold vectors and initialize them; then execute the following steps: Step 4.1, multiply the output results of the two networks with their corresponding threshold vectors, and then add them to complete the fusion between the networks and calculate the loss value; Step 4.2, calculate the gradient according to the loss value obtained in the last step, perform back propagation, and complete the dynamic update of the threshold vector; Step 5, feature matching and output; map the fusion result of the last step to the probability of each class through the Softmax function, and complete the feature matching; Step 6, repeat steps 3 to 5, when all data in the training set are trained once, it means that an Epoch iteration is completed, and the number of Epochs is set according to the needs; save the model parameters with the best result in the training process; Step 7: after completing the iteration, load the optimal network parameters, input the vein image to be identified, and obtain the matching result; finally, output the label with the maximum probability in the matching result, which is the identity information corresponding to the image.

2. The ResNet and DenseNet-based finger vein recognition method of claim 1, wherein: In step 3, ResNet neural network is used to process dataset B, which is composed of residual block structure stacking; DenseNet neural network is used to process dataset A, which uses dense network connection, and the output of the previous layer network is used as the additional input of the latter layer network to maximize the extraction of feature information. 3.The ResNet and DenseNet-based finger vein recognition method of claim 1, wherein: In step 4, X and X' are respectively the output vectors of ResNet and DenseNet networks, X=(x1, x2, …, xt), X'=(x'1, x'2, …, x't) t ) T , t is the dimension of the vector; Q and Q' are threshold vectors, Q=(q1, q2, …, qt), Q'=(q'1, q'2, …, q't) t ) T , where the initial value of each item q i ,…, q t ) T , Q'=(q'1, q'2, …, q't) i ,…, q' t ) T , where the initial value of each item q i ,…, q' i , 1≤i≤t; Y is the feature vector obtained after adaptive threshold fusion, Y=(y1, y2, …, yt) t ) T .

4. The ResNet and DenseNet-based finger vein recognition method of claim 1, wherein: In step 4.1, the network fusion method is shown in formula (3): Y=X⊙Q+X’⊙Q’ (3) Wherein, ⊙ represents the multiplication of corresponding elements of two vectors.

5. The ResNet and DenseNet-based finger vein recognition method of claim 1, wherein: In step 4.2, the update method is shown in formula (4): Wherein, G, G' are the calculated gradient vectors, and L is the learning rate of updating the threshold vector.

Images