A Finger Vein Recognition Method Based on Joint Loss and Convolutional Neural Network

Abstract

This invention discloses a finger vein recognition method based on a joint loss and convolutional neural network. The method includes the following steps: acquiring a finger vein image, preprocessing the image, extracting the region of interest (ROI) of the finger vein image, and normalizing the image size; constructing a deep convolutional neural network for finger vein recognition, using a feature extraction network based on the ResNet34 architecture; training the model by using a joint loss function in Euclidean and cosine metric spaces to train the deep convolutional neural network so that the network model learns effective feature representations of finger veins; and determining a recognition threshold, using the false acceptance rate and false recognition rate as metrics, and finding the optimal recognition threshold through enumeration to achieve finger vein image recognition. The method is used for finger vein image feature extraction and matching, effectively performing finger vein identity recognition.

Description

Technical Field

[0001] The present invention relates to computer vision and biometric recognition technology, and in particular to a method suitable for finger vein identity verification. Background Art

[0002] With the development of information technology, people's demand for information security has gradually increased, and biometric recognition has therefore gained widespread application. Biometric recognition technology uses human biological or behavioral characteristics for identity authentication. Commonly used methods include fingerprint recognition, facial recognition, and iris recognition. Fingerprints and faces are external biological features that are convenient to use and have fast recognition speeds, but they carry the risk of theft and counterfeiting. The iris is an internal biological feature that offers higher security, but it does not provide a good user experience for identity authentication. Finger veins are an internal biological feature that offers uniqueness, contactless collection, and high security, making them of great research value.

[0003] Finger vein recognition first uses a sensor to capture a finger vein image, preprocesses the vein image, and then extracts the finger vein features. This is then compared with vein feature templates in a database to complete identity authentication. Existing finger vein recognition methods can be roughly divided into two categories: traditional vein recognition methods and deep learning-based recognition methods.

[0004] Traditional vein recognition methods, based on conventional image processing techniques, use features such as vein patterns, geometric topology, and local binary codes as features to measure finger vein differences. Using a pre-set model, these features are extracted as quantifiable attributes, serving as a template. During the recognition phase, the differences between the sample under test and the template are compared. These methods rely on manual design by experts, resulting in limited algorithm generalization capabilities. Recognition performance degrades significantly when the finger is translated or rotated.

[0005] Deep learning-based methods build artificial neural networks to learn finger vein feature representations from large amounts of data. Compared to traditional methods, these methods offer better generalization and higher recognition accuracy. However, these methods often suffer from issues such as small training sample sizes and inappropriate loss function design, which limits the application of deep learning in finger vein recognition. Summary of the Invention

[0006] Aiming at the shortcomings of the existing technology in finger vein feature extraction, such as limited expression ability and restricted application, a finger vein recognition method is provided to enable the network to have stronger feature representation ability.

[0007] The technical solution adopted by the present invention to solve the above technical problems is:

[0008] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0009] This paper proposes a finger vein recognition method based on a joint loss and convolutional neural networks. Based on a deep convolutional neural network, the finger vein recognition network can represent deeper features. Using a joint loss function in Euclidean and cosine metric spaces, the sample features are kept compact within classes and dispersed between classes, giving the network stronger feature representation capabilities. Furthermore, a training method based on sample pairing overcomes the limitation of limited training data required by deep learning methods, thereby improving recognition performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 Schematic diagram of the data set partitioning protocol of the present invention;

[0011] Figure 2 Schematic diagram of the training process of the present invention;

[0012] Figure 3 It is a schematic diagram of the combined loss of the present invention; DETAILED DESCRIPTION

[0013] Step 1: Collect finger vein images, preprocess them to obtain finger vein samples, mark them with classification labels, and establish a finger vein dataset;

[0014] Step 2: Build and train a deep convolutional neural network for finger vein recognition:

[0015] The deep convolutional neural network includes a feature extraction network and a classification network; the feature extraction network is used to extract vein feature vectors from finger vein samples and input them into the classification network; the classification network is used to receive the vein feature vectors and output the finger vein recognition results;

[0016] The feature extraction network is trained based on a joint loss optimization network; the joint loss function L = L1 + L2, where L1 constrains the feature representation of the sample in Euclidean space and L2 constrains the feature representation of the sample in cosine space:

[0017]

[0018]

[0019] Among them, α and β are adjustment factors, which are used to adjust the size of L1 and L2 losses respectively to prevent the difference between the two from being too large. ||·||2 represents the 2-norm of the vector, f(·) represents the feature extraction function, and x i (i∈N) represents the i-th input finger vein image, N represents the number of samples in a batch, They represent the anchor sample a of the i-th input finger vein image, the positive sample p of the same class as the anchor sample, and the negative sample n of a different class from the anchor sample. β represents the increase interval between the inter-class distance and the intra-class distance. In L2, Represents the i-th sample feature and network weight The angle between the vectors, due to ||x i ||2 and The feature vectorization is performed, and the result is 1, which can be simplified. s represents the scaling factor, m represents the increase interval between weights and features, and θ j Represents the weight w yj The angle between the vector of the jth sample (i≠j). The joint loss function allows the network to learn the feature representation of finger veins from the Euclidean metric space and the cosine metric space, and through the sample pairing training method, the data scale of the original number of samples is expanded from N to

[0020] During the training phase, the classification network predicts the category to which the sample belongs, and uses the backpropagation algorithm to update the weights of the feature extraction network and the classification network based on the error between the predicted value and the true value;

[0021] Step 3: Calculate the recognition threshold:

[0022] After training the deep convolutional neural network, the classification network is removed, retaining the feature extraction network. The feature extraction network is then used to extract feature representations from the finger vein dataset. Cosine similarity is used to measure the distance between the extracted features and the corresponding vein sample features in the dataset. This is then compared to a given threshold to determine whether the two features are similar. The similarity judgment results are then used to determine the recognition threshold. Specifically, the false rejection rate (FRR) and false acceptance rate (FAR) can be calculated from the similarity judgment results to evaluate the performance of the feature extraction network and determine the recognition threshold.

[0023] Step 4: Output the prediction result of the image to be judged based on the finger vein feature extraction network:

[0024] For the registered finger vein image, a feature representation of the registered finger vein is obtained based on the finger vein feature extraction network, and the feature representation is stored in the feature database as a registration template;

[0025] For the queried finger vein image to be identified, the feature representation of the finger vein to be identified is obtained through the feature extraction network, and the cosine similarity between the feature representation and the template in the feature database is calculated and compared with the recognition threshold to determine whether it has been registered and complete identity recognition.

[0026] Specifically, in step 1, image preprocessing includes extracting the region of interest (ROI) of the finger vein image and normalizing the image size.

[0027] Furthermore, the image ROI extraction uses the Sobel operator to obtain the finger boundary, fits the finger center axis based on the least squares method, corrects the finger rotation, locates the knuckle based on the finger joint cavity, and intercepts the image region of interest of the knuckle.

[0028] Specifically, in step 1, the finger vein dataset is divided into a training set, a validation set, and a test set based on an open set protocol, and the test set is further divided into a registration set and a query set.

[0029] Furthermore, in step 2, the deep convolutional neural network for finger vein recognition is improved based on the existing deep convolutional neural network in the following ways: the feature extraction network adopts the ResNet34 structure, and replaces the 7×7 convolution kernel with a small 3×3 convolution kernel, which retains more local detail information while reducing the number of network parameters; at the same time, the BatchNorm in the original residual module is adjusted to before the convolution operation, which can ensure the stability of the input distribution and allow the network to learn from the original data distribution; the ReLU activation function is replaced by the Leaky ReLu function to avoid mean shift; at the end of the residual module, a channel attention module is introduced to allow the network to focus on more important information; the final output features of the network are L2 regularized to facilitate the calculation of cosine similarity.

[0030] The present invention will be described in further detail below in conjunction with the embodiments and accompanying drawings.

[0031] First, step 1 preprocesses the acquired image as follows:

[0032] In step S1-1, a finger vein image is input and a convolution operation is performed on the image using the Sobel operator to coarsely filter the background information of the image and obtain the X-direction and Y-direction gradients, which are calculated as follows:

[0033]

[0034] Among them, A represents the original image, G x represents the gradient of the finger vein image in the X direction, G y Indicates the gradient of the finger vein image in the Y direction.

[0035] In step S1-2, since other noises are distributed relatively discretely in the image after the background information of the coarse-step filtering image is filtered, the number of connected non-zero pixels in the image is calculated by union-find set, and the connected blocks with a connection number less than 5 are set to zero to obtain a binary image of the finger boundary.

[0036] Step S1-3, according to the finger at the corresponding horizontal coordinate x i The coordinates of the upper and lower boundaries of and Calculate the coordinates of the finger's central axis (where (i∈[0,W-1], W is the image width), and the finger center axis is fitted based on the least squares method.

[0037] In step S1-4, the slope is calculated based on the fitted finger midline. The angle corresponding to the slope is the finger rotation angle. The finger vein image is rotationally corrected using the rotation angle θ to obtain the corrected image G′. The rotation angle is calculated as follows:

[0038]

[0039] Among them, y1 and y2 are the vertical coordinates of any two points on the central axis of the finger, and x1 and x2 are the corresponding horizontal coordinates.

[0040] Step S1-5, determine the ROI boundary of the finger vein image. Since the finger joint cavity appears as a brighter area in the finger vein image, there will be two peaks in the pixel value along the central axis of the finger. The left and right boundaries of the ROI area can be determined based on the horizontal coordinates of the two peaks. At the same time, the upper and lower boundaries of the ROI area can be determined based on the upper and lower boundaries of the finger. Specifically, first, obtain the pixel value G′[x i ][y](i∈[0,W-1]), where W is the width of the image. The sliding window traverses the pixels and obtains the coordinates x of the highest and second highest pixel values. l 、x r , i.e. the left and right boundaries of the ROI area. Then, according to step S1-2, the upper and lower boundaries y of the finger of the corrected image G′ are obtained. up 、y down To determine the upper boundary of the vein image ROI area and the lower boundary H is the width of the image, Represents the upper and lower boundaries of the finger at the horizontal position h in the image, h∈[0,H].

[0041] In step S1-6, a region of interest is cut out from the rectified image G′ according to the boundary coordinates of the ROI, and the size of the cut out image is normalized to 300×100.

[0042] Step S1-7, divide the finger vein dataset based on the open set protocol, see Figure 1 , half of the samples in the finger vein category are taken as the training set, and the rest are used as the validation set and test set (1:1). The test set is further divided into the registration set and the query set.

[0043] In step 2, the deep convolutional neural network for finger vein recognition constructed by the present invention includes a feature extraction network and a classification network. In this specific embodiment, the pytorch deep learning framework is used to construct a deep convolutional neural network. The feature extraction network is based on the framework of ResNet34, replacing the 7×7 convolution kernel with a small 3×3 convolution kernel to retain more local detail features, and modifying the residual module - adjusting BatchNorm before the convolution operation to ensure that the input feature distribution is stable, so that the network can learn from the distribution of the original data, and replacing the ReLU activation function with the Leaky ReLU activation function to avoid mean shift in learning. The channel attention module SE is introduced at the end of the module to make the network pay more attention to features with large amounts of information. The final output features of the network are L2 regularized to facilitate the calculation of cosine similarity. See the table below for specific parameters:

[0044]

[0045]

[0046] Table 1 Improved ResNet34 network structure parameters

[0047] The classification network consists of two fully connected layers. The first layer contains 512 neurons and uses the swish activation function. The second layer contains n neurons and outputs a 1×n matrix, where n is the number of sample categories for training, indicating the probability distribution of the category to which the sample belongs.

[0048] In step 3, the deep convolutional neural network for finger vein recognition is trained on the training set as follows:

[0049] make Represents the triplet of finger vein image pair input, represents the anchor point sample, represents the positive sample of the same category as the anchor sample, represents the negative samples of different categories from the anchor samples. i ) represents the feature extraction network extracting sample x i feature representation.

[0050] The model training process is as follows Figure 2 As shown in the figure, first, the finger vein image is enhanced and preprocessed using pytorch's transforms, including random rotation [-30°, 30°] and random contrast adjustment [0, 2]. Then, the triplet consisting of three image samples is transformed into Input the feature extraction network to obtain a 3×512-dimensional feature vector and calculate the triplet feature pair Intra-class distance in Euclidean space and inter-class distance Finally, each finger vein feature is represented by f(x j ) Input the classification network to obtain the probability distribution of sample categories, and calculate the cosine similarity cos(f(x),w) between the sample feature f(x) and the weight w of the last layer of the classification network. The cosine similarity calculation formula is as follows:

[0051]

[0052] At the beginning of training, the Xavier initialization method is used to initialize the parameters of the convolutional network. The batch size of the training is 64. In each training iteration, the loss value of the convolutional network is calculated. Based on the joint loss, the parameters of the convolutional network are optimized using Adam. The maximum number of epochs for training is 150, and the weight with the smallest loss is selected for saving.

[0053] In particular, the present invention also sets a joint loss function to enhance the feature representation ability of the network and improve the prediction effect. Figure 3 The joint loss used in training the finger vein recognition network in this invention consists of two parts, namely L = L1 + L2. L1 constrains the feature representation of the sample in Euclidean space, while L2 constrains the feature representation of the sample in cosine space. The L1 and L2 loss function formulas are as follows:

[0054]

[0055]

[0056] Where α and β are adjustment factors used to adjust the size of L1 and L2 losses to prevent the difference between the two from being too large, ||·||2 represents the 2-norm of the vector, f(·) represents the feature extraction function, and x i (i∈N) represents the finger vein image of the i-th input, N represents the number of samples in a batch, a, p, n represent the anchor sample, the positive sample of the same class as the anchor sample, and the negative sample of a different class from the anchor sample, respectively, γ represents the increase interval between the inter-class distance and the intra-class distance; in L2, θ yi Represents the i-th sample feature and network weight The angle between the vectors, s represents the scaling factor, m represents the increase interval between weights and features, cosθ j is the network weight The cosine angle between the jth sample and the

[0057] In step 4, the recognition threshold is a decision boundary that distinguishes the same and different types of features in the finger vein image. The following steps are required:

[0058] Step S4-1: Pair the samples of the validation set divided in step 1 into two categories: pairs of similar samples and pairs of different samples, and mark the data as 1 and 0; A triplet is three samples, and the validation set is a pair of samples. The similarity between the two samples is used to determine whether they belong to the same category.

[0059] After the convolutional network training in step S4-2 and step 3 is completed, the classification network is removed, the feature extraction network is retained, and the network weights are loaded using pytorch;

[0060] Step S4-3, use the tripartite library numpy to generate [th l ,th r ] 100 threshold arrays of equal length within the interval {th l ,th l+1 ,…,th r};

[0061] Step S4-4: Input the sample pairs of the validation set into the feature extraction network, and obtain two feature representation vectors v for each sample pair. i and v j Since the vector is L2 regularized and its modulus is 1, the inner product of the two is the cosine similarity. Calculate the cosine similarity of the feature representation vector cos(v i ,v j ), traverse each threshold th in the threshold array i , if cos(v i ,v j )>th i , then the output 1 indicates that the network predicts v i and v j Belong to the same category, otherwise the output is 0, indicating that they do not belong to the same category.

[0062] In step S4-5, the results predicted by the statistical model and the actual labels are compared, and the performance of the feature extraction network is evaluated by the false rejection rate (FRR) and the false acceptance rate (FAR). The calculation formula is as follows:

[0063]

[0064]

[0065] Among them, according to the data annotation of S4-1, TP means that the actual label is 1 and the model prediction result is also 1, FN means that the actual label is 1 and the model prediction result is 0, FP means that the actual label is 0 and the model prediction result is 1, and TN means that the actual label is 0 and the model prediction result is 1.

[0066] Step S4-6: After traversing all elements in the threshold array, the corresponding FAR and FRR are obtained, and the threshold when FAR and FRR are equal is selected as the optimal recognition threshold T.

[0067] In step 5, the prediction result of the image to be judged is output based on the finger vein feature extraction network:

[0068] Step S5-1: The finger vein images of the test set are divided into a registration set and a query set. For the registered finger vein images, the finger vein feature representation v is obtained based on the finger vein feature extraction network. r , the feature representation is appended and stored as a template in the feature database;

[0069] Step S5-2, after registration is completed, enter the query phase and retrieve the feature matrix M from the feature database;

[0070] Step S5-3, for the finger vein image to be identified in the query set, the feature representation v is obtained through the feature extraction network u , and use the Python broadcast mechanism to broadcast v u Expand to the same dimension as the feature matrix M, denoted as V;

[0071] Step S5-4, calculate the transposed V of the sample feature representation to be judged T The cosine similarity with the feature matrix M is calculated by setting the diagonal elements to 0 and comparing the maximum similarity θ in the matrix with the optimal recognition threshold T to determine whether the sample to be identified has completed registration. If the maximum similarity θ is greater than the recognition threshold T, it indicates that the sample to be identified has been registered and the corresponding information can be queried according to the index of the feature matrix in which it is located.

[0072] The above description is only a specific embodiment of the present invention. Any feature disclosed in this specification, unless otherwise stated, can be replaced by other equivalent or alternative features with similar purposes; all disclosed features, or all steps in the methods or processes, except for mutually exclusive features and / or steps, can be combined in any way.

Claims

1. A finger vein recognition method based on joint loss and convolutional neural network, characterized in that: The following steps are involved: Step 1: Collect finger vein images, preprocess them to obtain finger vein samples, and mark them with classification labels to establish a finger vein dataset; the vein data in the vein dataset is in the form of triples, including anchor samples, positive samples of the same category as the anchor samples, and negative samples of different categories from the anchor samples; Step 2: Build and train a deep convolutional neural network for finger vein recognition: The deep convolutional neural network includes a feature extraction network and a classification network; the feature extraction network is used to extract vein feature vectors from finger vein samples and input them into the classification network; the classification network is used to receive the vein feature vectors and output the finger vein recognition results; The feature extraction network is trained based on a joint loss optimization network; the joint loss function L = L1 + L2, where L1 constrains the feature representation of the sample in Euclidean space and L2 constrains the feature representation of the sample in cosine space: Among them, α and β are adjustment factors, which are used to adjust the size of L1 and L2 losses respectively to prevent the difference between the two from being too large. ||·||2 represents the 2-norm of the vector, f(·) represents the feature extraction function, and x i (i∈N) represents the i-th input finger vein image, N represents the number of samples in a batch, They represent the anchor sample a of the i-th input finger vein image, the positive sample p of the same class as the anchor sample, and the negative sample n of a different class from the anchor sample. γ represents the increase interval between the inter-class distance and the intra-class distance. In L2, is the i-th sample feature and network weight The angle between the vectors, s represents the scaling factor, m represents the increase interval between weights and features, θ j Represents weight The angle between the vector of the jth sample (i≠j), i and j are both sample numbers; During the training phase, the classification network predicts the category to which the sample belongs, and uses the backpropagation algorithm to update the weights of the feature extraction network and the classification network based on the error between the predicted value and the true value; Step 3: Calculate the recognition threshold: After the deep convolutional neural network training is completed, the classification network is removed and the feature extraction network is retained. The feature extraction network is used to extract feature representations on the finger vein dataset, and the cosine similarity is used to measure the distance between the extracted features and the corresponding vein sample features in the dataset. The distance is compared with a given threshold to determine whether the two features are similar. The value of the recognition threshold is then determined based on the similarity judgment results. Specifically, the false rejection rate (FRR) and false acceptance rate (FAR) can be obtained from the similarity judgment results to evaluate the effect of the feature extraction network and determine the value of the recognition threshold. Step 4: Output the prediction result of the image to be judged based on the finger vein feature extraction network: For the registered finger vein image, a feature representation of the registered finger vein is obtained based on the finger vein feature extraction network, and the feature representation is stored in the feature database as a registration template; For the queried finger vein image to be identified, the feature representation of the finger vein to be identified is obtained through the feature extraction network, and the cosine similarity between the feature representation and the template in the feature database is calculated and compared with the recognition threshold to determine whether it has been registered and complete identity recognition.

2. The method according to claim 1, wherein: Image preprocessing includes region of interest (ROI) extraction and image size normalization.

3. The method according to claim 2, wherein: The region of interest (ROI) extraction of the finger vein image uses the Sobel operator to obtain the finger boundary, the finger rotation is corrected based on the least squares method to fit the finger center axis, and the image region of interest (ROI) of the finger joint is intercepted based on the finger joint cavity to locate the finger joint.

4. The method according to claim 3, wherein: The specific method of using the Sobel operator to obtain the finger boundary is: First, roughly filter the background information of the finger vein image: Among them, A represents the original image, G x represents the gradient of the finger vein image in the X direction, G y Represents the gradient of the finger vein image in the Y direction; Then, the number of connected non-zero pixels in the finger vein image after coarse step filtering is calculated by union-find, and the connected blocks with a connection number less than 5 are set to zero to obtain a binary image of the finger boundary.

5. The method according to claim 4, wherein: The method for correcting finger rotation based on fitting the finger midline using the least squares method is: According to the coordinate y of the upper and lower boundaries of the finger on the binary image up 、y down , calculate the coordinates of the finger's central axis And the finger midline is fitted based on the least square method; According to the fitted finger central axis finger rotation angle θ, the finger vein image is rotationally corrected using the finger rotation angle θ to obtain the corrected image G′. Among them, y1 and y2 are the vertical coordinates of any two points on the central axis of the finger, and x1 and x2 are the corresponding horizontal coordinates.

6. The method according to claim 1, wherein: The feature extraction network adopts an improved ResNet34 structure. The 7×7 convolution kernel in ResNet34 is replaced with a 3×3 convolution kernel. The batch normalization module BatchNorm in the residual module is adjusted to before the convolution operation; the ReLU activation function is replaced with the Leaky ReLu function; the channel attention module is introduced before the output of the residual module; and the final output features of the network are L2 regularized.

Images