A label-guided dorsal vein recognition method

Abstract

The application discloses a label-guided dorsal vein recognition method, aiming at relieving the problem of the limitation of model performance caused by the existence of liquid drops. The method comprises the following steps: collecting a dorsal vein image dataset and dividing it into a training set, a validation set and a test set; performing data enhancement operation on the training set and the validation set; constructing a dorsal vein classification task matching task, and taking the network structure as a basic model; introducing a parallel liquid drop classification task, and fusing the output vector of the task with the basic model; jointly training the three tasks to obtain an optimal model; calculating the similarity distance between the test image and the image in the image library by using the model, and then obtaining a prediction result; constructing intra-class image pairs and inter-class image pairs, and evaluating the model performance. The application solves the problem that the prior art lacks a corresponding solution under the condition that liquid drops exist on the back, and experimentally verifies that the proposed method can better improve the dorsal vein recognition performance when liquid drops exist.

Description

TECHNICAL FIELD

[0001] The present application belongs to the field of biometric recognition, and in particular, relates to a label-guided dorsal hand vein recognition method. BACKGROUND

[0002] As a kind of vein recognition, dorsal hand vein recognition has the characteristics of living body detection, difficult to forge, stronger user acceptance compared with other biometric technologies such as fingerprint, face and iris.In addition, compared with finger and palm vein, the advantage of dorsal hand vein recognition is that its texture is usually coarser, and is relatively not susceptible to factors such as temperature, pollution, position and other surrounding environment, so a more robust model can usually be obtained.

[0003] Dorsal hand vein recognition models can be divided into traditional methods and deep learning-based methods. Among them, traditional methods such as principal component analysis (PcA), sparse representation, graph matching, etc. However, the disadvantage of this kind of method is that the extracted features are usually low-level or intermediate-level features, it is difficult to extract high-level semantic features of dorsal hand vein, and different data sets need to be designed for specific manual features, so the generalization ability is limited.

[0004] In contrast, image classification and image matching models in deep learning methods can be an effective solution to the above problems. As a data-driven model, deep learning-based methods do not need to design specific manual features for different data sets. In addition, considering the design of nonlinear and deep network structure, deep learning model can extract high-level semantic features closer to the essence of things. Therefore, compared with traditional methods, it can usually obtain higher recognition accuracy.

[0005] However, it is found in practice that the presence of liquid drops on the back of the hand has a great limitation on the performance of the model, and through research it is found that related research work is still relatively lacking. SUMMARY

[0006] Because of the above problems existing in the prior art, the present application proposes a label-guided dorsal hand vein recognition method to improve the performance of vein recognition under the condition that there are liquid drops on the back of the hand.

[0007] The present application can be realized by the following technical solutions:

[0008] A label-guided dorsal hand vein recognition method, comprising the following steps:

[0009] 1) Take the back of the hand by a near-infrared camera to obtain a back of the hand vein image dataset, which includes different testers' back of the hand vein images respectively, and multiple images of each tester are collected, wherein part of the testers' back of the hand vein images contain droplets;

[0010] 2) First, based on the MobileNet lightweight backbone network, a back of the hand vein classification task and a matching task are constructed as a basic model, wherein the two tasks share the backbone network features, then a droplet classification task is introduced in parallel with the two tasks based on the middle layer features of the backbone network, and finally the output vector of the droplet classification task is fused with the output vector in the above basic model to guide the model to learn the differences between the droplet-containing samples and the droplet-free samples.

[0011]

[0012] 3) The optimization goal of the above droplet classification task and the back of the hand vein classification task is to minimize the cross-entropy loss, while the optimization goal of the matching task is to minimize the triplet loss, and the above three tasks are jointly trained until the validation set loss no longer decreases in a certain period of time, thereby obtaining the best model and weight;

[0013] 4) Based on the above best model and weight, each image in the image library used for matching is encoded, and the feature vector of the test image is predicted, then the similarity distance between the feature vector of the test image and the encoded feature vector of each image in the image library is calculated, thereby obtaining the minimum value corresponding to the encoded feature vector, and the label name corresponding to the encoded feature vector, which is the matching result;

[0014] 5) Construct intra-class image pairs and inter-class image pairs, and evaluate the performance of the model through comparative experiments to verify the effectiveness of the proposed method.

[0015] Further, the dataset used in the experiment includes a training set, a validation set and a test set, wherein the same data enhancement operation is adopted for the training set and the validation set.

[0016] Further, the process of constructing the back of the hand vein classification task and the matching task is to connect the global maximum pooling and the fully connected layer after the output features of the backbone network, then connect the fully connected layer related to the above classification task and matching task based on the output of the above fully connected layer, and perform L2 regularization operation, thereby constructing the back of the hand vein classification task and the matching task.

[0017] Further, the droplet classification task is to judge whether the input image contains droplets, which is essentially a binary classification problem.

[0018] ​Further, in the step 2, the feature fusion process, a binary classification task of judging whether the droplet exists is added, and the feature vector of the task is fused with the feature vector in the basic model in a concatenate / Add (channel stacking / addition) manner.

[0019] Advantages

[0020] The problem that the prior art lacks a corresponding vein recognition scheme when a droplet exists on the back of the hand is solved by the present application, and the back vein recognition performance of the improved model is further improved compared with the conventional method represented by the basic model. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 is the overall flowchart of the present application;

[0022] Figure 2 is the MobileNet model structure, wherein LFE (low-level feature extraction) and HFE (high-level feature extraction) represent low-level and high-level feature extraction modules, respectively;

[0023] Figure 3 is the basic model structure;

[0024] Figure 4 is the improved model structure;

[0025] Figure 5 is an example of the improved model prediction structure. DETAILED DESCRIPTION

[0026] The embodiments of the present application will be described in detail below with specific specific embodiments. Those skilled in the art can easily understand other advantages and effects of the present application from the content disclosed in the present specification.

[0027] For the label-guided back vein recognition method proposed by the present application, the overall flowchart is as shown in Figure 1 , which can be summarized as follows:

[0028] Step 1: Obtain a data set by collecting vein images through a near-infrared camera, and divide it into a training set, a validation set and a test set,

[0029] wherein the same data enhancement operation is used for the training set and the validation set, and the test set remains unchanged;

[0030] Step 2: Construct the dorsal hand vein classification and matching task as a base model from the backbone network, at the same time, construct a parallel droplet classification task sharing part of the backbone network features with the above base model, and fuse the output feature vectors of the task with the corresponding feature vectors of the base model, so as to guide the model to pay attention to the learning of the feature difference between the droplet-containing and droplet-free samples;

[0031] Step 3: Jointly train the above three tasks until the validation set loss no longer decreases within the preset epoch (selected generation), then the model corresponding to the minimum value of the validation set is the optimal model, wherein the optimization objectives of the droplet classification task and the dorsal hand vein classification task are to minimize the cross-entropy loss, and the optimization objective of the matching task is to minimize the triplet loss;

[0032] Step 4: Based on the above optimal model, the feature vector of the test image is predicted, and the feature vector of the image in the matching image library is encoded, so as to calculate the similarity distance between the predicted feature vector and each encoded feature vector, then the matching image corresponding to the minimum value is mapped to its dorsal label name, which is the recognition result of the test image;

[0033] Step 5: Based on the intra-class and inter-class image pairs constructed from the test set, the performance of the model is evaluated to verify the superiority of the improved model.

[0034] Embodiment

[0035] The experiment is run on an NVIDIA TITAN Xp / 12G graphics card, an Intel(R) Xeon(R) CPU E5-2680 v4@2.40GHz / 16G CPU hardware environment. The deep learning framework adopted is Tensorflow 1.9.0 and keras 2.2.4. In order to evaluate the performance of the model, we call the interpolate function and brentq function in the optimize sub-library of the scipy library. The data set used in the experiment is a self-built data set, part of which contains droplets of different quantities, sizes and shapes on the dorsal hand. The original image resolution is 720*720, and the size is scaled to 128*128 before model processing.

[0036] The specific implementation steps of the present application are as follows:

[0037] Step 1: Acquire dorsal hand vein images through a near-infrared camera and construct a data set

[0038] The experimental images are obtained by the subject group through a near-infrared camera shooting the back of the hand, wherein the wavelength of the near-infrared light source used is 850 nm. The self-built data set includes 797 images, which are obtained by shooting the left and right hands of 33 testers of different ages from 18 to 50 years old, so there are 66 categories (distinguish left and right hands) in total. In the experiment, the data set is divided into a training set, a validation set and a test set, wherein the number of the training set and the test set is 414 and 383 respectively, and the validation set is obtained by dividing 20% from the training set. In order to alleviate the problem of overfitting and improve the robustness of the model, the application adopts image scaling, flipping, size and color distortion to enhance the image data. In addition, each image is designed as a double-label mode, that is, it has both the class label of the back of the hand vein classification and the class label of the droplet classification.

[0039] Step 2: Model structure design

[0040] Firstly, the back of the hand vein classification task and the matching task are constructed based on the MobileNet lightweight model, and the network structure thereof is taken as a basic model. The MobileNet model and the basic model involved are as shown in Figure 2 、 Figure 3 It can be seen that the basic model uses the output features of the MobileNet model from the first convolutional layer to the last convolutional layer, and takes the corresponding structure as the backbone network. Then, from the output features of the last convolutional layer, after global maximum pooling dimension reduction and linear enhancement of the features of the fully connected layer, on the one hand, the back of the hand vein classification task is constructed through a specific task-related fully connected layer, the number of categories of this task is 66, and the target to be optimized is the cross-entropy loss. On the other hand, the back of the hand vein matching task is constructed through L2 feature normalization operation, and the target to be optimized of this task is the cross-entropy loss.

[0041] Secondly, a parallel droplet classification task is constructed based on the middle layer features of the above-mentioned backbone network, as shown in Figure 4 The network structure of this task is basically the same as the back of the hand vein classification task in the basic model, the difference is that the droplet classification task here belongs to a binary classification task, which is used to judge whether the sample contains droplets, so the number of neurons in the fully connected layer is 2, not the number of categories 66 in the aforementioned back of the hand vein classification task.

[0042] Finally, the output feature vector in the droplet classification task and the output feature vector in the basic model are fused through the concatenate (channel stacking) operation.

[0043] Step 3: Model training

[0044] The hand back vein classification task, the hand back vein matching task, and the droplet classification task are jointly trained. The training set is used to learn features, and the validation set is used to monitor the change of the validation loss to obtain the best model. The training epoch (selected generation round) is 100, wherein the initial learning rate of the first 50 epochs (selected generation round) is 1e-3, the initial learning rate of the last 50 epochs (selected generation round) is 1e-4, and when the validation loss does not decrease for 4 consecutive epochs (selected generation round), the learning rate is reduced to 0.1 times.

[0045] Step 4: Model prediction

[0046] An image matching library is constructed, and there is a reference image for each category to match with the image to be tested. The name of the category is the name of the tester. Based on the above best trained model, the feature vector of each image in the image library is predicted, and the feature vector of the image to be tested is also predicted. The above encoding process is also the process of predicting the feature vector of each image in the image library based on the best trained model. Then the Euclidean distance, i.e. the similarity, between the predicted feature vector of the test image and the encoded feature vector of each image in the image library is calculated, and the similarity value is the minimum.

[0047] The corresponding image name is the final prediction result. A typical prediction result is shown in the following table. Figure 5

[0048] Step 5: Model evaluation

[0049] Based on the test set, 1766 image pairs are constructed, wherein the image pairs of the same category and the image pairs of different categories are each half. The feature vectors of each image in each image pair are predicted by the model to obtain paired feature vectors, and then the similarity distance value of each pair of feature vectors is calculated, and the best distance threshold value corresponding to the best accuracy is obtained. After that, the above similarity distance value is compared with the best distance threshold value, and when the distance threshold value is greater than the distance threshold value, it indicates that the image pair is predicted as an inter-class image pair, i.e. a different category image pair. When the distance threshold value is less than the distance threshold value, it indicates that the model predicts the image pair as an intra-class image pair, i.e. an image pair of the same category. Finally, the previously set label (issame = True; issame = False) of each image pair is compared, and based on the comparison result, the following evaluation indexes can be calculated.

[0050] Accuracy = (TP + TN), (TP + TN + FP + FN)

[0051] EER = FRR = FAR

[0052] FAR = FP / (FP + TN)

[0053] FAR = FP / (FP + TN)​

[0054] FRR = FN / (TP + FN)

[0055] TAR = 1 - FRR

[0056] Where TP (True Positive), FP (False Positive), TN (True Negative), FN (False Negative) represent the number of true positive matching pairs, false positive matching pairs, true negative matching pairs and false negative matching pairs respectively. And TAR (True Acceptance Rate), FAR (False Acceptance Rate), FRR (False Rejection Rate), EER (Equal Error Rate) represent the correct recognition rate, false recognition rate, rejection rate and equal error rate respectively. In addition, considering that FRR and FAR change with the distance threshold, and the two are in a trade-off relationship, therefore, in order to facilitate comparison, the EER (Equal Error Rate), i.e. equal error rate is usually used to evaluate the comprehensive performance of the two indicators of FRR and FAR. The specific meanings of the above indicators are shown in Table 1.

[0057] Table 1 Evaluation index and specific meaning

[0058]

[0059] As shown in Table 2, the model performance comparison is shown in Table 2, using the commonly used evaluation indicators of accuracy (Accuracy) and equal error rate (EER), and the improved model prediction result examples are shown in Table 2. Figure 3 , Figure 4 Figure 5

[0060] Table 2 Comparison of model evaluation results (bold display indicates better)

[0061]

[0062] In summary, the problem of the lack of model under the condition of liquid droplets on the back of the hand in the prior art is solved by the embodiment. At the same time, compared with the basic model, the improved model also has further performance improvement.

[0063] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Any modification, equivalent replacement and improvement within the spirit and principle of the present application shall be included in the protection scope of the present application.​​

Claims

1. A label-guided based dorsal hand vein recognition method, characterized in that, The steps include the following: 1) Take the back of the hand by a near-infrared camera to obtain a back of the hand vein image dataset, including different testers' back of the hand vein images, and multiple images of each tester are collected, wherein part of the testers' back of the hand vein images contain droplets; 2) First, based on the MobileNet lightweight backbone network, construct the back of the hand vein classification task and the matching task as a basic model, wherein the two tasks share the backbone network features, then introduce a droplet classification task in parallel with the two tasks based on the middle layer features of the backbone network, and finally fuse the output vector of the droplet classification task with the output vector in the above basic model to guide the model to learn the difference between the droplet-containing sample and the droplet-free sample features; 3) The optimization objectives of the above droplet classification task and the back of the hand vein classification task are to minimize the cross-entropy loss, while the optimization objective of the matching task is to minimize the triplet loss, and the above three tasks are jointly trained until the validation set loss no longer decreases within a certain period of time, thereby obtaining the best model and weights; 4) Based on the above best model and weights, encode each image in the image library used for matching, and predict the feature vector of the test image, then calculate the similarity distance between the test image feature vector and the encoded feature vector of each image in the image library, thereby obtaining the minimum value corresponding to the encoded feature vector, and the encoded feature vector corresponds to the back of the hand label name, which is the matching result; 5) Construct intra-class image pairs and inter-class image pairs, and evaluate the model performance through comparative experiments to verify the effectiveness of the proposed method.

2. The label-guided dorsal hand vein recognition method according to claim 1, characterized in that, The dataset used in the experiment includes a training set, a validation set and a test set, wherein the same data enhancement operation is adopted for the training set and the validation set.

3. The label-guided dorsal hand vein recognition method according to claim 1, characterized in that, The process of constructing the back of the hand vein classification task and the matching task is to connect the global maximum pooling and the fully connected layer after the output features of the backbone network, then connect the fully connected layers related to the above classification task and matching task based on the output of the above fully connected layer, and perform L2 regularization operation, thereby constructing the back of the hand vein classification task and the matching task.

4. The label-guided dorsal hand vein recognition method according to claim 1, characterized in that, The droplet classification task is to judge whether the input image contains droplets, which is essentially a binary classification problem.

5. The method of claim 1, wherein the method is a label-guided hand dorsal vein recognition method. In step 2), the feature fusion process adds a binary classification task to judge whether the droplet exists, and the task-related feature vector and the feature vector in the basic model are fused in a concatenate / Add manner.

Images