Transfer learning-based palm vein recognition improvement method and system

Abstract

The invention discloses a palm vein recognition improvement method and system based on transfer learning, and the method comprises the steps: obtaining a plurality of public palm vein data sets, and carrying out the data mixing of the plurality of public palm vein data sets; carrying out image augmentation processing and ROI extraction on the mixed public palm vein data, and taking obtained palm vein ROI data as a pre-training data set; pre-training the deep convolutional neural network model based on the pre-training data set to obtain a pre-training model; small-scale palm vein sample data are collected by adopting unified collection equipment; carrying out image augmentation processing and ROI extraction on the small-scale palm vein sample data, and taking obtained palm vein ROI data as a fine tuning training data set; and performing fine tuning on the pre-training model based on the fine tuning training data set by adopting a transfer learning method to obtain a fine tuning model. While the training accuracy is improved, the model has better generalization ability, and particularly can have better adaptability when being applied to an actual scene.

Description

Technical Field

[0001] The present invention relates to the technical field of biometric recognition, and in particular to an improved palm vein recognition method and system based on transfer learning. Background Art

[0002] Palm vein recognition, as a biometric technology based on the distribution characteristics of palm blood vessels, is widely used in security authentication, access control and attendance, financial payment and other fields due to its anti-counterfeiting and privacy protection advantages.

[0003] Existing palm vein recognition includes:

[0004] 1. Dedicated Algorithm Based on Palm Vein Structural Features

[0005] Methods designed for palm vein structural features include extracting palm vein point features through scale-invariant feature transform (SIFT) for identification; using subspace methods, applying bidirectional two-dimensional linear discriminant analysis (LDA) and principal component analysis (PCA) to project or transform the image into a low-dimensional space for representation and matching; and using texture features, such as local binarization (LBP) feature extraction, GaussianRandom, Gabor filter, etc. to extract palm vein texture information and then encode it for matching and identification.

[0006] 2. Palm vein recognition method based on deep convolutional neural network

[0007] Deep learning technology (especially convolutional neural networks) has made significant progress in the field of image recognition, and its application in palm vein recognition has also achieved many results.

[0008] At present, the mainstream deep learning model is to directly perform feature learning and classification on the collected palm vein ROI images. For example, in "A Novel Palm Vein Recognition Method Based on Deep ConvolutionalNeural Networks" (INFORMATICA, 2021, Vol. 32, No. 4, 687–708), pre-trained CNN models (AlexNet, VGG-16 and VGG-19) are used for palm vein image classification and recognition. Among them, AlexNet contains 5 convolutional layers and 3 fully connected layers, and VGG uses VGG-16 (13 layers of convolution) and VGG-19 (16 layers of convolution). Using the corresponding pre-trained models of the above models for fine-tuning, the recognition accuracy reached more than 96% (VGG-19 (98.5%), VGG-16 (97.5%), AlexNet (96%)).

[0009] The paper "A Performance Evaluation of Classic Convolutional Neural Networks for 2D and 3D Palmprint and Palm Vein Recognition" systematically evaluates the effectiveness of classic convolutional neural networks (CNNs) for palm vein recognition (including 2D palmprints, 3D palmprints, and palm veins). These models include AlexNet, VGG, Inception_v3, ResNet, DenseNet, and EfficientNet. The paper's validation data demonstrates that, under mixed data sources, classic CNN models outperform traditional methods in palm vein recognition.

[0010] The existing palm vein recognition methods have defects, including:

[0011] 1. Dedicated Algorithm Based on Palm Vein Structural Features

[0012] Recognition methods based on palm vein structural features require palm vein image preprocessing, palm vein feature design, and dedicated feature extraction algorithms. These methods have low overall recognition accuracy. For example, recognition is poor on low-quality images and is easily affected by factors such as noise and lighting. Dimensionality reduction methods can also miss important information. Consequently, traditional methods suffer from low extraction efficiency and accuracy.

[0013] 2. Palm vein recognition method based on deep convolutional neural network

[0014] Currently, palm vein recognition technology primarily relies on capturing palm vein images and extracting vascular characteristics to accurately identify individuals. Public palm vein databases are often sourced from multiple open-source sources, with sample sizes in the thousands. Due to significant variations in acquisition equipment, conditions, and ROI (region of interest) extraction methods, the palm vein images within these databases are highly diverse and unequal.

[0015] A deep convolutional neural network was trained on preprocessed palm vein images. While the final model achieved a Rank-1 recognition accuracy of over 96%, the model's generalization ability was limited by the size of the training data and the non-uniformity of the image ROIs. Testing using a palm vein dataset not previously used for model training also reduced Rank-1 recognition accuracy, sometimes as low as around 80%.

[0016] While these two mainstream technologies already offer high palm vein recognition accuracy, they are still limited by inconsistent multi-source data collection and insufficient ROI standardization. Therefore, how to further improve palm vein recognition accuracy through data standardization and model transfer learning with limited training data has become a pressing technical challenge. Summary of the Invention

[0017] In response to the shortcomings of the existing technology, the present invention provides an improved method and system for palm vein recognition based on transfer learning, which not only improves training accuracy but also enables the model to have better generalization ability, especially better adaptability when applied in actual scenarios.

[0018] The present invention discloses an improved method for palm vein recognition based on transfer learning, comprising:

[0019] Obtain multiple public palm vein datasets and perform data shuffling on them;

[0020] Perform image augmentation and ROI extraction on the shuffled public palm vein data, and use the obtained palm vein ROI data as the pre-training dataset;

[0021] Pre-training the deep convolutional neural network model based on the pre-training dataset to obtain a pre-training model;

[0022] Use unified collection equipment to collect small-scale palm vein sample data;

[0023] Perform image augmentation and ROI extraction on the small-scale palm vein sample data, and use the obtained palm vein ROI data as the fine-tuning training dataset;

[0024] The transfer learning method is used to fine-tune the pre-trained model based on the fine-tuning training dataset to obtain a fine-tuned model.

[0025] As a further improvement of the present invention, the image augmentation processing includes random image rotation, random image scaling, random translation, random shearing, random contrast adjustment, random brightness adjustment, random gamma adjustment, Gaussian blur, and noise addition.

[0026] As a further improvement of the present invention, the public palm vein data and the small-scale palm vein sample data after shuffling are subjected to image augmentation processing and ROI extraction using the same image augmentation processing method and ROI extraction method.

[0027] As a further improvement of the present invention, the pre-trained model locks the base layer during fine-tuning training, and only fine-tunes the model parameters of the first few layers of the classification output layer.

[0028] As a further improvement of the present invention, the model layers before blocks 3 of the pre-trained model EfficientNet b5 are locked.

[0029] As a further improvement of the present invention, it also includes:

[0030] A recognition effect evaluation dataset is constructed to evaluate the EER and TAR@FAR=0.50% indicators of the fine-tuned model.

[0031] The present invention also discloses an improved palm vein recognition system based on transfer learning, which is used to implement the above-mentioned improved palm vein recognition method based on transfer learning, including:

[0032] The shuffling module is used to shuffle multiple public palm vein datasets;

[0033] The first image augmentation processing and ROI extraction module is used to perform image augmentation processing and ROI extraction on the shuffled public palm vein data, and the obtained palm vein ROI data is used as a pre-training dataset;

[0034] A pre-training module is used to pre-train the deep convolutional neural network model based on the pre-training dataset to obtain a pre-trained model;

[0035] The collection module is used to collect small-scale palm vein sample data using a unified collection device;

[0036] The second image augmentation processing and ROI extraction module is used to perform image augmentation processing and ROI extraction on the small-scale palm vein sample data, and the obtained palm vein ROI data is used as the fine-tuning training dataset;

[0037] The fine-tuning module is used to fine-tune the pre-trained model based on the fine-tuning training dataset using a transfer learning method to obtain a fine-tuned model.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] 1. This invention uses multiple open data sources for palm vein data shuffling and image augmentation to increase data diversity, thereby pre-training deep convolutional neural network models such as EfficientNet. This allows the pre-trained models to have better generalization and basic recognition capabilities.

[0040] 2. This invention uses a unified near-infrared device to simulate actual application scenarios, collect a small amount of data samples and fine-tune the model training, solving the contradiction between the need for a large number of training samples in deep convolutional neural network training and the difficulty of data collection;

[0041] 3. The present invention uses a consistent ROI method to intercept key areas of the palm vein sample data for pre-training and fine-tuning training, and uses these ROI sample data to pre-train the model and fine-tune training to ensure the palm vein feature coverage of the training samples as much as possible;

[0042] 4. This invention locks the base layer during fine-tuning training of deep convolutional neural network models such as EfficientNet, and only fine-tunes the model parameters of the first few layers of the classification output layer. This makes the pre-trained model more adaptable to specified near-infrared acquisition equipment and actual application scenarios, effectively reducing the false recognition rate and rejection rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a flowchart of the improved palm vein recognition method based on transfer learning disclosed in the present invention. DETAILED DESCRIPTION

[0044] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0045] The present invention is described in further detail below with reference to the accompanying drawings:

[0046] Existing deep convolutional neural network methods offer advantages in the accuracy of palm vein recognition model training, but they still place high demands on the quantity and quality of training data. Compared to model training challenges such as facial recognition, palm vein recognition faces significant challenges in obtaining both the quantity and quality of training data samples. First, palm vein data requires specialized near-infrared camera terminals for collection, and such publicly available data is relatively rare. Second, these near-infrared cameras require different standards for wavelength range, resolution, and light source configuration, resulting in significant variation in the quality of publicly available data.

[0047] In order to overcome the difficulties of small number of data samples and large quality differences when training deep convolutional neural networks for palm vein recognition, the present invention provides an improved palm vein recognition method based on transfer learning. On the basis of the existing palm vein recognition method based on deep convolutional neural networks, it proposes to use a unified infrared camera to collect small-scale high-quality palm vein samples; use special algorithms such as ROI to extract ROI from the original image to ensure the standardization of image areas; use transfer learning to fine-tune the pre-trained model to further improve the recognition accuracy; thereby improving the training accuracy while enabling the model to have better generalization ability, especially better adaptability when applied in actual scenarios.

[0048] Specifically:

[0049] like Figure 1 As shown, the present invention provides an improved method for palm vein recognition based on transfer learning, comprising:

[0050] Step 1: Obtain multiple public palm vein datasets and perform data shuffling on them to improve data diversity.

[0051] Step 2: Perform image augmentation on the shuffled public palm vein data to further improve data diversity. Image augmentation includes random image rotation, random image scaling, random translation, random cropping, random contrast adjustment, random brightness adjustment, random gamma adjustment, Gaussian blurring, and noise addition.

[0052] Step 3: Perform ROI extraction on the image after image augmentation processing, and the obtained palm vein ROI data is used as a pre-training data set; it should be noted that: Step 3 and Step 2 can also be switched in order, that is, ROI extraction is performed first, and then image augmentation processing is performed.

[0053] Step 4: Pre-train the deep convolutional neural network model based on the pre-training dataset to obtain a pre-trained model to obtain basic palm vein recognition capabilities;

[0054] Step 5: Collect small-scale palm vein sample data using a unified collection device; the number of samples of the small-scale palm vein sample data is not less than 100;

[0055] Step 6: Perform image augmentation and ROI extraction on the small-scale palm vein sample data, and use the obtained palm vein ROI data as the fine-tuning training dataset; wherein, the image augmentation and ROI extraction in step 6 are the same as those in steps 2 to 3;

[0056] Step 7: Use transfer learning to fine-tune the pre-trained model based on the fine-tuning training dataset to obtain a fine-tuned model to improve the recognition accuracy of the model on new data;

[0057] Step 8: Construct a recognition effect evaluation dataset to evaluate the EER and TAR@FAR=0.50% indicators of the fine-tuning model.

[0058] Based on this, the present invention clarifies the relationship between data preprocessing, feature extraction and model fine-tuning, which can improve the generalization of the model and enhance the recognition effect under limited sample data.

[0059] The present invention provides an improved palm vein recognition system based on transfer learning, which is used to implement the above-mentioned improved palm vein recognition method based on transfer learning, including:

[0060] A shuffling module, used to implement the above step 1;

[0061] The first image augmentation processing and ROI extraction module is used to implement the above steps 2 to 3;

[0062] Pre-training module, used to implement the above step 4;

[0063] Acquisition module, used to implement the above step 5;

[0064] A second image augmentation processing and ROI extraction module is used to implement the above step 6;

[0065] A fine-tuning module for implementing the above step 7;

[0066] Evaluation module, used to implement the above step 8.

[0067] Example:

[0068] The present invention provides an improved method for palm vein recognition based on transfer learning, comprising:

[0069] S1. Obtain multiple public palm vein datasets and perform data shuffling on them to improve the diversity of training data.

[0070] Specifically:

[0071] Multiple public palm vein datasets are shown in Table 1. The public palm vein datasets 1-4 in Table 1, a total of 1204 palm sample data, are aggregated into a mixed dataset PalmVeinMix, and the public palm vein dataset 5 (VERA) is retained as the fine-tuning training dataset.

[0072] Table 1

[0073] Dataset number Dataset Palm number Data volume Image size 1 CASIA 200 7200 768×576 2 FYOD 320 640 800×600 3 PLUSVein 84 840 1280×1024 4 Tongji 600 12000 800×600 5 VERA 220 2200 480×680

[0074] The public palm vein dataset 5 (VERA dataset) was selected as the verification data because:

[0075] 1. The license agreement used by this dataset is for research use only and cannot be used as a commercial training model;

[0076] 2. The sensor used in this dataset uses near-infrared (NIR) illumination with a wavelength of 940 nanometers and is equipped with an Imaging Source camera with a Sony ICX618 sensor. A total of 110 subjects were collected, including 40 women and 70 men, aged 18 to 60, with an average age of 33. The data were collected in two different indoor locations. The first 78 subjects were from one location, and the remaining 32 were from the other. The uniformity of the collection equipment and the diversity of the population and environment of this dataset are guaranteed.

[0077] S2. Using traditional image processing methods and a specially designed ROI extraction algorithm (e.g., "An Improved Method for Extracting Regions of Interest (ROIs) in Palm Print and Palm Vein Recognition," Patent No. CN119027652A), obtain a standardized palm vein region ROI image;

[0078] Specifically:

[0079] Using this ROI method, the palm vein region of interest (ROI) images were uniformly extracted to obtain the ROI dataset palm_vein_poi_images, which contains 1204 palm ROI samples and 20680 image files.

[0080] S3, perform image augmentation processing on the data of S2 to further improve data diversity;

[0081] Specifically:

[0082] Data augmentation technology is used to improve data diversity and quantity. The number of images of each palm vein sample after augmentation reaches 100, and the pre-training dataset palm_vein_poi_augment_images is obtained; among them, the image augmentation processing includes random image rotation (-10°~10°), random image scaling (size varies between 90% and 110%), random translation (translation amount between ±%5), random shearing (angle -8°~8°, cropping up to 5% of the area), random adjustment of contrast {adjustment area (0.8, 1.2)}, random adjustment of brightness {adjustment area (0.9, 1.1)}, random adjustment of gamma {adjustment area (0.8, 1.2)}, Gaussian blur, and adding noise {range (0, 0.02*255)}.

[0083] S4. Pre-training a deep convolutional neural network model based on a pre-training dataset to obtain a pre-trained model to acquire basic palm vein recognition capabilities;

[0084] Specifically:

[0085] The pre-training dataset is split into training, validation, and test datasets with a split ratio of 70% for train, 15% for validate, and 15% for test. The standard EfficientNet b5 model is used as the base model and pre-trained using the above ROI data to obtain the pre-trained model / efficientnet_b5 / model_best.pth.tar.

[0086] S5. Use a unified collection device to collect small-scale palm vein sample data, namely the public palm vein dataset 5 (VERA dataset) in Table 1;

[0087] S6. Perform the same image augmentation and ROI extraction as S2-S3 on the small-scale palm vein sample data, and obtain the standard palm vein ROI data as the fine-tuning training dataset;

[0088] Specifically:

[0089] For the VERA dataset collected using equipment of uniform specifications and scenarios, the datasets S2 to S3 were also constructed to obtain the vera_finetuning_dataset dataset, which has a total of 220 samples (numbered 001-220), with 100 augmented image data for each sample; samples 001-144 in the vera_finetuning_dataset dataset were taken as fine-tuning data to obtain the vera_finetuning_known dataset, from which the train, validate, and test datasets used for fine-tuning were extracted in proportions of 70%, 15%, and 15%; samples 145-220 in the vera_finetuning_dataset dataset were used as unknown data vera_finetuning_unknown (the sample data in this dataset does not participate in pre-training or fine-tuning training, but only in recognition effect evaluation);

[0090] S7. Use transfer learning to fine-tune the pre-trained model based on the fine-tuning training dataset to obtain a fine-tuned model to improve the recognition accuracy of the model on new data;

[0091] Specifically:

[0092] Use the pre-trained model obtained in step 4: / efficientnet_b5 / model_best.pth.tar, and fine-tune the model on the vera_finetuning_known dataset. During fine-tuning, freeze the model parameters before block 3, block 4, and block 5 respectively to obtain the fine-tuned model:

[0093] efficientnet_b5.sw_in12k_ft_in1k_best_finetuning_model_block3.pth,

[0094] efficientnet_b5.sw_in12k_ft_in1k_best_finetuning_model_block4.pth

[0095] efficientnet_b5.sw_in12k_ft_in1k_best_finetuning_model_block5.pth

[0096] S8. Construct a recognition effect evaluation dataset to evaluate the EER and TAR@FAR=0.50% indicators of the fine-tuned model;

[0097] Specifically:

[0098] a_finetuning_known is used as the known data set knowndata, vera_finetuning_unknown is used as the unknown data set, and 1000 pairs of positive samples and 10000 pairs of negative sample data pairs are randomly constructed. The above models are subjected to EER and TAR@FAR=0.50% evaluation, the results are shown in Table 2 As shown:

[0099] Table 2

[0100]

[0101]

[0102] According to the model recognition performance evaluation table above, after fine-tuning the model layers before blocks 3 of EfficientNet b5, the model's generalization ability improved by nearly 15 percentage points, reducing the error rate from 16.4% to 1.7%. With a false positive rate below 0.5%, the correct recognition rate increased from 56.8% to 95.4%. In financial scenarios where extremely low FAR (e.g., 0.50%) is essential, this model has high security.

[0103] Based on the above evaluation data, it can be seen that palm vein data obtained under a set of unified standard near-infrared palm vein collection equipment and ROI method, after fine-tuning the parameters near the output layer of the pre-trained model, retains the basic palm vein recognition capabilities of the pre-trained model while having better adaptability to specific near-infrared palm vein collection equipment, greatly improving the generalization ability of the basic model in practical applications.

[0104] The above are merely preferred embodiments of the present invention and are not intended to limit the present invention. Those skilled in the art will readily appreciate that various modifications and variations of the present invention are possible. Any modifications, equivalent substitutions, or improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the present invention.

Claims

1. An improved palm vein recognition method based on transfer learning, characterized in that: include: Obtain multiple public palm vein datasets and perform data shuffling on them; Perform image augmentation and ROI extraction on the shuffled public palm vein data, and use the obtained palm vein ROI data as the pre-training dataset; Pre-training the deep convolutional neural network model based on the pre-training dataset to obtain a pre-training model; Use unified collection equipment to collect small-scale palm vein sample data; Perform image augmentation and ROI extraction on the small-scale palm vein sample data, and use the obtained palm vein ROI data as the fine-tuning training dataset; The transfer learning method is used to fine-tune the pre-trained model based on the fine-tuning training dataset to obtain a fine-tuned model.

2. The improved palm vein recognition method based on transfer learning according to claim 1, characterized in that: The image augmentation processing includes random image rotation, random image scaling, random translation, random shearing, random contrast adjustment, random brightness adjustment, random gamma adjustment, Gaussian blur, and noise addition.

3. The improved palm vein recognition method based on transfer learning according to claim 1, characterized in that: The public palm vein data and small-scale palm vein sample data after shuffling are processed using the same image augmentation processing method and ROI extraction method.

4. The improved palm vein recognition method based on transfer learning according to claim 1, characterized in that: The pre-trained model locks the base layer during fine-tuning training and only fine-tunes the model parameters of the first few layers of the classification output layer.

5. The improved palm vein recognition method based on transfer learning according to claim 4, characterized in that: Lock the model layers before blocks3 of the pre-trained model EfficientNet b5.

6. The improved palm vein recognition method based on transfer learning according to claim 1, characterized in that: Also includes: A recognition effect evaluation dataset is constructed to evaluate the EER and TAR@FAR=0.50% indicators of the fine-tuned model.

7. A palm vein recognition improvement system based on transfer learning, used to implement the palm vein recognition improvement method based on transfer learning as described in any one of claims 1 to 6, characterized in that: include: The shuffling module is used to shuffle multiple public palm vein datasets; The first image augmentation processing and ROI extraction module is used to perform image augmentation processing and ROI extraction on the shuffled public palm vein data, and the obtained palm vein ROI data is used as a pre-training dataset; A pre-training module is used to pre-train the deep convolutional neural network model based on the pre-training dataset to obtain a pre-trained model; The collection module is used to collect small-scale palm vein sample data using a unified collection device; The second image augmentation processing and ROI extraction module is used to perform image augmentation processing and ROI extraction on the small-scale palm vein sample data, and the obtained palm vein ROI data is used as the fine-tuning training dataset; The fine-tuning module is used to fine-tune the pre-trained model based on the fine-tuning training dataset using a transfer learning method to obtain a fine-tuned model.

Images