A Finger Vein Recognition Method Based on Lightweight Network Model
By simplifying ROI region extraction and cropping the MobileNetV2 structure with a lightweight network model, a lightweight finger vein recognition network model LFVRN was designed. This solves the problems of large number of parameters and high training cost of deep learning models, and achieves efficient and energy-saving finger vein recognition.
Patent Information
- Application Number
- CN202210416878.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-04-20
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-04-20
AI Technical Summary
Existing deep learning models for finger vein recognition suffer from large parameter counts, high training costs, and slow processing speeds, leading to insufficient device memory and low recognition efficiency.
A lightweight network model is adopted to simplify the ROI region extraction process. By pruning the first seven bottleneck layers of the pre-trained MobileNetV2 model and combining it with a custom auxiliary layer, a lightweight finger vein recognition network model LFVRN is designed to adapt to ROI extraction methods of different databases. End-to-end training and cross-entropy loss function are used.
It reduces the computational cost of computers, improves recognition speed and accuracy, reduces training costs, and achieves energy-saving and environmentally friendly high-efficiency recognition.
Smart Images

Figure CN114821040B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of finger vein recognition, and in particular to a finger vein recognition method based on a lightweight network model. Background Art
[0002] With the development of information technology and the sophistication of hardware manufacturing processes, people are paying more and more attention to the protection of private information. In the past, people mostly used digital passwords, fingerprints, smart cards, etc. to control access and protect private information. Due to its easy cracking and easy counterfeiting, biometric recognition technology has received widespread attention. It is stable, unique, universal, and difficult to counterfeit and crack. Among them, finger veins have been used in automatic personal identification systems for more than ten years. Its scale and resistance to non-living attacks make it suitable for various commercial and government applications.
[0003] In the past, the network architecture designed based on deep learning methods was deep, with a large number of parameters, high training costs, and prone to insufficient device memory and slow processing speed. At the same time, the miniaturization of neural networks is the development trend of the entire field, which ensures the accuracy of deep models while being smaller in size. Summary of the invention
[0004] In view of the technical problems that most of the existing high-precision deep learning models have a large number of parameters, high training costs, and are prone to insufficient device memory and slow processing speed, the present invention provides a finger vein recognition method based on a lightweight network model. The method simplifies the finger vein recognition process, and the image preprocessing process only requires ROI (region of interest) extraction of the collected samples; the model has a small number of parameters, which reduces the computer calculation cost, thereby improving the recognition speed and reducing the training cost, which is more energy-saving and environmentally friendly.
[0005] To this end, the technical solution of the present invention is a finger vein recognition method based on a lightweight network model, using different ROI region extraction methods for different finger vein databases, the segmented ROI image provided by the FVUSM database is directly used as input data, and the ROI region extraction method of the HKPU, SDUMLA and UTFVP databases includes the following steps:
[0006] S1, using two masks to distinguish the finger area and the background area, assigning the pixel value of the finger area to 1 and the pixel value of the background area to 0, thereby obtaining a black and white binary image;
[0007] S2. Draw a maximum circumscribed rectangular frame on the original image according to the binary image, wherein the height value of the maximum circumscribed rectangular frame is W and the width value is L;
[0008] S3. Determine the upper and lower boundaries of the ROI area and then determine the height of the ROI area, where the upper boundary value of the ROI area is b2, the lower boundary value of the ROI area is b1, and the height value of the ROI area is S. The formula group for determining the upper and lower boundaries is:
[0009] ①b1=W / 4
[0010] ②b2=W-49
[0011] ③S=b2-b1
[0012] S4, use a sliding window with a width of 50 pixels and a height equal to that of region R1 to move from the rightmost column of region R1 to the left, and in the process calculate the sum of the grayscale values of all pixels in the sliding window, denoted as Sum g , and then determine the proximal finger joint position P1 and the distal finger joint position P2, d1 is equal to the distance between P1 and P2, d2 is equal to the distance between P1 and the left border of the ROI area, and determine the finger joint formula group:
[0013] ④
[0014] ⑤P1=argmax(Sum g )floor(2×3 / L)≤P1≤L-49
[0015] ⑥P2=argmax(Sum g )1≤P2≤L / 4
[0016] S5. Determine the left and right boundaries of the ROI area and then determine the width, wherein the left boundary value of the ROI area is g1, and the right boundary value of the ROI area is g2. The left and right boundary formula group is determined as follows:
[0017] ⑦
[0018] ⑧
[0019] S6, segmenting the ROI image from the R1 region according to the upper, lower, left, and right boundary positions;
[0020] The finger vein recognition method adopts a lightweight finger vein recognition network model, which cuts the first seven bottleneck layers of the pre-trained MobileNetV2, followed by a Dropout layer (Dropout1), a convolutional layer (Conv), a normalization layer (BNorm), an ELU layer (ELU), a Dropout layer (Dropout2) and a fully connected layer (FC). The finger vein ROI image is used as a training sample, and the network is trained in an end-to-end manner. Part of the structure of the pre-trained MobileNetV2 model is fine-tuned, and the auxiliary layer is trained from scratch.
[0021] Furthermore, the size of the ROI region image is adjusted to 64*192 pixels before training and testing, where 64 and 192 represent the height and width of the ROI region, respectively.
[0022] Furthermore, the original images in the SDUMLA database have three color channels, while the images in the FVUSM, HKPU and UTFVP databases have only one color channel, which are converted into three-channel images by copying the pixel values of the one color channel to other channels before input.
[0023] Furthermore, the details of the lightweight finger vein recognition network model are as follows:
[0024] (1) The input layer inputs a finger vein image with a size of 224×224×3 (height×width×number of channels). A ROI image with a size of 64×192 is extracted from the original image. Every three ROI images are self-filled and spliced into a square input image with a size of 224×224. For a single-channel image, the pixel value of the channel is copied to other channels before input to convert it into a three-channel image.
[0025] (2) The network from the pre-trained MobileNetV2 model converts the input image into a 7×7×160 size, where the parameter n of the sixth bottleneck layer is set to 1;
[0026] (3) In the auxiliary block, the dropout rates of Dropout1 and Dropout2 are set to 0.2 and 0.1 respectively, Conv has 512 convolutional filters of size 7×7×160, the batch size of BNorm is set to 16, and the final output size of the auxiliary block is 1×1×512;
[0027] (4) FC has C convolutional filters of size 1×1×512, and the output size is 1×1×C; where C represents the number of finger vein categories, and the cross entropy function is used as the loss function for back propagation.
[0028] Furthermore, the training strategy of the lightweight finger vein recognition network model is as follows:
[0029] The lightweight finger vein recognition network model is trained based on four databases: HKPU, SDUMLA, FV-USM and UTFVP. The learning rate is set to 1e-2, the batch size is set to 16, and the back-propagation loss function is the cross-entropy function. Each finger of each individual is regarded as a class. If the database has a training set and a test set, the training set samples will be used for training, and the test set samples will be used for testing. For databases that are not divided into training sets and test sets, each class will be divided into two parts equally. For finger vein classes with an odd number of images, the extra image will be put into the training set.
[0030] The beneficial effect of the present invention is that the method simplifies the finger vein recognition process, and the image preprocessing process only needs to extract the ROI (region of interest) of the collected samples. It integrates multiple ROI extraction methods, making the ROI extraction process on different databases more stable and effective; a finger vein recognition model (LFVRN) based on a lightweight network is designed, and part of the structure of the pre-trained MobileNetV2 is intercepted and fine-tuned, followed by a carefully designed dedicated auxiliary layer and the auxiliary layer is trained from scratch. The overall network parameter volume is only 4.9 million (the traditional network VGG16 has 136 million, and ResNet50 has 24.38 million), which has the advantages of short training time, low energy consumption, fast processing speed, and high recognition accuracy. The model parameter volume is small, which reduces the computer calculation cost, thereby improving the recognition speed and reducing the training cost, which is more energy-saving and environmentally friendly. BRIEF DESCRIPTION OF THE DRAWINGS
[0031] Figure 1 It is a flow chart of finger vein ROI region extraction implemented by the present invention;
[0032] Figure 2 It is the original finger vein image and the corresponding ROI image;
[0033] Figure 3 is the classification accuracy of each model on ImageNet and CIFAR10;
[0034] Figure 4 It is the network structure of LFVRN (Lightweight Finger Vein Recognition Network). DETAILED DESCRIPTION
[0035] The present invention will be further described below in conjunction with embodiments.
[0036] like Figure 1-2As shown, a finger vein recognition method based on a lightweight network model is provided. In order to simplify the extraction steps of the ROI (Region of Interest) region and make the method easy to reproduce, the present invention adopts different ROI region extraction methods for different finger vein databases. The segmented ROI image provided by the FVUSM database is directly used as input data. The ROI region extraction method of the HKPU, SDUMLA and UTFVP databases includes the following steps:
[0037] S1, using two masks to distinguish the finger area and the background area, assigning the pixel value of the finger area to 1 and the pixel value of the background area to 0, thereby obtaining a black and white binary image;
[0038] S2. Draw a maximum circumscribed rectangular frame on the original image according to the binary image, wherein the height value of the maximum circumscribed rectangular frame is W and the width value is L;
[0039] S3. Determine the upper and lower boundaries of the ROI area and then determine the height of the ROI area, where the upper boundary value of the ROI area is b2, the lower boundary value of the ROI area is b1, and the height value of the ROI area is S. The upper and lower boundary formula group a is determined as follows:
[0040] ①b1=W / 4
[0041] ②b2=W-49
[0042] ③S=b2-b1
[0043] S4, using a sliding window with a width of 50 pixels and a height equal to that of region R1, move from the rightmost column of region R1 to the left, and in the process calculate the sum of the grayscale values of all pixels in the sliding window, recorded as, and then determine the proximal finger joint position P1 and the distal finger joint position P2, d1 is equal to the distance between P1 and P2, d2 is equal to the distance between P1 and the left border of the ROI area, and determine the finger joint formula group b:
[0044] ④
[0045] ⑤P1=argmax(Sum g )floor(2×3 / L)≤P1≤L-49
[0046] ⑥P2=argmax(Sum g )1≤P2≤L / 4
[0047] S5. Determine the left and right boundaries of the ROI area and then determine the width, wherein the left boundary value of the ROI area is g1, and the right boundary value of the ROI area is g2, and determine the left and right boundary formula group c:
[0048] ⑦
[0049] ⑧
[0050] S6. Segment the ROI image from the R1 region according to the upper, lower, left, and right boundary positions.
[0051] The present invention integrates a variety of ROI extraction methods, making the ROI extraction process on different databases more stable and effective.
[0052] In this embodiment, the size of the ROI region image is adjusted to 64*192 pixels before training and testing, where 64 and 192 represent the height and width of the ROI region, respectively.
[0053] The original images in the SDUMLA database have three color channels, while the images in the FVUSM, HKPU, and UTFVP databases have only one color channel. Before input, the images are converted into three-channel images by copying the pixel values of the one color channel to other channels.
[0054] The present invention adopts a lightweight finger vein recognition network model. Since accuracy is the main concern of finger vein recognition, the lightweight finger vein recognition network model uses a lightweight pre-trained network with good classification ability as the backbone network. Figure 3 The classification accuracy of each lightweight network on ImageNet and CIFAR10 is shown in Figure 2. The first and second best results are highlighted in bold and underlined, respectively. The classification performance of MobileNetV2 on the two databases is the second best and the best, respectively.
[0055] like Figure 4 As shown, the network designed by the present invention (LFVRN, Lightweight Finger-vein Recognition Network) cuts the first seven bottleneck layers of the pre-trained MobileNetV2, followed by a Dropout layer (Dropout1), a convolutional layer (Conv), a normalization layer (BNorm), an ELU layer (ELU), a Dropout layer (Dropout2) and a fully connected layer (FC). The finger vein ROI image is used as a training sample, and the network is trained in an end-to-end manner. Part of the structure of the pre-trained MobileNetV2 model is fine-tuned, and the auxiliary layer is trained from scratch. The detailed information of the lightweight finger vein recognition network model is as follows:
[0056] (1) The input layer inputs a finger vein image with a size of 224×224×3 (height×width×number of channels). A ROI image with a size of 64×192 is extracted from the original image. Every three ROI images are self-filled and spliced into a square input image with a size of 224×224. For a single-channel image, the pixel value of the channel is copied to other channels before input to convert it into a three-channel image.
[0057] (2) The network from the pre-trained MobileNetV2 model converts the input image into a 7×7×160 size, where the parameter n of the sixth bottleneck layer is set to 1;
[0058] (3) In the auxiliary block, the dropout rates of Dropout1 and Dropout2 are set to 0.2 and 0.1 respectively, Conv has 512 convolutional filters of size 7×7×160, the batch size of BNorm is set to 16, and the final output size of the auxiliary block is 1×1×512;
[0059] (4) FC has C convolutional filters of size 1×1×512, and the output size is 1×1×C; where C represents the number of finger vein categories, and the cross entropy function is used as the loss function for back propagation.
[0060] In this embodiment, the training strategy of the lightweight finger vein recognition network model is as follows:
[0061] The lightweight finger vein recognition network model is trained based on four databases: HKPU, SDUMLA, FV-USM and UTFVP. The learning rate is set to 1e-2, the batch size is set to 16, and the back-propagation loss function is the cross-entropy function. Each finger of each individual is regarded as a class. If the database has a training set and a test set, the training set samples will be used for training, and the test set samples will be used for testing. For databases that are not divided into training sets and test sets, each class will be divided into two parts equally. For finger vein classes with an odd number of images, the extra image will be put into the training set.
[0062] The lightweight network-based finger vein recognition model (LFVRN) extracts part of the structure of the pre-trained MobileNetV2 and makes fine-tuning, followed by a carefully designed dedicated auxiliary layer and training the auxiliary layer from scratch. The total number of network parameters is only 4.9 million (the traditional network VGG16 has 136 million, and ResNet50 has 24.38 million). It has the advantages of short training time, low energy consumption, fast processing speed and high recognition accuracy. The average recognition time is 10.27ms on a system configured with Intel(R)Core(TM)i7-9700K CPU@3.60GHz, 16GB RAM and NVIDIA RTX 2060SUPER 8GB GPU card. The correct recognition accuracy on the four databases of HKPU, SDUMLA, FV-USM and UTFVP can reach 96.98%, 97.75%, 98.58% and 98.61% respectively, and the verification error rates are only 1.90%, 1.42%, 0.61% and 0.83% respectively.
[0063] However, what is described above is only a specific embodiment of the present invention and should not be used to limit the scope of implementation of the present invention. Therefore, the replacement of equivalent components, or equivalent changes and modifications made according to the scope of protection of the present invention should still fall within the scope covered by the claims of the present invention.
Claims
1. A finger vein recognition method based on a lightweight network model, characterized in that: Different ROI region extraction methods are used for different finger vein databases. The segmented ROI image provided by the FVUSM database is directly used as input data. The ROI region extraction method of the HKPU, SDUMLA and UTFVP databases includes the following steps: S1, using two masks to distinguish the finger area and the background area, assigning the pixel value of the finger area to 1 and the pixel value of the background area to 0, thereby obtaining a black and white binary image; S2. Draw a maximum circumscribed rectangular frame on the original image according to the binary image, wherein the height value of the maximum circumscribed rectangular frame is W and the width value is L; S3. Determine the upper and lower boundaries of the ROI area and then determine the height of the ROI area, where the upper boundary value of the ROI area is b2, the lower boundary value of the ROI area is b1, and the height value of the ROI area is S. The formula group for determining the upper and lower boundaries is: ①b1=W / 4 ②b2=W-49 ③S=b2-b1 S4, use a sliding window with a width of 50 pixels and a height equal to that of region R1 to move from the rightmost column of region R1 to the left, and in the process calculate the sum of the grayscale values of all pixels in the sliding window, denoted as Sum g , and then determine the proximal finger joint position P1 and the distal finger joint position P2, d1 is equal to the distance between P1 and P2, d2 is equal to the distance between P1 and the left border of the ROI area, and determine the finger joint formula group: ⑤P1=argmax(Sum g )floor(2×3 / L)≤P1≤L-49 ⑥P2=argmax(Sum g 1≤P2≤L / 4 S5. Determine the left and right boundaries of the ROI area and then determine the width, wherein the left boundary value of the ROI area is g1, and the right boundary value of the ROI area is g2. The left and right boundary formula group is determined as follows: S6, segmenting the ROI image from the R1 region according to the upper, lower, left, and right boundary positions; The finger vein recognition method adopts a lightweight finger vein recognition network model, which cuts the first seven bottleneck layers of the pre-trained MobileNetV2, followed by a Dropout layer Dropout1, a convolution layer Conv, a normalization layer BNorm, an ELU layer ELU, a Dropout layer Dropout2 and a fully connected layer FC. The finger vein ROI image is used as a training sample, and the network is trained in an end-to-end manner. Part of the structure of the pre-trained MobileNetV2 model is fine-tuned, and the auxiliary layer is trained from scratch.
2. According to claim 1, a finger vein recognition method based on a lightweight network model is characterized in that: Before training and testing, the size of the ROI area image is adjusted to 64*192 pixels, where 64 and 192 represent the height and width of the ROI area, respectively.
3. The finger vein recognition method based on a lightweight network model according to claim 2, characterized in that: The original images in the SDUMLA database have three color channels, while the images in the FVUSM, HKPU, and UTFVP databases have only one color channel. Before input, the images are converted into three-channel images by copying the pixel values of the one color channel to other channels.
4. According to claim 3, a finger vein recognition method based on a lightweight network model is characterized in that: The details of the lightweight finger vein recognition network model are as follows: (1) The input layer inputs a finger vein image with a size of 224×224×3 (height×width×number of channels). A ROI image with a size of 64×192 is extracted from the original image. Every three ROI images are self-filled and spliced into a square input image with a size of 224×224. For a single-channel image, the pixel value of the channel is copied to other channels before input to convert it into a three-channel image. (2) The network from the pre-trained MobileNetV2 model converts the input image into a 7×7×160 size, where the parameter n of the sixth bottleneck layer is set to 1; (3) In the auxiliary block, the dropout rates of Dropout1 and Dropout2 are set to 0.2 and 0.1 respectively, Conv has 512 convolutional filters of size 7×7×160, the batch size of BNorm is set to 16, and the final output size of the auxiliary block is 1×1×512; (4) FC has C convolutional filters of size 1×1×512, and the output size is 1×1×C; where C represents the number of finger vein categories, and the cross entropy function is used as the loss function for back propagation.
5. According to claim 4, a finger vein recognition method based on a lightweight network model is characterized in that: The training strategy of the lightweight finger vein recognition network model is as follows: The lightweight finger vein recognition network model is trained based on four databases: HKPU, SDUMLA, FV-USM and UTFVP. The learning rate is set to 1e-2, the batch size is set to 16, and the back-propagation loss function is the cross-entropy function. Each finger of each individual is regarded as a class. If the database has a training set and a test set, the training set samples will be used for training, and the test set samples will be used for testing. For databases that are not divided into training sets and test sets, each class will be divided into two parts equally. For finger vein classes with an odd number of images, the extra image will be put into the training set.
