A deep learning-based finger part multi-modal identity authentication method

Abstract

The application discloses a kind of based on deep learning finger part multimodal identity authentication method, comprising: S1: to original knuckle print feature picture and finger vein feature picture are collected and preprocessed;S2: construct knuckle print feature recognition neural network model, and the knuckle print feature picture after pre-processing is identified;S3: construct finger vein feature recognition neural network model, and the finger vein feature picture after pre-processing is identified;S4: the recognition result of knuckle print feature recognition neural network model is fused with the recognition result of finger vein feature recognition neural network model, matching, and matching result is as final fusion decision result.The application can effectively improve the accuracy, stability and attack resistance of identity authentication.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of image processing and computer vision, and particularly relates to a finger multi-modal identity authentication method based on deep learning. BACKGROUND

[0002] In today's information society, personnel flow frequently and widely, and there is a huge amount of interactive information, how to quickly identify the identity of individuals in so much information is a problem to be faced. Traditional identity authentication is mainly divided into two kinds, the first is based on specific knowledge, such as password, password, etc. The second is based on token type, such as notes, keys and IC cards, etc. However, these identity recognition methods have many disadvantages, such as forgetting password, key loss, notes being stolen, certificate being forged, etc. If the identity is stolen, the impact on individuals or society is immeasurable. In recent years, biometric identity recognition and authentication technology has developed rapidly, attracting more and more researchers' attention, which makes identity authentication more secure and solves many shortcomings of traditional identity authentication.

[0003] Biometric recognition is a technology method of using human body characteristics or identifying behavior characteristics to identify identity, which has the advantages of convenient carrying and no need to remember. Today, biometric recognition technology in the three fields of iris, face and fingerprint has very mature application technology. As a safe and convenient identity authentication technology, biometric recognition technology has gradually replaced the traditional identity authentication method and plays an important role in many occasions.

[0004] However, most of the existing biometric recognition technologies mainly use single modal biometric features, and single modal biometric features may have problems such as loss of feature information, damage, easy to be affected by environment, easy to be forged, etc., which leads to that the single modal recognition effect cannot meet the demand of people on high-performance identity identification. SUMMARY

[0005] In order to solve the defects of the existing biometric recognition technology using single modal biometric features, the application provides a finger multi-modal identity authentication method based on deep learning, which can effectively improve the accuracy, stability and attack resistance of identity authentication.

[0006] The technical scheme adopted by the application to solve the technical problem is: a finger multi-modal identity authentication method based on deep learning, comprising:

[0007] S1: collecting and preprocessing the original knuckle print feature picture and the finger vein feature picture;

[0008] S2: constructing a knuckle print feature recognition neural network model to recognize the preprocessed knuckle print feature picture;

[0009] S3: Construct a finger vein feature recognition neural network model to recognize the preprocessed finger vein feature pictures;

[0010] S4: Fuse and match the recognition results of the knuckle print feature recognition neural network model and the finger vein feature recognition neural network model, and the matching result is used as the final fusion decision result.

[0011] As a further embodiment of the present application, the preprocessing is to input the original knuckle print feature pictures and finger vein feature pictures into the MPRNet multi-stage progressive image restoration network model for denoising.

[0012] As a further embodiment of the present application, the knuckle print feature recognition neural network model is to add knuckle print pooling and frozen training in the twin neural network, use VGG16 as the feature extraction network, and use SeLU as the activation function.

[0013] As a further embodiment of the present application, the finger vein feature recognition neural network model is to add a channel attention mechanism in the ResNet50 residual network classification model, perform compression operation on the finger vein feature pictures obtained by convolution to obtain channel-level finger vein global features, then perform excitation operation on the finger vein global features to learn the relationship between channels and obtain activation values of different channels, and multiply the activation values of each channel by the original finger vein feature pictures to obtain the final features.

[0014] As a further embodiment of the present application, the step S4 is specifically to fuse the knuckle print feature recognition neural network model and the finger vein feature recognition neural network model at the decision level, the fusion rule is and, a dual-modal finger feature recognition system is constituted, fusion decision is made according to the results of separate recognition of knuckle print features and finger vein features, if the matching results of knuckle print features and finger vein features are consistent, the matching is successful; if the matching results of knuckle print features and finger vein features are inconsistent, the matching is failed; and the matching result is used as the final fusion decision result.

[0015] The beneficial effects of the present application include that the finger multi-modal identity authentication method based on deep learning can effectively improve the accuracy of identity authentication, and compared with traditional biological features, knuckle prints and finger veins are more stable and not easy to wear, counterfeit and steal; and the fusion of the two features can complement each other, thereby improving the recognition accuracy, stability and attack resistance of the dual-modal finger feature recognition system. BRIEF DESCRIPTION OF DRAWINGS

[0016] Figure 1 It is a schematic diagram of the overall method of the present application;

[0017] Figure 2A schematic diagram of a twin neural network involved in the present application;

[0018] Figure 3 A schematic diagram of a knuckle pattern pooling process involved in the present application;

[0019] Figure 4 A comparison diagram of ordinary pooling and knuckle pattern pooling of the present application;

[0020] Figure 5 A schematic diagram of a channel attention mechanism involved in the present application;

[0021] Figure 6 A schematic diagram of a finger vein feature recognition neural network structure of the present application. DETAILED DESCRIPTION

[0022] The technical solutions of the present application will be described clearly and completely below with reference to the accompanying drawings. Obviously, the described embodiments are part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0023] In the description of the present application, it should be noted that the orientations or positional relationships indicated by the terms "vertical", "horizontal", "inner", "outer" and the like are based on the orientations or positional relationships shown in the drawings, and are only for the convenience of describing the present application and simplifying the description, and therefore cannot be understood as indicating or implying that the devices or components referred to must have a particular orientation, be constructed and operated in a particular orientation, and therefore cannot be understood as limiting the present application. In addition, the terms "first", "second", "third" are only used to distinguish components, and cannot be understood as indicating or implying relative importance.

[0024] In addition, the technical features involved in different embodiments of the present application described below can be combined with each other as long as they do not conflict with each other.

[0025] Embodiment 1

[0026] A finger part multi-modal identity authentication method based on deep learning, comprising:

[0027] S1: collecting and preprocessing the original knuckle pattern feature picture and the finger vein feature picture;

[0028] Since the existing biometric recognition technology mostly uses traditional image processing methods for pre-processing of biometric feature images, it cannot well preserve the fine edge information of the images, and is time-consuming. Therefore, the original knuckle print feature picture and the finger vein feature picture are transmitted as input to the MPRNet (Multi-Stage Progressive Image Restoration Net) model which has been trained for denoising, and the output is the denoised knuckle print feature picture and the finger vein feature picture. Then, the knuckle print feature picture dataset and the finger vein feature picture dataset are respectively divided into training set and test set according to the ratio of 8:2, and the format of the pictures is bmp. In the above implementation, the denoised picture is clearer than the traditional method, the denoising effect is better, and the information of the picture is better protected.

[0029] S2: Construct a knuckle print feature recognition neural network model to recognize the pre-processed knuckle print feature picture;

[0030] Based on deep metric learning, the knuckle print recognition is studied, and the main network used is a twin neural network. In order to better extract the long stripe texture information of the knuckle print, a knuckle print pooling is proposed, which is added to the twin neural network, effectively solving the problem of insufficient texture feature information extraction of the knuckle print recognition algorithm, and greatly improving the recognition rate. The specific implementation is as follows: Since the knuckle print feature picture is mainly composed of vertical stripes, a 1*N pooling kernel is defined, the average of the pixel values of the corresponding positions on the knuckle print feature picture is taken as the output of the pooling, and then a 1D convolution with a convolution kernel of 3 is performed on the knuckle print feature picture to expand it up and down. Then, through 1x1 convolution and SeLU operation, the introduction of SeLU activation function can solve the problem of gradient disappearance in network training; the obtained knuckle print feature picture is superimposed with the original knuckle print feature picture to obtain the final output result. As shown in Figure 4 The texture information of the knuckle print feature picture is mainly long strips, and using a large square pool window will inevitably merge the pollution information from unrelated areas, while the knuckle print pooling considers a longer and narrower vertical strip range, rather than the entire feature map, avoiding unnecessary connections between positions far apart.

[0031] Regarding the superimposition of the obtained knuckle line feature picture and the original knuckle line feature picture, the final output result is obtained. Specifically, a Siamese neural network, also known as a twin neural network, is a coupled architecture based on two neural networks. In this embodiment, the Siamese neural network takes two pre-processed knuckle line feature pictures as sample inputs and outputs their representations embedded in a high-dimensional space to compare the similarity of the two samples. If two neural networks are used to extract features from the knuckle line feature pictures, the extracted features are likely to be in different domains. Therefore, in this embodiment, two identical neural networks are used to extract features, obtaining a multi-dimensional feature. The feature extraction network used in this embodiment is VGG16, and then the multi-dimensional feature is flattened into one dimension using the flatten method, obtaining a one-dimensional vector of the two input knuckle line feature pictures. Since two identical neural networks are used, they are given the same weight, i.e., weight sharing. Finally, the two one-dimensional vectors are subtracted and the absolute value is summed, which is equivalent to calculating the L1 norm of the two feature vectors. Then, two fully connected layers are used, and the second fully connected layer is connected to a neuron. The result of the neuron is taken as sigmoid, making the value between 0 and 1, representing the similarity of the two input knuckle line feature pictures. When the two inputs point to the same type of picture, the label is 1; when the two inputs point to different types of pictures, the label is 0. In this way, a number of training data are formed. Then, the cross-entropy operation is performed on the output result of the network and the true label, which can be used as the final loss. Finally, the test set is put into the trained knuckle line feature recognition neural network model to identify the knuckle line image. If the similarity is 1, it is considered that the two pictures are the knuckle lines of the same person, and the identity matching is successful. If it is 0, the matching fails, and the final recognition result is obtained.

[0032] Preferably, in order to speed up the training efficiency, a frozen training is added between the feature extraction network and the comparison network. Freezing is mainly achieved by adding a filter to the optimizer, which filters the parameters of the frozen feature extraction network. It mainly borrows the idea of transfer learning. In the freezing stage, the backbone of the model is frozen, so the parameters of the feature extraction network do not change, the occupied memory is small, and only the network is fine-tuned. In the unfreezing stage, the backbone of the model is not frozen, the feature extraction network will change, the occupied memory is larger, and all the parameters of the network will change. Because the features extracted by the backbone of the Siamese neural network are general, freezing the training can speed up the training efficiency and prevent the weights from being damaged.

[0033] S3: Construct a knuckle line feature recognition neural network model to identify the pre-processed knuckle line feature picture;

[0034] Using ResNet50 (Residual net) classification model, in order to enhance the feature value useful for classification, inhibit useless feature value, add improved channel attention mechanism in the finger vein feature recognition neural network. The preprocessed finger vein feature picture data set is imported into the finger vein feature recognition neural network model for training, and the mature finger vein feature recognition neural network model is used for identity recognition of finger vein image, and finally the recognition result is obtained.

[0035] Specifically, the channel attention mechanism is added in the ResNet50 residual network, the finger vein feature picture obtained by convolution is compressed to obtain the channel level finger vein global feature, then the finger vein global feature is excited to learn the relationship between channels, and the activation value of different channels is obtained. Multiply the activation value of each channel by the original finger vein feature picture to obtain the final feature.

[0036] The compression operation is to encode the entire spatial feature in a channel into a global feature, which is realized by global average pooling. In order to reduce the complexity of the model and improve the generalization ability, the bottleneck structure containing two fully connected layers is adopted in the excitation operation, wherein the first fully connected layer plays a role of dimension reduction, and the dimension reduction coefficient r is a hyperparameter. Then, the SeLU activation function is activated, and the last fully connected layer restores the original dimension. Finally, the learned activation value (sigmoid activation, value 0-1) of each channel is multiplied by the original feature. The preprocessed finger vein feature picture data set is imported into the finger vein feature recognition neural network model for training, and the mature finger vein feature recognition neural network model is used for identity recognition of finger vein image, and finally the recognition result is obtained.

[0037] S4: The recognition results of the knuckle line feature recognition neural network model and the finger vein feature recognition neural network model are fused and matched, and the matching result is used as the final fusion decision result:

[0038] The knuckle line feature recognition neural network and the finger vein feature recognition neural network are fused at the decision level, the fusion rule is and, a dual-mode finger feature recognition system is constructed, the test sample is fused according to the recognition result of the knuckle line feature and the finger vein feature, if the matching results of the knuckle line feature and the finger vein feature are consistent, the matching is successful; if the matching results of the knuckle line feature and the finger vein feature are inconsistent, the matching fails; the matching result is used as the final fusion decision result.

[0039] Obviously, the above embodiments are merely example for clearly illustrating but not limitation to the embodiments. Based on the above description, other different forms of changes or variations can be made by those skilled in the art. Here, all the embodiments need not and can not be enumerated. The obvious changes or variations derived from the above description are still within the protection scope of the present application.

Claims

1.A deep learning based multi-modal identity authentication method for finger part, characterized in that, The method comprises the following steps: S1: collecting and preprocessing original knuckle line feature pictures and finger vein feature pictures; S2: constructing a knuckle line feature recognition neural network model to recognize the preprocessed knuckle line feature pictures; A 1*N pooling kernel is defined, the pixel values of the corresponding positions on the knuckle line feature pictures are averaged to obtain the output of the pooling, then a 1D convolution with a convolution kernel of 3 is performed on the knuckle line feature pictures to expand them upward and downward, then a 1*1 convolution and a SeLU operation are performed, the introduction of the SeLU activation function can solve the problem of gradient disappearance in network training; the obtained knuckle line feature pictures are superimposed with the original knuckle line feature pictures to obtain the final output result; The obtained knuckle line feature pictures are superimposed with the original knuckle line feature pictures to obtain the final output result, specifically: two identical neural networks are used for feature extraction to obtain a multi-dimensional feature, a VGG16 feature extraction network is used, then the multi-dimensional feature is flattened to one dimension using the flatten method to obtain one-dimensional vectors of the two input knuckle line feature pictures, the same weight is assigned to them, that is, the weight is shared; finally, the two one-dimensional vectors are subtracted and the absolute value is summed, which is equivalent to calculating the L1 norm of the interpolation of the two feature vectors, then two fully connected operations are performed, the second fully connected operation is performed on one neuron, and the result of the neuron is taken as sigmoid to make the value between 0 and 1, representing the similarity of the two input knuckle line feature pictures; when the two inputs point to the same type of picture, the label is 1; when the two inputs point to different types of pictures, the label is 0; then the output result of the network and the real label are subjected to cross-entropy operation as the final loss; finally, the test set is put into the trained knuckle line feature recognition neural network model to perform identity recognition on the knuckle line image, if the similarity is 1, it is considered that the two pictures are the knuckle lines of the same person, and the identity matching is successful, if it is 0, the matching fails, and finally the recognition result is obtained; a frozen training is added between the feature extraction network and the comparison network; S3: constructing a finger vein feature recognition neural network model to recognize the preprocessed finger vein feature pictures; The finger vein feature recognition neural network model is: a channel attention mechanism is added to the ResNet50 residual network classification model, a compression operation is performed on the finger vein feature pictures obtained by convolution to obtain channel-level finger vein global features, then an excitation operation is performed on the finger vein global features to learn the relationship between channels to obtain activation values of different channels, and the activation values of the channels are multiplied by the original finger vein feature pictures to obtain the final features; The compression operation is to encode the entire spatial feature on a channel into a global feature, and the global average pooling is used to realize the encoding; the excitation operation adopts a bottleneck structure containing two fully connected layers, wherein the first fully connected layer plays a role of dimension reduction, the dimension reduction coefficient r is a hyperparameter, then a SeLU activation function is used for activation, and the last fully connected layer restores the original dimension; finally, the learned activation values of each channel are multiplied by the original features; The preprocessed finger vein feature picture dataset is imported into the finger vein feature recognition neural network model for training, the mature finger vein feature recognition neural network model is used for identity recognition of the finger vein image, and finally the recognition result is obtained. S4: The recognition result of the finger joint line feature recognition neural network model is fused and matched with the recognition result of the finger vein feature recognition neural network model, and the matching result is used as the final fusion decision result. 2.The finger part multi-modal identity authentication method based on deep learning according to claim 1, characterized in that, The preprocessing is to input the original finger joint line feature pictures and finger vein feature pictures to the MPRNet multi-stage progressive image restoration network model for denoising. 3.The finger multi-modal identity authentication method based on deep learning according to claim 1, characterized in that, The step S4 is specifically: the finger joint line feature recognition neural network model and the finger vein feature recognition neural network model are fused at the decision level, the fusion rule is and, a dual-mode finger feature recognition system is constituted, the fusion decision is made according to the recognition results of the finger joint line feature and the finger vein feature, if the matching results of the finger joint line feature and the finger vein feature are consistent, the matching is successful; if the matching results of the finger joint line feature and the finger vein feature are inconsistent, the matching fails; and the matching result is used as the final fusion decision result.

Images