Identity recognition system based on finger veins and finger creases
By combining the channel importance of MobileNet network and triple state loss extraction technology, and fusing finger vein and finger crease characteristics, the stability and security problems of single-modal biometric technology are solved, achieving high accuracy and high security identity recognition effects.
Patent Information
- Application Number
- CN202411712996.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-27
- Publication Date
- 2025-03-21
AI Technical Summary
Single-modal biometric technology has risks of loss, damage and forgery of feature information, and it is difficult to meet the needs of high-stability and high-security identity verification systems.
The MobileNet network combining channel importance is used to combine the two characteristics of finger veins and finger creases for identity identification, and the network model is extracted through triple state loss to improve the network's discrimination ability.
It significantly improves the accuracy and security of identity recognition, reduces the misidentification rate and rejection rate, and maintains stable identification performance under different lighting, angles and environments.
Smart Images
Figure CN119672817A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of identity recognition, and in particular to an identity recognition system based on finger veins and finger creases. Background Art
[0002] As people's requirements for authentication become higher and higher, traditional authentication technology can no longer meet current needs. Compared with traditional authentication methods, biometric technology can more effectively protect personal identity information, and its security and effectiveness are significantly better than traditional authentication methods.
[0003] Biometric identification technology refers to the use of the inherent physiological characteristics or behavioral characteristics of the human body to identify a person. As a powerful means of protecting personal identity and material property, it has been widely used in various fields.
[0004] At present, single-modal biometric technology has developed rapidly and has been widely put into production. However, in actual applications, it is found that single-modal biometric features have the risk of feature information loss and damage, and are also easily affected by the environment and changed, and may even be forged, resulting in single-modal biometric information cannot meet people's requirements for highly stable and highly secure identity authentication systems.
[0005] For this reason, multimodal recognition systems were born. Multimodal recognition integrates different biometric information in one system and uses the feature information of different modules to complement each other. It can overcome the shortcomings of single biometric recognition. The use of multiple sensors can also increase the stability of the system. It also has better anti-counterfeiting properties, and ultimately provides a safer and more effective identity authentication system. However, multi-biometric fusion and recognition is still in its infancy, and we still lack in-depth research on many key issues. Finger veins and finger creases are both finger features, with close locations and strong feature correlation. They can be collected simultaneously using simple devices, which greatly reduces costs. At the same time, finger veins have higher security due to their internal features and the characteristics of liveness detection. Summary of the invention
[0006] (I) Purpose of the invention
[0007] In order to improve the ability of convolutional neural networks to identify finger veins and creases, a MobileNet network combining channel importance is proposed, which simultaneously fuses vein and crease features for identity recognition. Specifically, the features extracted by MobileNet are analyzed for channel importance, and unimportant feature channels are compressed to improve the network feature representation capability. Vein and crease features are fused at the feature level and decision level for identity recognition. At the same time, the network model extracted using triplet loss has the characteristics of inter-class distribution and intra-class compactness, which improves the discrimination ability of the network model.
[0008] (II) Technical solution
[0009] An identity recognition system based on finger veins and finger creases includes: a finger vein recognition model for extracting and analyzing finger vein features;
[0010] Finger crease recognition model, used to extract and analyze finger crease features;
[0011] Lightweight model structure, using MobileNet V3 as the backbone network, combined with optimization methods such as deep separable convolution, linear bottleneck structure, reverse residual structure, SE module and H-Swish activation function to improve model efficiency and performance;
[0012] Channel importance evaluation method, which distinguishes important channels from unimportant channels in feature maps by calculating the importance of each channel, thereby optimizing model performance;
[0013] Loss function, the triplet loss method and softmax loss function are used to jointly train the model to bring the same finger vein and finger crease features closer, while pushing away the different finger vein and finger crease features to improve the recognition accuracy.
[0014] Based on the two-level fusion method of finger creases and finger veins, the vein and crease features are fused for identity recognition at the feature level and decision level.
[0015] Preferably, the finger vein recognition model and the finger crease recognition model both adopt the MobileNet V3 network structure, and perform model training respectively to extract finger vein and finger crease features.
[0016] Preferably, the channel importance assessment method includes:
[0017] 1) Determine an input image x;
[0018] 2) Pass it to the feature generator G(.) to get the feature map f. The derived f contains c channels, and then we set the individual channels in f to 0 (ranging from 1 to c) in turn, and keep the rest of the channels unchanged, and get c new feature maps {f, f1, f2, ... fc};
[0019] 3) Input the original feature map f and all c new feature maps f, f1, f2, ... fc into the feature embedding layer E(.), the classification layer C(.) and the softmax layer to obtain the corresponding recognition accuracy r and {r1, r2, ..., rc};
[0020] 4) Finally, the channel distance array dis is obtained by calculating the distance between r and ri (the value range of i is 1 to c). According to the degree of change in the recognition rate, the importance of a certain channel feature to the system recognition can be judged, and the importance of each channel in f is calculated to obtain the c value dis = {dis[1], dis[2], ..., dis[c]} and then select the first 30 channels as important channels.
[0021] Preferably, the three-loss method in the loss function is used for fine-grained image recognition, and by optimizing the loss target, the distance between fingers of different categories is increased, and the distance between fingers of the same category is made closer.
[0022] Preferably, the secondary fusion method based on finger creases and finger veins specifically includes:
[0023] 1) The finger vein and finger crease data from the same finger are randomly combined with the finger vein and finger crease data of other fingers to form a finger vein pair and a finger crease pair respectively. The feature vectors of the finger crease and finger vein are extracted through the network as a=(a1,a2…), b=(b1,b2…);
[0024] 2) a and b are fused at the feature level. Feature level fusion refers to fusing feature vectors obtained from several feature sources; the fused feature c = (a1, a2…, b1, b2…);
[0025] 3) After the softmax function, three different matching scores score1, score2, and score3 are obtained, and the matching scores are fused at the decision layer. Decision layer fusion means that a group of different matchers obtain their own matching results for the current input, and then fuse them to make the final decision. When the total matching score score is obtained, score=a1*score1+a2*score2+a3*score3, a1, a2 and a3 are weighted coefficients, satisfying a1+a2+a3=1, and finally a comprehensive judgment is made.
[0026] Preferably, experimental settings and data sets are also included to collect finger vein and finger crease images from multiple different angles and construct a challenging database for training and testing models.
[0027] Preferably, a comparative experiment is also included to verify the effectiveness of the model improvement and compare the impact of adding different improved modules on the recognition results.
[0028] It can be seen from the above technical solutions that the present application has the following beneficial effects:
[0029] 1. The invention combines the two biometric features of finger vein and finger crease, and utilizes their respective uniqueness to significantly improve the accuracy of identity recognition. Finger vein features are located inside the finger and are difficult to replicate, while finger crease features have a high degree of individual differences. The combination of the two can greatly reduce the false recognition rate and rejection rate, and improve the security and reliability of identity recognition.
[0030] 2. By adopting advanced lightweight model structures such as MobileNet V3, as well as technical innovations such as channel importance evaluation and loss function optimization, this identity recognition technology can maintain stable recognition performance under different lighting, angles and environments, enhancing the robustness and adaptability of the system.
[0031] 3. The use of lightweight model structure in this invention makes the recognition process faster and more efficient, reducing the waiting time of users. At the same time, the dual-modal biometric recognition method provides multiple recognition methods, increasing the flexibility and convenience of users and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS
[0032] Figure 1 It is the overall flow chart of the research scheme of the present invention;
[0033] Figure 2 It is a schematic diagram of the framework of the finger vein recognition model of the present invention;
[0034] Figure 3 It is a schematic diagram of the importance of the channel of the present invention;
[0035] Figure 4 It is a schematic diagram of the secondary fusion strategy of the finger vein and finger crease images of the present invention;
[0036] Figure 5 Schematic diagram of finger vein and finger crease images in the FV dataset of the present invention;
[0037] Figure 6 It is a schematic diagram of the framework of the crease recognition model of the present invention. DETAILED DESCRIPTION
[0038] The following description is merely exemplary in nature and is not intended to limit the present disclosure, application, and use. It should be understood that in all of these figures, the same or similar reference numerals indicate the same or similar parts and features. The various drawings only schematically represent the concepts and principles of the embodiments of the present disclosure, and do not necessarily show the specific dimensions and proportions of the various embodiments of the present disclosure. Specific parts in specific drawings may be exaggerated to illustrate the relevant details or structures of the embodiments of the present disclosure.
[0039] See also Figure 1-6 , an embodiment provided by the present invention:
[0040] An identity recognition system based on finger veins and finger creases includes: a finger vein recognition model for extracting and analyzing finger vein features;
[0041] Finger crease recognition model, used to extract and analyze finger crease features;
[0042] Lightweight model structure, using MobileNet V3 as the backbone network, combined with optimization methods such as deep separable convolution, linear bottleneck structure, reverse residual structure, SE module and H-Swish activation function to improve model efficiency and performance;
[0043] The channel importance evaluation method distinguishes important and unimportant channels in the feature map by calculating the importance of each channel, thereby optimizing model performance; these improvements enable MobileNet V3 to perform well even on resource-constrained devices.
[0044] The pre-trained MobileNet v3 is used as the backbone of this method. The feature generator G(.), feature embedding E(.) and classifier C(.) are all substructures of MobileNet v3. As shown in Table 1, we use f to represent the feature map output by G(.) and e to represent the feature map output by E(.). Among them, G(.) outputs 160 feature channels, and its output features are recorded as fC(.) inputs 160 feature channels and outputs k channels, where k represents the number of finger types.
[0045] Table 1 Network structure parameters
[0046]
[0047]
[0048] Among them, SE: Squeeze-and-Excite structure, compression and excitation;
[0049] NL: Non-Linearity, nonlinear; HS: h-swish activation function, RE: ReLU activation function;
[0050] bottleneck: bottleneck layers, bottleneck layers;
[0051] exp size: expansion factor, expansion parameter;
[0052] The loss function uses the triplet loss method and the softmax loss function to jointly train the model to bring the same finger vein and finger crease features closer and push the different finger vein and finger crease features apart to improve the recognition accuracy. The specific calculation method is shown in the following formula:
[0053]
[0054] By optimizing this loss target, the distance between fingers of different categories can be increased, and the distance between fingers of the same category can be brought closer. f(A) and f(P) have feature maps with the same label; f(A) and f(N) have different feature maps and different labels, where f is the feature generator. d(A,P) represents the distance between the anchor point and the positive sample, and d(A,N) represents the distance between the anchor point and the negative sample. a represents the edge parameter, which increases the gap between the anchor finger pair and the positive finger pair, and between the anchor finger pair and the negative finger pair. By optimizing this objective function, d(A,P)-d(A,N)+A is not greater than 0, and ultimately the finger features of the same category are more clustered, while the finger features of different categories are more dispersed.
[0055] Softmax is used in the multi-classification process. It maps the output of multiple neurons to the interval (0,1), which can be understood as probability, so as to perform multi-classification. Suppose we have an array, Z, Zi represents the i-th element in Z, then the softmax value of this element is:
[0056]
[0057] Furthermore, the finger vein recognition model and the finger crease recognition model both adopt the MobileNet V3 network structure, and perform model training respectively to extract finger vein and finger crease features. The finger vein recognition model and its structure combine the advantages of MobileNet V3 and the importance of channels. The proposed finger vein classification deep network model is as follows Figure 2 As shown. In the first stage, the model is introduced to extract features under MobileNet V3. In the second stage, the importance of each channel is evaluated. In the last stage, the triplet loss is used to pull all the same finger veins closer while pushing different finger veins apart. The goal of the proposed network is to minimize the triplet loss and the soft maximum loss. The smaller the loss value, the stronger the recognition ability of the network. The finger crease recognition model and its structure, its network model is the same as the finger vein structure, but the model training is performed separately.
[0058] Combining the advantages of MobileNet V3 and the importance of channels, the proposed finger crease classification deep network model is as follows Figure 6As shown in the figure. In the first stage, the model is introduced to extract features under MobileNet V3. In the second stage, the importance of each channel is evaluated. The goal of the proposed network is to minimize the triplet loss and softmax loss. The smaller the loss value, the stronger the recognition ability of the network.
[0059] Furthermore, the channel importance evaluation method includes:
[0060] 1) Determine an input image x;
[0061] 2) Pass it to the feature generator G(.) to get the feature map f. The derived f contains c channels, and then we set the individual channels in f to 0 (ranging from 1 to c) in turn, and keep the rest of the channels unchanged, and get c new feature maps {f, f1, f2, ... fc};
[0062] 3) Input the original feature map f and all c new feature maps f, f1, f2, ... fc into the feature embedding layer E(.), the classification layer C(.) and the softmax layer to obtain the corresponding recognition accuracy r and {r1, r2, ..., rc};
[0063] 4) Finally, the channel distance array dis is obtained by calculating the distance between r and ri (the value range of i is 1 to c). According to the degree of change in the recognition rate, the importance of a certain channel feature to the system recognition can be judged, and the importance of each channel in f is calculated to obtain the c value dis = {dis[1], dis[2], ..., dis[c]} and then select the first 30 channels as important channels. Figure 3 shown.
[0064] If channel i is very important, the corresponding result means that acc will change significantly from acc_i. The larger the value of abs(acc-acc_i), the more important the channel is. On the contrary, if the value of abs is small, it means that the network output is not sensitive to the suppression of the i-th channel, and the channel is unimportant.
[0065] Furthermore, the three-loss method in the loss function is used for fine-grained image recognition, which increases the distance between fingers of different categories and makes the distance between fingers of the same category closer by optimizing the loss target.
[0066] Furthermore, the secondary fusion method based on finger creases and finger veins specifically includes:
[0067] 1) The finger vein and finger crease data from the same finger are randomly combined with the finger vein and finger crease data of other fingers to form a finger vein pair and a finger crease pair respectively. The feature vectors of the finger crease and finger vein are extracted through the network as a=(a1,a2…), b=(b1,b2…);
[0068] 2) a and b are fused at the feature level. Feature level fusion refers to fusing feature vectors obtained from several feature sources; the fused feature c = (a1, a2…, b1, b2…);
[0069] 3) After the softmax function, three different matching scores score1, score2, and score3 are obtained. The matching scores are fused at the decision layer. The fusion at the decision layer is that a group of different matchers obtain their own matching results for the current input, and then fuse them to make the final decision. When the total matching score is score, score = a1*score1+a2*score2+a3*score3, a1, a2 and a3 are weighted coefficients, satisfying a1+a2+a3=1, and finally a comprehensive judgment is made. The overall structure is as follows Figure 4 shown.
[0070] Furthermore, it also includes experimental settings and data sets. Finger vein and finger crease images are collected from multiple different angles to build a challenging database for training and testing models. Specifically, three images are collected from three different angles of three cameras, and this process is repeated four times to collect 12 images for each finger. Figure 5 As shown, the first row is the finger vein image, and the second row is the finger crease image. The dataset contains 135 subjects, including 72 females and 63 males.
[0071] During the acquisition process, the finger veins of the subjects are collected, that is, the vein and finger crease database (FV). Since the acquisition device has three cameras at different angles for acquisition, that is, digital images from three angles are collected at the same time, the images in the database are quite different and closer to the actual scene. Database FV provides a more challenging database. The first 6 samples of each type of image are selected as the training set, and the remaining samples are tested as the test set.
[0072] In the experimental environment parameters, the optimizer uses Adam, the learning rate is 0.001, the weight decay is 0.0001, the batch size is set to 8, and the number of training epochs is set to 100. The parameters during the experiment are shown in Table 2.
[0073] Table 2 Experimental parameters
[0074]
[0075] Furthermore, comparative experiments are also included to verify the effectiveness of model improvement and compare the impact of adding different improved modules on the recognition results. Specifically, in order to verify the effectiveness of model improvement, this study conducted experiments. The same training environment and training parameters were used during the experiment. Table 3 shows the impact of different improved components on the recognition accuracy in our proposed finger recognition algorithm. In the finger recognition algorithm, we perform channel importance analysis on the features obtained from the original image and compress unimportant feature channels. In addition, we use a lightweight deep network in the recognition model and train the deep network using a triple loss function. These innovations have improved the accuracy of finger recognition to varying degrees. The baseline represents the direct use of the MobileNet algorithm. In Table 3, we compare the impact of adding improved modules on the recognition results.
[0076] Table 3 Performance comparison of different modules
[0077]
[0078] It will be apparent to those skilled in the art that the invention is not limited to the details of the exemplary embodiments described above and that the invention can be implemented in other specific forms without departing from the spirit or essential features of the invention. Therefore, the embodiments should be considered exemplary and non-limiting in all respects, and the scope of the invention is defined by the appended claims rather than the foregoing description, and it is intended that all variations falling within the meaning and scope of the equivalent elements of the claims be included in the invention. Any reference numeral in a claim should not be considered as limiting the claim to which it relates.
Claims
1. An identity recognition system based on finger veins and finger creases, characterized by: include: Finger vein recognition model, used to extract and analyze finger vein features; Finger crease recognition model, used to extract and analyze finger crease features; Lightweight model structure, using MobileNet V3 as the backbone network, combined with optimization methods such as deep separable convolution, linear bottleneck structure, reverse residual structure, SE module and H-Swish activation function to improve model efficiency and performance; Channel importance evaluation method, which distinguishes important channels from unimportant channels in feature maps by calculating the importance of each channel, thereby optimizing model performance; Loss function, a triplet loss method and a softmax loss function are used to jointly train the model to bring the same finger vein and finger crease features closer together, while pushing away the different finger vein and finger crease features to improve recognition accuracy; Based on the two-level fusion method of finger creases and finger veins, the vein and crease features are fused for identity recognition at the feature level and decision level.
2. According to claim 1, the identity recognition system based on finger veins and finger creases is characterized by: The finger vein recognition model and the finger crease recognition model both adopt the MobileNet V3 network structure and perform model training respectively to extract finger vein and finger crease features.
3. According to claim 1, the identity recognition system based on finger veins and finger creases is characterized by: The channel importance evaluation method comprises: 1) Determine an input image x; 2) Pass it to the feature generator G(.) to get the feature map f. The derived f contains c channels, and then set the individual channels in f to 0 (ranging from 1 to c) in turn, and keep the other channels unchanged, to get c new feature maps {f, f1, f2, ... fc}; 3) Input the original feature map f and all c new feature maps f, f1, f2, ... fc into the feature embedding layer E(.), the classification layer C(.) and the softmax layer to obtain the corresponding recognition accuracy r and {r1, r2, ..., rc}; 4) Finally, the channel distance array dis is obtained by calculating the distance between r and ri (the value range of i is 1 to c). According to the degree of change in the recognition rate, the importance of a certain channel feature to the system recognition can be judged, and the importance of each channel in f is calculated to obtain the c value dis = {dis[1], dis[2], ..., dis[c]} and then select the first 30 channels as important channels.
4. The identity recognition system based on finger veins and finger creases according to claim 1 is characterized in that: The three-loss method in the loss function is used for fine-grained image recognition. By optimizing the loss target, the distance between fingers of different categories is increased and the distance between fingers of the same category is made closer.
5. The identity recognition system based on finger veins and finger creases according to claim 1 is characterized in that: The secondary fusion method based on finger creases and finger veins specifically includes: 1) The finger vein and finger crease data from the same finger are randomly combined with the finger vein and finger crease data of other fingers to form a finger vein pair and a finger crease pair respectively. The feature vectors of the finger crease and finger vein are extracted through the network as a=(a1,a2…), b=(b1,b2…); 2) a and b are fused at the feature level. Feature level fusion refers to fusing feature vectors obtained from several feature sources; the fused feature c = (a1, a2…, b1, b2…); 3) After the softmax function, three different matching scores score1, score2, and score3 are obtained. The matching scores are fused at the decision layer. The fusion at the decision layer is that a group of different matchers obtain their own matching results for the current input, and then fuse them to make the final decision. Wait until the total matching score score is obtained. score=a1*score1+a2*score2+a3*score3, a1, a2 and a3 are weighted coefficients, satisfying a1+a2+a3=1, and finally a comprehensive judgment is made.
6. The identity recognition system based on finger veins and finger creases according to claim 1 is characterized in that: It also includes experimental settings and datasets, collecting finger vein and finger crease images from multiple different angles to build a challenging database for training and testing models.
7. The identity recognition system based on finger veins and finger creases according to claim 7 is characterized in that: It also includes comparative experiments to verify the effectiveness of model improvements and compare the impact of adding different improved modules on the recognition results.