A Transformer-based vein authentication method

Abstract

The present invention discloses a Transformer-based vein authentication method, comprising the following steps: S1, preprocessing a vein image; S2, inputting the preprocessed vein image into a Transformer-based vein authentication model for feature extraction, and outputting the features of the vein image to be authenticated; S3, measuring the similarity between the features of the vein image to be authenticated and the vein features in the sample library, and assuming that the authentication is successful when the similarity is greater than a set threshold. The present invention introduces a pyramid hierarchical structure into a visual Transformer model, which not only has the advantages of a small number of parameters and a fast calculation speed, but is also conducive to extracting multi-level features of the vein image, thereby making the extracted vein features more expressive.

Description

Technical Field

[0001] The invention belongs to the field of computer vision, and in particular relates to a vein authentication method based on Transformer. Background Art

[0002] With the rapid development of the information age, information is transmitted in different forms in daily life, which also brings about problems such as personal information leakage. In people's daily lives, there are more and more occasions where identity authentication is required, such as mobile payment, security systems, etc. People's awareness of information protection is increasing, and safe, effective and convenient identity authentication technology has gradually become a rigid demand. Compared with some traditional identity authentication methods such as passwords and electronic chips, the method of identity authentication using biometrics has gradually become the mainstream because of its safer and more convenient characteristics.

[0003] Biometrics, as unique characteristic information of organisms, have shown strong security performance in identity authentication technology. Biometric recognition technology identifies and verifies a person's identity information based on a set of unique individual characteristics. Common biometric recognition methods include face recognition, fingerprint recognition, iris recognition, gait recognition, etc. However, the above biometric recognition methods all have their limitations. Face recognition may fail due to large differences in age, object occlusion, etc., fingerprint recognition may reduce recognition accuracy due to wear and stains, etc., and iris recognition and gait recognition are not widely used due to low convenience. In addition, as the more widely used biometric recognition modalities - face and fingerprint, they also have security risks. Face pictures are very easy to be collected by people, resulting in problems such as counterfeiting, and fingerprints in some high-definition images are also likely to be stolen.

[0004] Among many biometric identification technologies, vein recognition has become a safe, easy-to-use and widely used biometric identification technology with its high stability, difficulty in counterfeiting, convenience in use and liveness detection. The acquisition of vein images is different from that of faces and fingerprints. It uses near-infrared light of a fixed wavelength range to illuminate the fingers. The living red blood cells in the subcutaneous blood vessels of the fingers absorb the near-infrared light, while the near-infrared light in other non-vascular positions will be transmitted. The vein shadow image, that is, the vein image, can be obtained by using a near-infrared camera. The distribution of veins is mainly determined by genes, which ensures the uniqueness of their characteristics and is not easy to change with age. Therefore, vein recognition technology has high stability. Because veins are hidden under the skin and have special imaging principles, vein images will not be affected by epidermal damage or changes in the external environment, and are difficult to counterfeit. Venous imaging requires sufficient hemoglobin in the blood vessels, which provides a guarantee for liveness detection. At the same time, during the vein authentication process, the user only needs to place his hand on the authentication device to complete the identity authentication, which is very convenient to use. From the above analysis, it can be seen that compared with some widely used biometric recognition technologies such as face recognition and fingerprint recognition, vein recognition not only has the stability and uniqueness of other biometrics, but also has the non-obviousness of vein characteristics. Only special equipment can be used to collect vein image information.

[0005] Research on vein authentication based on convolutional neural networks has made breakthrough progress, which has stimulated researchers' interest in using deep learning models. Prior to this, convolutional neural networks were a common method in computer vision and the mainstream of vein authentication research. However, with the introduction of Visual Transformer (Dosovitskiy A, Beyer L, Kolesnikov A, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale[C] / / International Conference on Learning Representations. 2020.), a large response has been aroused in the field of computer vision research based on deep learning. The most important part of the Visual Transformer is the self-attention mechanism, which assigns an attention value between every two tokens (features of image blocks). This self-attention mechanism that enables global information interaction is an important reason why the Visual Transformer performs well in various visual tasks. After the visual transformer was proposed, researchers tried to explore new implementation methods in multiple fields to obtain results similar to convolutional neural networks; some results even surpassed the convolutional neural network method and have successfully completed tasks such as image classification, image generation (Jiang Y, Chang S, Wang Z. Transgan: Two pure transformers can make one strong gan, and that can scale up [J]. Advances in Neural Information Processing Systems, 2021, 34.), fine-grained classification and image segmentation (Chen J, Lu Y, Yu Q, et al. Transunet: Transformers make strong encoders for medical images segmentation [J]. arXiv preprint arXiv: 2102.04306, 2021.). However, the existing classic ViT model has the problems of large number of model parameters and large amount of calculation. The main reason is that the ViT model directly applies the encoding structure in the Transformer to achieve image classification, and the modules of the same size are stacked in the shallow and deep layers of the model. The straight-tube structure of ViT not only results in a large number of parameters and a large amount of computation, but is also not conducive to extracting multi-level information of the image.Since multi-level information is particularly critical in images with mainly fine-grained textures, models that lack the ability to extract multi-level information have obvious defects. To this end, the Transformer-based vein authentication model proposed in the present invention, which is more suitable for vein image feature extraction, will focus on solving this problem. In the Transformer-based vein authentication model proposed in the present invention, there are a total of three stages. During the feature extraction process of the vein image, the number of blocks will decrease after each stage, thereby reducing the corresponding module size and amount of calculation. At the same time, this method of gradually reducing the number of feature blocks is conducive to extracting multi-level vein image features, which greatly enhances the ability of this model to extract effective information from vein images. Summary of the invention

[0006] The purpose of the present invention is to propose a Transformer-based vein authentication method based on the performance of visual Transformer in vein authentication. The Transformer-based vein authentication model for vein image feature extraction of the present invention focuses on solving this problem. In the Transformer-based vein authentication model proposed in the present invention, there are three stages in total. During the feature extraction process of the vein image, the number of blocks will decrease after each stage, thereby reducing the corresponding module size and amount of calculation. At the same time, this method of gradually reducing the number of feature blocks is conducive to extracting multi-level vein image features, which greatly enhances the ability of the model to extract effective information from vein images.

[0007] The present invention is achieved by at least one of the following technical solutions.

[0008] A Transformer-based vein authentication method comprises the following steps:

[0009] S1, preprocessing the vein image;

[0010] S2, input the preprocessed vein image into the Transformer-based vein authentication model for feature extraction, and output the features of the vein image to be authenticated;

[0011] S3. Measure the similarity between the features of the vein image to be authenticated and the vein features in the sample library. If the similarity is greater than the set threshold, the authentication is considered successful, otherwise the authentication is considered unsuccessful.

[0012] Furthermore, the preprocessing is to divide the vein image into multiple small blocks on average, and input the small blocks into the Transformer-based vein authentication model.

[0013] Furthermore, the feature extraction of the Transformer-based vein authentication model includes three stages. The first stage includes two VT modules, and the second and third stages each include a VT module (Vein Transformer Block, VTBlock); the structure of each VT module is similar, but the parameters such as input dimension, number of tiles, and output dimension are different;

[0014] In the VT module, the vein image tiles are first passed through the position code generator to generate the corresponding position code. The position code is superimposed with the input tile and then sent to the scalable multi-layer perceptron for encoding. Each tile is encoded into three features, namely the first feature Q, the second feature K and the third feature V; the Q, K, and V features of all tiles are then sent to the multi-head attention mechanism for further feature extraction; the features output by the multi-head self-attention mechanism are then sent to the local feature enhancement forward propagation network for feature extraction after passing through the linear layer, and finally the output tile features are passed through the deep and fine structured convolutional neural network layer to achieve dimensionality reduction of feature dimensions;

[0015] The vein image passes through four VT modules in sequence and outputs the final vein features.

[0016] Furthermore, in the training process of the Transformer-based vein authentication model, the vein image is encoded through four VT Blocks in three stages to output vein features; a classifier and two losses are used for supervised training.

[0017] Furthermore, the two losses are center loss and NLL loss.

[0018] Furthermore, during the training process of the Transformer-based vein authentication model, the output vein features are first randomly discarded by 50% of the information when entering the classifier; finally, two losses are used for supervised training to increase the difficulty of training and improve the expressiveness of the features.

[0019] Furthermore, in the training process of the Transformer-based vein authentication model, the vein feature is randomly discarded with 50% of its information and then mapped to the classification space through a linear layer; the dimension of the classification space is the classification dimension of the NLL loss.

[0020] Furthermore, in the training process of the Transformer-based vein authentication model, the vein features are mapped to the classification space through the linear layer and then processed by the activation layer for nonlinear mapping; the output nonlinear mapping processing results are then input into the NLL loss and center loss for supervised training.

[0021] Furthermore, in the feature extraction process of the Transformer-based vein authentication model, the number of input and output patch features in each stage gradually decreases.

[0022] Furthermore, the vein features in the sample library are extracted by the Transformer-based vein authentication model when the user registers, and the vein features saved after all users register constitute the sample library.

[0023] Compared with the existing technology, the present invention has the following effects:

[0024] 1) The present invention adopts a new deep learning structure, namely the Transformer structure, to realize vein authentication. It provides a new research idea and a new implementation scheme for this field;

[0025] 2) Compared with the current visual Transformer model, the present invention pays more attention to local information and is more beneficial to extracting local area features of vein images.

[0026] 3) The present invention introduces the pyramid hierarchical structure into the visual Transformer model, which not only has the advantages of small number of parameters and fast calculation speed, but also is conducive to extracting multi-level features of vein images, thereby making the extracted vein features more expressive. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] Figure 1 The Transformer-based vein authentication model proposed by the present invention;

[0028] Figure 2 Detailed diagram of the Vein Transformer Block (VT Block);

[0029] Figure 3 This is a schematic diagram of the implementation of the tile dimensionality reduction mechanism of the Transformer-based vein authentication model proposed in the present invention. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the scheme of the present invention, the present invention will be further described in detail below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0031] Example 1

[0032] like Figure 1As shown, a Transformer-based vein authentication method includes the following steps:

[0033] S1, preprocessing the vein image;

[0034] S2, input the preprocessed vein image into the trained Transformer-based vein authentication model for feature extraction, and output the features of the vein image to be authenticated;

[0035] The core structure of the Transformer-based vein authentication model is the vein Transformer module, namely the VT module (Vein Transformer Block, VT Block).

[0036] During the training process of the Transformer-based vein authentication model, the vein image is encoded through four VT Blocks in three stages to output vein features; supervised training is performed using a classifier and two classic losses (center loss and NLL loss).

[0037] like Figure 1 As shown, the classifier includes a random information discarding layer (Droupout), a linear mapping layer (LinearLayer) and a nonlinear activation layer (specifically Log SoftMax Layer).

[0038] As a preferred embodiment, in the training process of the Transformer-based vein authentication model, the output vein features are first randomly discarded by 50% of the information when entering the classifier; finally, supervised training is performed using two losses to increase the training difficulty and improve the expressiveness of the features.

[0039] As a preferred embodiment, in the training process of the Transformer-based vein authentication model, the vein feature is randomly discarded 50% of the information and then mapped to the classification space through a linear layer; the dimension of the classification space is the classification dimension of the NLL loss.

[0040] During the training process of the Transformer-based vein authentication model, the vein features are mapped to the classification space through the linear layer, and then processed by the activation layer for nonlinear mapping; the output nonlinear mapping processing results are then input into the NLL loss and center loss for supervised training.

[0041] In the feature extraction process of the Transformer-based vein authentication model, the number of input and output tile features in each stage gradually decreases; from the first stage to the third stage, the number of input tile features is 64, 16, and 4, respectively, and the number of output tile features is 16, 4, and 1, respectively. Its deep and fine structure dimensionality reduction mechanism is specifically implemented by the convolutional neural network layer.

[0042] During the authentication process, the classifier and loss function are removed, and the main part of the model is used to extract vein features.

[0043] Specifically, if Figure 2 The Vein Transformer Block (VTBlock) diagram shown in the figure includes a conditional position encoding part, which uses a position encoding generator (Position Embedding Generator, PEG) to generate conditional position encoding; a self-attention encoding mechanism, which uses an expansion multi-layer perceptron (Expansion Multi-layer Perception, EMLP) to re-encode the block features, a multi-head self-attention mechanism (Multi-Head Self-Attention, MHSA) to achieve self-attention encoding, and a linear layer (Linear) to integrate the features after multi-head self-attention encoding; a local feature enhancement forward propagation mechanism, which uses a local forward propagation network (LocalFeedForwardNetwork, LFFN) to achieve feature re-encoding and local feature enhancement, and the local information forward propagation network is composed of a convolutional neural network with channel attention; a dimensionality reduction mechanism, which uses a deep and fine structured convolutional neural network layer (Deep-Narrow Layer, DN Layer) to achieve dimensionality reduction of the number of features. First, the features of all tiles are spliced ​​into features in the form of images. A depth-wise separable convolutional neural network module with a step size of 2 is used for spatial dimensionality reduction. The length and width are reduced to half of the original. The output after spatial dimensionality reduction is split into tile features. At this time, the number of tile features is reduced to one-fourth of the original. The flowchart is shown in the attached figure. Figure 3 The vein feature extraction flow chart proposed by the present invention is shown in Figure 1 As shown, the information of the vein image is extracted and transmitted through the three stages in the figure, and finally output as the vein image to be authenticated.

[0044] S3. Perform similarity measurement (using cosine distance similarity calculation) on the features of the vein image to be authenticated and the vein features in the sample library (the vein features in the sample library are extracted by the model proposed by the present invention when the user registers, and the vein features saved after all users register constitute the sample library). If it is greater than the set threshold, the authentication is considered successful, otherwise it is considered unsuccessful.

[0045] Example 2

[0046] A Transformer-based vein authentication method comprises the following steps:

[0047] S1. Preprocessing the vein image; the preprocessing is to divide the vein image into multiple small blocks on average, and input the small blocks into the Transformer-based vein authentication model.

[0048] S2. Input the preprocessed vein image into the trained ViT model for feature extraction, and output the vein image features to be authenticated.

[0049] S3. Perform similarity measurement (using cosine distance similarity calculation) on the features of the vein image to be authenticated and the vein features in the sample library (the vein features in the sample library are extracted by the model proposed by the present invention when the user registers, and the vein features saved after all users register constitute the sample library). If it is greater than the set threshold, the authentication is considered successful.

[0050] Example 3

[0051] A Transformer-based vein authentication method comprises the following steps:

[0052] S1, preprocessing the vein image;

[0053] S2. Input the preprocessed vein image into the trained ResT model for feature extraction, and output the vein image features to be authenticated.

[0054] S3. Perform similarity measurement (using cosine distance similarity calculation) on the features of the vein image to be authenticated and the vein features in the sample library (the vein features in the sample library are extracted by the model proposed by the present invention when the user registers, and the vein features saved after all users register constitute the sample library). If it is greater than the set threshold, the authentication is considered successful.

[0055] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications made without departing from the spirit and principles of the present invention shall be equivalent replacement methods and shall be included in the protection scope of the present invention.

Claims

1. A Transformer-based vein authentication method, characterized in that: The following steps are involved: S1. Preprocessing the vein image; the preprocessing is to divide the vein image into a plurality of small blocks on average, and input the small blocks into the Transformer-based vein authentication model; S2, input the preprocessed vein image into the Transformer-based vein authentication model for feature extraction, and output the features of the vein image to be authenticated; The feature extraction of the Transformer-based vein authentication model includes three stages. The first stage includes two VT modules, and the second and third stages each include one VT module. The structure of each VT module is similar, but the input dimension, number of tiles, and output dimension are different. In the VT module, the vein image tiles are first passed through the position code generator to generate the corresponding position code; the position code is superimposed with the input tile and then sent to the scalable multi-layer perceptron for encoding. Each tile is encoded into three features, namely the first feature Q, the second feature K and the third feature V; the Q, K and V features of all tiles are then sent to the multi-head attention mechanism for further feature extraction; the features output by the multi-head self-attention mechanism are then sent to the local feature enhancement forward propagation network for feature extraction after passing through the linear layer, and finally the output tile features are passed through the deep and fine structured convolutional neural network layer to achieve dimensionality reduction; The vein image passes through four VT modules in sequence to output the final vein features; During the training process of the Transformer-based vein authentication model, the vein image is encoded through four VTBlocks in three stages to output vein features; Supervised training using a classifier and two losses; In the training process of the Transformer-based vein authentication model, the vein features are randomly discarded with 50% information and then mapped to the classification space through a linear layer; the dimension of the classification space is the classification dimension of the NLL loss; S3. Measure the similarity between the features of the vein image to be authenticated and the vein features in the sample library. If the similarity is greater than the set threshold, the authentication is considered successful, otherwise the authentication is considered unsuccessful.

2. According to claim 1, a Transformer-based vein authentication method is characterized in that: The two losses are center loss and NLL loss.

3. According to claim 1, a Transformer-based vein authentication method is characterized in that: During the training process of the Transformer-based vein authentication model, the output vein features are first randomly discarded by 50% of the information when entering the classifier; finally, supervised training is performed using two losses to increase the training difficulty and improve the expressiveness of the features.

4. According to claim 1, a Transformer-based vein authentication method is characterized in that: During the training process of the Transformer-based vein authentication model, the vein features are mapped to the classification space through the linear layer, and then processed by the activation layer for nonlinear mapping; the output nonlinear mapping processing results are then input into the NLL loss and center loss for supervised training.

5. According to claim 1, a Transformer-based vein authentication method is characterized in that: During the feature extraction process of the Transformer-based vein authentication model, the number of input and output patch features gradually decreases at each stage.

6. A Transformer-based vein authentication method according to any one of claims 1 to 5, characterized in that: The vein features in the sample library are extracted by the Transformer-based vein authentication model when the user registers. The vein features saved by all users after registration constitute the sample library.

Images