Palm vein recognition method based on GAttention neural network

By using GAttention neural network and self-attention mechanism in palm vein recognition, combined with Gabor filter, the problem of poor robustness to noise and lighting changes in the existing technology is solved, unsupervised learning and efficient recognition are achieved, and the Capture ability of vein shape and structure.

CN118918618APending Publication Date: 2024-11-08CHONGQING TECH & BUSINESS UNIV +2
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202410912648.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-09
Publication Date
2024-11-08

AI Technical Summary

Technical Problem

The existing encoding-based palm vein recognition method is poorly robust to noise, posture changes and lighting changes, and cannot effectively capture the overall shape and context information of the palm vein. The deep learning-based method relies on supervised training processes and lacks Unsupervised learning ability.

Method used

Using a method based on GAttention neural network, combining fixed and adaptive filters, adding self-attention mechanisms, optimizing Gabor filter parameters through unsupervised training process, extracting venous texture features, and using self-attention module to learn long-term in venous images Dependency, generate global receptive fields.

Benefits of technology

It improves recognition accuracy and generalization ability, simplifies the training process, and can effectively identify palm veins without class labels, enhancing the robustness of the overall shape and structural information of the venous image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918618A_ABST
    Figure CN118918618A_ABST
Patent Text Reader

Abstract

The invention belongs to the technical field of palm vein recognition, and particularly discloses a palm vein recognition method based on a GAttention neural network. Comprising the following steps: step 1, establishing a neural network formed by Gabor filters and an attention module, step 2, utilizing a palm vein image training set to carry out unsupervised training on the neural network to determine parameters of each Gabor filter, enabling a filter of an L1 layer to be a two-dimensional filter with a fixed size, enabling a filter of an L2 layer to be a two-dimensional filter with an adaptive size, and enabling each Gabor filter to be a two-dimensional filter with an adaptive size; the method comprises the following steps: aiming at selecting a filter response of the maximum magnitude, and carrying out adaptive direction calculation and multi-scale Gabor filter calculation based on a palm vein training set; and step 3, extracting a feature vector of the input palm vein image through the trained neural network, and identifying or verifying the input palm vein image based on the feature vector.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of palm vein recognition, and specifically discloses a palm vein recognition method based on GAttention neural network. Background Art

[0002] Common and commonly used biometric features used for identity authentication include face, fingerprint, and iris. In contrast, vein features are hidden under human skin and therefore have the following advantages. 1) High stability: Veins are located under the skin of human fingers, so the verification results are not easily degraded by skin humidity. Generally speaking, they are not easily damaged and contaminated by external factors. 2) High security and privacy: Since veins are difficult to observe under visible light, it becomes more difficult to obtain vein patterns without user consent. Therefore, it becomes more difficult to forge vein patterns to attack systems based on vein imaging. 3) Liveness detection: Human skin includes the outermost epidermis, dermis, and subcutaneous layer, which contains subcutaneous veins and arteries. Different skin layers respond differently to different wavelengths of light. The blood returning to the heart through the veins contains deoxyhemoglobin, which absorbs light in the near-infrared part of the spectrum, so vein patterns can be recorded based on near-infrared cameras. Since deoxyhemoglobin only exists in active bodies, vein patterns are a biometric pattern with intrinsic liveness detection.

[0003] Coding-based methods typically involve applying a set of filters to the image, quantizing the magnitude or phase of the filter responses, and encoding the results to compute the biometric template, which is then compared using a global matching procedure based on the Hamming distance. Coding-based methods can be divided into two categories: single-direction-based methods and multiple-direction-based methods.

[0004] The basic principle of local texture description is to extract texture features of local areas from the image, and then use these features to identify or classify the target. First, the image is divided into several local areas or blocks. These local blocks are usually fixed-size windows, and can also be obtained using image segmentation algorithms (such as region growing or region segmentation). Each block represents a local area of ​​the image. Then feature extraction For each local block, texture features are extracted to describe the local texture information of the block. Secondly, feature encoding is usually required to represent the extracted texture features as a vector of fixed dimension. This can be achieved by generating a histogram, PCA (principal component analysis) or clustering technology. Model training Using the extracted and encoded texture feature vector, a machine learning model such as support vector machine (SVM), random forest, neural network, etc. can be trained for object recognition or image classification tasks. The model will learn how to distinguish different categories based on texture features. Finally, recognition or classification For new images or targets, the same segmentation, feature extraction and encoding process is used, and then the trained model is used to recognize or classify them.

[0005] In the palm vein recognition method based on deep learning, CNN is usually used as the most common method for image recognition in deep learning. It includes multiple convolutional layers, pooling layers, and fully connected layers to extract features from images and classify them. In image recognition, the attention mechanism is used to focus on different areas of the image to obtain better feature representation.

[0006] However, the main disadvantage of encoding-based methods is that they involve using a global matching method based on Hamming distance to compare biometric templates. Using a global matching method based on Hamming distance, the system is less robust to factors such as noise, posture changes, and illumination changes. Even small changes in the biometric template may cause matching failures, so a highly stable acquisition environment is required. Moreover, this global method does not consider local changes in rotation and translation; therefore, they only show high recognition accuracy on databases captured under partial constraints. Recent methods have developed methods that focus on local texture descriptors, which are more robust to local changes in illumination and thus achieve higher recognition accuracy than encoding-based methods on palm vein databases. However, texture-based methods may not capture high-level semantic information of objects. They usually only focus on pixel-level texture features, while ignoring the overall shape, structure, and contextual information of the palm vein. In deep learning-based methods, both the overall shape and contextual information of the palm vein are more robust than those based on local textures, but deep learning-based methods rely on supervised training processes. Summary of the invention

[0007] In order to solve the problem raised in the background technology, the present invention proposes a palm vein recognition method based on GAttention neural network, which constructs a neural network model based on fixed and adaptive filters and with a self-attention mechanism for palm vein recognition.

[0008] A palm vein recognition method based on GAttention neural network, comprising:

[0009] Step 1: Establish a neural network formed by Gabor filters and attention modules, including a three-layer network with two filter layers L1 and L2 and a binarization layer L3. The configuration of each layer is as follows:

[0010] L0: input layer, dimension is u×v, corresponding to the size of palm vein ROI;

[0011] L1: The first convolutional layer, consisting of k n filters, each filter is used to process the input image, and the output of this layer is composed of k n The image is composed of a dimension of u×v;

[0012] L2: The second convolutional layer, consisting of k m filters, each of which is used to process k outputs of the L1 layer. n The output of this layer consists of k images. n ×k m It is composed of images with dimension u×v;

[0013] L3: Binarization layer, used for k output of L2 layer n ×k m Each pixel of the image is binarized;

[0014] The attention module is used to input k n ×k m The binary image is used for self-attention calculation to update k n ×k m Binarized image;

[0015] Finally, the updated k n ×k m Encode the binary image to obtain the feature vector H of the input image;

[0016] Step 2, using the palm vein image training set, unsupervised training is performed on the neural network to determine the parameters of each Gabor filter, wherein:

[0017] Let the filter of L1 layer be a fixed-size two-dimensional filter with direction θ f Use from the set Θ fThe sampled value is calculated using the following equation:

[0018]

[0019] Wherein, F is a given integer;

[0020] The filter of the L2 layer is set as a two-dimensional filter of adaptive size, which is constructed by adaptive direction calculation and multi-scale Gabor filter calculation based on the palm vein training set with the purpose of selecting the filter response of the maximum magnitude;

[0021] Step 3: extracting a feature vector of the input palm vein image through the trained neural network, and identifying or verifying the input palm vein image based on the feature vector.

[0022] Furthermore, in the binarization layer, the bin function is applied to the k outputs of the L2 layer. n ×k m For each pixel of the image, the bin function is defined by the following formula:

[0023]

[0024] Furthermore, in step 2, let F = k n .

[0025] Furthermore, in step 2, a fixed-scale two-dimensional Gabor filter is defined by the product of a sine wave and a Gaussian function, as follows:

[0026]

[0027] Where σ is the standard deviation of a given Gaussian function, μ is the frequency of a given sine wave, In actual calculations, only G f The real part of (x,y,θ,μ,σ).

[0028] Furthermore, in step 2, the adaptive direction calculation includes: performing a low-pass filter on each ROI in the training set and applying a gradient-based operator to determine an image I describing the local direction of the texture pattern orient ;

[0029] Then calculate I orient The histogram of , takes the average of the histograms of all training ROIs, and extracts the S most frequent directions to form a set Θ s ; Let Θ c =Θ s ∪Θ f When the adaptive direction set Θ is obtained c , including C = |Θ c | directions;

[0030] Furthermore, the calculation of the multi-scale Gabor filter includes: defining two sets of multi-scale two-dimensional Gabor filters G with even symmetry and odd symmetry a,even , G a,odd , together form a multi-scale Gabor filter set G a , specifically using the following equations:

[0031]

[0032] Where real(·) and imag(·) represent the real and imaginary parts of the parameter, respectively, which are used to obtain even-symmetric and odd-symmetric filters; a0 is the scaling factor; r is the aspect ratio of the filter; ψ represents the frequency; and θ c ∈Θ c ,m f The value controls the scale of the filter and is calculated as follows:

[0033]

[0034] Then, we define x using the following formula: rot (θ c ) and y rot (θ c ):

[0035]

[0036] So far, we have obtained a filter set G a,even , including A = M·C even symmetric multiscale filters and another set G a,odd , contains A = M·C odd-symmetric multi-scale filters, the size of each filter is

[0037] Further, the selection process of the filter with the largest magnitude response includes: adapting the multi-scale Gabor filter to the training data by selecting the most relevant filter based on the magnitude response;

[0038] First, the ROI training subset image I is passed through the set G a,even and G a,odd Each filter in is filtered to obtain two sets of filtered images {I even} and {I odd}, for each pair of corresponding filtered images, the combined amplitude response is calculated Get a set of images {I′ a};

[0039] Secondly, for {I′ a}, and take the first A·u·v amplitude responses to form a one-dimensional vector s m, which contains a decreasing amplitude response;

[0040] Then, for G a For each pair of filters in, count the image I′ of each pair of filters a The amplitude response in s m The corresponding number in , and select the first A′ / 2 pairs of filters with the largest number as the required adaptive multi-scale two-dimensional Gabor filters.

[0041] Furthermore, let A′=k n =k m .

[0042] Furthermore, in the attention module, given the input feature map First by reshaping X into It is divided into S×S non-overlapping regions, each of which is flattened to dimension The feature vector of , and the query tensor Q, key tensor K, and value tensor V are obtained as follows:

[0043] Q=X r ,K=X r ,V=X r

[0044] Then, the region-level query and key matrices Q are derived by taking the region average of Q and K respectively. r , Then through Q r and K r The adjacency matrix of the regional affinity graph is obtained by matrix multiplication between the transposes of have:

[0045] A r =Q r (K r ) T .

[0046] By comparing the values ​​in the adjacency matrix, the first k values ​​are selected as the final adjacency matrix A as follows:

[0047] A=topkIndex(A r );

[0048] After obtaining the final adjacency matrix A, multiply A and V to get the final feature map plus the input feature map as the overall output O:

[0049] O=A*V+X.

[0050] Furthermore, the binary image output by the binarization layer has a height of u, a width of v, and a number of channels of k.n ×k m Input feature map Input to the self-attention module and update the original binary image with the final overall output of the module.

[0051] Furthermore, the encoding process of the updated binary image includes: m A binary image, denoted as B i , i = 1, 2, .., k m , each image in the group has the same u×v dimensions, and for each position (x, y), all k images are m The binary values ​​of the image are concatenated to construct a binary vector b:

[0052]

[0053] The binary vector b is then encoded into a decimal number d: Repeat this encoding process for each position (x, y) to obtain a decimal matrix D(x, y);

[0054] Similarly, calculate all k n The decimal matrix D of the group binary image l , where l = 1, 2, .., k n ;

[0055] Next, each image D l Divide into n B non-overlapping blocks of size b1×b2 are formed, and the corresponding histogram is calculated for each block, where each histogram is composed of The value composition;

[0056] Finally, by concatenating all images D l The histogram of all blocks is used to construct the feature vector H, where

[0057] The present invention proposes to train a neural network using a novel unsupervised process based on Gabor response tuning, and does not require class labels. The method can be used for any segmented palm vein image and output a one-dimensional feature vector, which can then be used to calculate the Euclidean distance and then use a classifier to classify the palm vein data. The recognition accuracy of the network can be effectively improved, while simplifying the training process.

[0058] The present invention further utilizes an improved self-attention module to learn long-term dependencies in vein images, thereby obtaining a global receptive field, making up for the shortcoming that convolutional neural networks can only obtain local receptive fields but cannot obtain the overall details of the image, thereby improving the performance and generalization ability of the network in vein recognition tasks.

[0059] In a further embodiment of the present invention, a strategy of not training the self-attention module and not updating its parameters is adopted on the basis that the self-attention module directly performs attention calculation on the feature map, thereby reducing the complexity of the recognition calculation and simplifying the training process.

[0060] In some embodiments of this work, a k-nearest neighbor algorithm classifier based on Euclidean distance is used to classify the generated templates, where k=1, so the classifier does not need to be trained, further simplifying the training process. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 The figure is an overall flow chart of the palm vein recognition method based on GAttention neural network in an embodiment of the present invention.

[0062] Figure 2 Schematic diagram of a GN model formed by a Gabor filter in an embodiment of the present invention. DETAILED DESCRIPTION

[0063] In this example, the designed network model is named GAttention-Net (GANet), which is a Gabor attention aggregation network for palm vein recognition. The overall framework is shown in the figure Figure 1 As shown in the figure, the vein texture features are extracted by forming a GN by combining a fixed-scale Gabor filter and an adaptive-scale Gabor filter, and then a GANet is formed by introducing an attention module after the GN, so that it can learn the long-term dependencies in the vein image and obtain the global receptive field, making up for the shortcoming that the convolutional neural network can only obtain the local receptive field but not the overall details of the image, thereby improving the performance and generalization ability of the network in the vein recognition task.

[0064] The GN model in this example is basically Figure 2 As shown, it is a 3-layer network with two filter layers L1 and L2 and a binarization layer L3. The configuration of each layer is as follows:

[0065] L0: Input layer, dimension is u×v, corresponding to the size of palm vein ROI.

[0066] L1: The first convolutional layer, consisting of k n filters, each of which is used to process the input image; therefore, the output of this layer consists of k n It consists of an image of dimension u×v.

[0067] L2: The second convolutional layer, consisting of k m Each filter is used to process the k output of the L1 layer. n Therefore, the output of this layer consists of kn ×k m It is composed of images with dimension u×v;

[0068] L3: Binarization layer, which applies the bin function to the k outputs of the L2 layer n ×k m For each pixel of the image, the bin function is defined by the following formula:

[0069]

[0070] Get k n ×k m The set of binary images {B i,j},,i=1,2,..,k m ,j=1,2,..,k n .

[0071] In this example, if Figure 2 As shown in Figure 1, the convolution operator in the traditional CNN is replaced by the Gabor filter. The Gabor filter can effectively extract the texture features of veins, thereby improving the accuracy of vein classification. It has been applied to various advanced vein recognition methods.

[0072] In this example, we first use a set of fixed-scale two-dimensional Gabor filters, which can be defined by the product of a sine wave and a Gaussian function as follows:

[0073]

[0074] Among them, θ f is the direction of the filter, σ is the standard deviation of the Gaussian function, μ is the frequency of the sine wave, G f (x, y, θ, μ, σ) contains two parts, the real part and the imaginary part. In the convolution operation in this example, only the real part is used. Here, fixed values ​​are used for μ and σ, and for the direction θ f , using f The sampled value is calculated using the following equation:

[0075]

[0076] In this way, we obtain a set G of fixed-scale two-dimensional Gabor filters in F directions. f .

[0077] On the other hand, a set of adaptive multi-scale Gabor filters is also used. To calculate the set of adaptive multi-scale Gabor filters, this example uses a training subset of the palm vein ROI. First, a set of adaptive directions is calculated from the palm vein ROI. Second, based on the set of adaptive directions, a multi-scale Gabor filter is constructed. Finally, the filter that obtains the maximum magnitude response is selected.

[0078] Computation of adaptive orientation: Low-pass filter each ROI in the training subset and apply a gradient-based operator to determine the image I describing the local orientation of the texture pattern orient Then calculate I orient The histogram of , takes the average of the histograms of all training ROIs, and extracts the S most frequent directions to form a set Θ s Let Θ c =Θ s ∪Θ f When the adaptive direction set Θ is obtained c , including C = |Θ c | directions;

[0079] Calculation of multi-scale Gabor filters: This example defines two sets of multi-scale two-dimensional Gabor filters G with even symmetry and odd symmetry. a,even , G a,odd , together form a multi-scale Gabor filter set G a , specifically using the following equations:

[0080]

[0081] Where real(·) and imag(·) represent the real and imaginary parts of the parameter, respectively, which are used to obtain even-symmetric and odd-symmetric filters; a0 is the scaling factor; r is the aspect ratio of the filter; ψ represents the frequency; and θ c ∈Θ c . m f The value controls the scale of the filter and is calculated as follows:

[0082]

[0083] Where: u is the horizontal size of ROI.

[0084] Then, we define x using the following formula: rot (θ c ) and y rot (θ c ):

[0085]

[0086] In general, this example obtains a filter set Ga,even , including A = M·C even symmetric multiscale filters and another set G a,odd , contains A = M·C odd-symmetric multi-scale filters, the size of each filter is

[0087] Selection of the filter with the largest magnitude response: The multiscale Gabor filter is adapted to the training data by selecting the most relevant filter based on the magnitude response. First, the ROI training subset images I are passed through the set G a,even and G a,odd Each filter in is filtered to obtain two sets of filtered images {I even} and {I odd}, and for each pair of corresponding filtered images the combined amplitude response is calculated Get a set of images {I′ a}, and secondly, we have {I′ a}, and take the first A·u·v amplitude responses to form a one-dimensional vector s m , which contains a decreasing amplitude response. Then, for G a For each pair of filters in, count the image I′ of each pair of filters a The amplitude response in s m The corresponding number in , and select the first A′ / 2 pairs of filters with the largest number as the required adaptive multi-scale two-dimensional Gabor filters.

[0088] The output of the Gabor filter conditioning process consists of F+A′ filters, corresponding to F fixed-scale 2D Gabor filters and A′ adaptive multi-scale 2D Gabor filters. The number of fixed and adaptive Gabor filters can be determined based on the topology of the GN and the number of filters k selected per layer of the network. n and k m To select, in this example, the first layer uses F = k n A fixed-scale two-dimensional Gabor filter; in the second layer, A′=k m adaptive multi-scale two-dimensional Gabor filters; in subsequent experiments, this example selects k n =k m =10.

[0089] The feature extraction step can be divided into two separate tasks as follows:

[0090] i) Apply the trained GN to obtain a binary image;

[0091] First, the trained GN is applied to the palm vein ROI to obtain k n ×k mA binary image, forming a binary image set {B i,j},

[0092] In order to learn the long-term dependencies in the vein images and obtain the global receptive field, in this example, these binary images are also sent to the self-attention module.

[0093] The general self-attention module will query key Sum As input, the attention function converts each query into a weighted sum of values, where the weight is calculated as the normalized dot product between the query and the corresponding key. It can be formally defined in a compact matrix form, namely:

[0094]

[0095] Here we introduce the scalar factor To avoid weight concentration and gradient vanishing.

[0096] The multi-sensor head self-attention mechanism MHSA uses "self-attention", that is, query Q, key value K and value V as the same input The linear projection of X is derived, X is a spatially flat feature map, that is, N = H × W, where H and W are the height and width of the feature map, respectively, and C represents the number of channels. As for "multi-head", it means that the output is split into h blocks (i.e., heads) along the channel dimension, and each block uses a set of independent projection weights. Formally,

[0097]

[0098] in is the i th Output of an attention head. The corresponding input projection weights are used. Using an additional linear transformation, the weight matrix Blend all the heads.

[0099] The complexity of MHSA is O(N 2 ), because there are N queries, each query involves N key-value pairs. Such high complexity brings serious input space resolution scalability issues. Therefore, a new self-attention calculation method is introduced in this example to reduce the complexity.

[0100] In this example, in the attention module, given the input feature map First, it is divided into S×S non-overlapping regions, each of which is flattened into dimensions This step is done by reshaping X into Then use linear projection to derive the query, key, and value tensors And because the self-attention calculation is performed directly on the feature map after the feature output is extracted by GN, there is no need to update the Q, K, V projection weights through training, so the values ​​of Q, K, and V are equal, and we have:

[0101] Q=X r ,K=X r ,V=X r (7)

[0102] Then, by constructing a directed graph, we can find the adjacency relationship, such as the region that each given region should focus on. Specifically, we first derive the region-level query and key matrix Q by taking the region average of Q and K respectively. r , Then through Q r and K r The adjacency matrix of the regional affinity graph is obtained by matrix multiplication between the transposes of

[0103] A r =Q r (K r ) T . (8)

[0104] Adjacency matrix A of the region affinity graph r The entries in measure the semantic relevance of two regions. The next core step is to prune the affinity graph by retaining only k value connections for each region. Specifically, by comparing the values ​​in the adjacency matrix, the first k values ​​are selected as the final adjacency matrix A as follows

[0105] A=topkIndex(A r ). (9)

[0106] After obtaining the final adjacency matrix A, multiply A and V to get the final feature map plus the input feature map as the overall output O:

[0107] O=A*V+X. (10)

[0108] Specifically, in this example, the binary image output by GN has a height of u, a width of v, and a number of channels k. n ×k m Input feature map The input is sent to the self-attention module mentioned above, and the original binary image is updated with the final overall output of the module.

[0109] ii) Encode the binary image;

[0110] For the updated binary graph, consider a set of k mA binary image, denoted as B i , i = 1, 2, .., k m , each image in the group has the same u×v dimensions, and for each position (x, y), all k images are m The binary values ​​of the image are concatenated to construct a binary vector b:

[0111]

[0112] The binary vector b is then encoded into a decimal number d: Repeating this encoding process for each position (x, y) results in a decimal matrix D(x, y) that describes all k m binary output images. Similarly, calculate all k n The decimal matrix D of the group binary image l , where l = 1, 2, .., k n .

[0113] Next, each image D l Divide into n B non-overlapping blocks of size b1×b2 are formed, and the corresponding histogram is calculated for each block, where each histogram is composed of values ​​(corresponding to the possible value of d).

[0114] Finally, by concatenating all images D l The histogram of all blocks is used to construct the feature vector H, where

[0115] The feature vector H obtained above can be further applied to palm vein image recognition and palm vein image verification.

[0116] In the image recognition stage, the feature vector H extracted from the palm vein image is used for image classification; the Euclidean distance between each group of feature vectors is calculated to form a Euclidean distance matrix; and then the kNN classifier is used for classification. In this example, the hyperparameter k of the kNN classifier is given as 1.

[0117] In the image verification stage, vein feature matching is performed by comparing the extracted vein feature vectors H1 and H2 to achieve feature matching verification.

[0118] The following example illustrates the practical effect of the method in this example through comparative experiments.

[0119] To verify the performance of the proposed method, a large number of experiments were conducted on three palm vein datasets. To evaluate the effectiveness of GANet in improving accuracy, the proposed method was compared with mainstream machine learning models and deep learning models.

[0120] Dataset A: CASIA palm vein dataset, containing 1200 palm vein images. These images come from 100 volunteers, each of whom provided images of their left and right hands. Specifically, each hand of each volunteer was photographed 3 times in two different sessions. Therefore, each volunteer provided a total of 6 images in each session (3 left hands + 3 right hands), making a total of 12 images in two sessions. The total number of images in the dataset is calculated as follows: 100 volunteers × 2 hands × 3 images per person per session × 2 sessions = 1200 images.

[0121] Dataset B: VERA palm vein dataset, including 2200 palm vein images. These images come from 110 volunteers, each of whom provided palm vein images of the left and right hands. Specifically, each volunteer's hand was captured in two independent acquisition sessions, resulting in a total of 20 images per volunteer (5 left hands + 5 right hands, acquired 2 times). Therefore, the total number of images in the dataset is calculated as 110 volunteers × 2 hands × 5 images / hand × 2 acquisitions = 2200 images. The resolution of each image is 200 × 200 pixels.

[0122] Dataset C: The Tongji University palm vein dataset contains 12,000 palm vein images of 300 volunteers, with an acquisition interval of about 60 days and two acquisitions. Ten images of each volunteer's hand were taken during each acquisition, totaling 12,000 images (300 subjects × 2 hands × 10 images × 2 sessions). The resolution of each image is 128 × 128 pixels.

[0123] In order to evaluate the effectiveness of the proposed method, each dataset was divided into two independent sets for two stages. For dataset A, each palm was considered as a separate category, resulting in 200 categories (the left and right palms of each volunteer were considered as a category, with a total of 100 volunteers). Two different training and testing methods were used: one method was to randomly select two images for model training in the first stage and use the remaining images for testing in the second stage; the other method was to use three images for training in the first stage and the remaining three images for testing in the second stage.

[0124] For dataset B, a similar approach was adopted: two and four images were randomly selected for training in the first stage, and the remaining five images were used for testing in the second stage.

[0125] Dataset C adopts a different grouping strategy: two, four, six, eight, and ten images are randomly selected for the first stage of training, and ten images from the second stage are used for testing.

[0126] Considering that the original image may contain irrelevant background areas, for classification, this example extracts the region of interest (ROI) from the original image, and the size of all ROI images is standardized to 32×32.

[0127] Extensive experiments were conducted on three public datasets to evaluate the performance of the proposed palm vein dataset method. Specifically, nine network models in the prior art were implemented in the experiments, namely ResNet, FVCNN, FVRASNet, LightWeightCNN, PCANet, HOG, LDP, LLDP and PalmNe; the recognition results and equivalent error rates of the models on the three datasets were obtained through a unified method.

[0128] Table 1 shows the comparison of ablation experiment results of different models on dataset A, where 2 and 3 represent the number of training samples in each category, respectively.

[0129] Table 1 Comparison of ablation experiment results on dataset A

[0130]

[0131]

[0132] Table 2 shows the ablation experiment comparison results of different models on dataset B, where 2 and 4 represent the number of training samples in each category.

[0133] Table 2. Comparison of ablation experiments on dataset B

[0134]

[0135] Table 3 shows the comparison results of ablation experiments of different models on dataset C, where 2, 4, 6, 8, and 10 represent the number of training samples in each category.

[0136] Table 3 Comparison of ablation experiment results on dataset C

[0137]

[0138] The experimental results in Table 1, Table 2, and Table 3 show that the proposed method improves the recognition rate and obtains the lowest equivalent error rate (EER) compared with other methods. For example, from Table 1, it can be seen that the accuracy (Acc) obtained on dataset A using the proposed method is 99.17%, and the equivalent error rate (EER) obtained is 2.47%.

Claims

1. A palm vein recognition method based on GAttention neural network, characterized in that: include, Step 1: Establish a neural network formed by Gabor filters and attention modules, including a three-layer network with two filter layers L1 and L2 and a binarization layer L3. The configuration of each layer is as follows: L0: input layer, dimension is u×v, corresponding to the size of palm vein ROI; L1: The first convolutional layer, consisting of k n filters, each filter is used to process the input image, and the output of this layer is composed of k n The image is composed of a dimension of u×v; L2: The second convolutional layer, consisting of k m filters, each of which is used to process k outputs of the L1 layer. n The output of this layer consists of k images. n ×k m It is composed of images with dimension u×v; L3: Binarization layer, used for k output of L2 layer n ×k m Each pixel of the image is binarized; The attention module is used to input k n ×k m The binary image is used for self-attention calculation to update k n ×k m Binarized image; Finally, the updated k n ×k m Encode the binary image to obtain the feature vector H of the input image; Step 2, using the palm vein image training set, unsupervised training is performed on the neural network to determine the parameters of each Gabor filter, wherein: Let the filter of L1 layer be a fixed-size two-dimensional filter with direction θ f Use from the set Θ f The sampled value is calculated using the following equation: Wherein, F is a given integer; The filter of the L2 layer is set as a two-dimensional filter of adaptive size, which is constructed by adaptive direction calculation and multi-scale Gabor filter calculation based on the palm vein training set with the purpose of selecting the filter response of the maximum magnitude; Step 3: extracting a feature vector of the input palm vein image through the trained neural network, and identifying or verifying the input palm vein image based on the feature vector.

2. The method according to claim 1, characterized in that In the binarization layer, the bin function is applied to the k outputs of the L2 layer. n ×k m For each pixel of the image, the bin function is defined by the following formula:

3. The method according to claim 1, characterized in that In step 2, let F = k n .

4. The method according to claim 1, characterized in that: In step 2, a fixed-scale two-dimensional Gabor filter is defined by the product of a sine wave and a Gaussian function as follows: Where σ is the standard deviation of a given Gaussian function, μ is the frequency of a given sine wave, In actual calculations, only G f The real part of (x,y,θ,μ,σ).

5. The method according to claim 1, characterized in that In step 2, the adaptive direction calculation includes: low-pass filtering each ROI in the training set and applying a gradient-based operator to determine the image I describing the local direction of the texture pattern orient ; Then calculate I orient The histogram of , takes the average of the histograms of all training ROIs, and extracts the S most frequent directions to form a set Θ s ; Let Θ c =Θ s ∪Θ f When the adaptive direction set Θ is obtained c , including C = |Θ c | direction.

6. The method according to claim 5, characterized in that The calculation of multi-scale Gabor filter includes: defining two sets of multi-scale two-dimensional Gabor filters G with even symmetry and odd symmetry a,even , G a,odd , together form a multi-scale Gabor filter set G a , specifically using the following equations: Where real(·) and imag(·) represent the real and imaginary parts of the parameter, respectively, which are used to obtain even-symmetric and odd-symmetric filters; a0 is the scaling factor; r is the aspect ratio of the filter; ψ represents the frequency; and θ c ∈Θ c ,m f The value controls the scale of the filter and is calculated as follows: Then, we define x using the following formula: rot (θ c ) and y rot (θ c ): So far, we have obtained a filter set G a,even , including A = M·C even symmetric multiscale filters and another set G a,odd , contains A = M·C odd-symmetric multi-scale filters, the size of each filter is 7. The method according to claim 6, characterized in that The selection process of the filter with the largest magnitude response includes: adapting the multi-scale Gabor filter to the training data by selecting the most relevant filter based on the magnitude response; First, the ROI training subset image I is passed through the set G a,even and G a,odd Each filter in is filtered to obtain two sets of filtered images {I even } and {I odd }, and for each pair of corresponding filtered images the combined amplitude response is calculated Get a set of images {I′ a }; Secondly, for {I′ a }, and take the first A·u·v amplitude responses to form a one-dimensional vector s m , which contains a decreasing amplitude response; Then, for G a For each pair of filters in, count the image I′ of each pair of filters a The amplitude response in s m The corresponding number in , and select the first A′ / 2 pairs of filters with the largest number as the required adaptive multi-scale two-dimensional Gabor filters.

8. The method according to claim 7, characterized in that Let A′=k n =k m .

9. The method according to claim 8, characterized in that In the attention module, given the input feature map First by reshaping X into It is divided into S×S non-overlapping regions, each of which is flattened to dimension The feature vector of , and the query tensor Q, key tensor K, and value tensor V are obtained as follows: Q=X r ,K=X r ,V=X r Then, the query and key matrices at the region level are derived by taking the region average of Q and K respectively. Then through Q r and K r The adjacency matrix of the regional affinity graph is obtained by matrix multiplication between the transposes of have: A r =Q r (K r ) T . By comparing the values ​​in the adjacency matrix, the first k values ​​are selected as the final adjacency matrix A as follows: A=topkIndex(A r ); After obtaining the final adjacency matrix A, multiply A and V to get the final feature map plus the input feature map as the overall output O: O = A*V+X; The binary image output by the binarization layer has a height of u, a width of v, and a number of channels of k. n ×k m Input feature map Input to the self-attention module and update the original binary image with the final overall output of the module.

10. The method according to claim 1, characterized in that The encoding process of the updated binary image includes: m A binary image, denoted as B i , i = 1, 2, .., k m , each image in the group has the same u×v dimensions, and for each position (x, y), all k images are m The binary values ​​of the image are concatenated to construct a binary vector b: The binary vector b is then encoded into a decimal number d: Repeat this encoding process for each position (x, y) to obtain a decimal matrix D(x, y); Similarly, calculate all k n The decimal matrix D of the binary image l , where l = 1, 2, .., k n ; Next, each image D l Divide into n B non-overlapping blocks of size b1×b2 are formed, and the corresponding histogram is calculated for each block, where each histogram is composed of The value composition; Finally, by concatenating all images D l The histogram of all blocks is used to construct the feature vector H, where