Bi-GRU vein recognition method based on attention mechanism

Abstract

The invention relates to the technical field of vein recognition, in particular to a Bi-GRU vein recognition method based on an attention mechanism. Comprising the steps that a Bi-GRU model based on an attention mechanism is established, and the Bi-GRU model is mainly composed of a down-sampling layer, a feature coding module, a time attention module, a scale attention module and a fusion module; the down-sampling layer maps images in the multi-view finger vein image sequence into characteristic spectrums of multiple scales, and the characteristic coding module divides each characteristic spectrum into N characteristic blocks and converts the N characteristic blocks into characteristic vectors; the time attention module learns a time dependency relationship between the characteristic spectrums under the same scale; scale relation dependence is learned in the scale attention module, and the fusion module fuses the characteristic spectrum from the two dimensions of scale and time. And step 2, performing classification and identification of vein images by using the trained Bi-GRU model based on the attention mechanism. The problem that in the prior art, the accuracy of an identification method based on a multi-view finger vein image sequence needs to be improved is solved.

Description

Technical Field

[0001] The present invention relates to the technical field of vein recognition, and in particular to a Bi-GRU vein recognition method based on an attention mechanism. Background Art

[0002] With the rapid development of the information industry and the widespread application of the Internet, information security has become increasingly important. As a new identity recognition technology, vein recognition is not easy to copy, forge, or destroy compared to fingerprints and faces. It is irreplaceable in high-security application fields and has broad application prospects. However, the imaging quality of vein features is affected by various factors, such as ambient lighting, ambient temperature, user behavior, etc., and finger vein identity recognition technology faces severe challenges. For example, during the recognition process, the flipping of the user's finger will change the projection angle of the three-dimensional blood vessels on the two-dimensional plane, resulting in the ambiguity of the vein features, that is, the same finger vein may have different two-dimensional plane projections, which reduces the recognition accuracy of the system.

[0003] To solve this problem, the prior art proposes a recognition method based on a multi-view finger vein image sequence, which processes the input finger vein images of different viewpoints as an image sequence in a certain order; however, the accuracy and stability of various recognition methods given in the prior art still need to be improved. Summary of the invention

[0004] In order to solve the problem that the accuracy of the recognition method based on multi-view finger vein image sequence in the prior art needs to be improved, the present invention provides a Bi-GRU vein recognition method based on attention mechanism, including:

[0005] Step 1: Use the following strategy to build a Bi-GRU model based on the attention mechanism:

[0006] The model mainly consists of a downsampling layer, a feature encoding module, a temporal attention module, a scale attention module, and a fusion module;

[0007] The downsampling layer maps the T images in the multi-view finger vein image sequence into feature spectra of multiple scales, where the same image is downsampled at different times to obtain feature spectra of M scales. t=1,2…T, m=1,2,…,M;

[0008] The feature encoding module divides each feature spectrum into N feature blocks, and then converts each feature block into a feature vector to form a feature matrix of a single image at a single size;

[0009] In the temporal attention module, the feature matrix of each image at a single scale is taken to form a sequence, and then the feature sequence at a single scale is obtained. The feature sequences at each scale are respectively input into the attention bidirectional GRU module, in which the temporal dependency between the feature spectra at the same scale is learned;

[0010] The attention bidirectional GRU module includes two rows of attention GRU modules, the number of attention GRU modules in each row is equal to the number of feature spectra in the feature spectrum sequence at a single scale, that is, the number of images in the multi-view finger vein image sequence, and the attention GRU modules in each row are aligned with the feature spectra in the feature spectrum sequence as input in sequence, but the attention GRU modules in each row are cascaded in sequence but in opposite directions, which are respectively called the forward process and the backward process;

[0011] The final output of the forward process and the backward process is concatenated and linearly mapped to obtain the feature matrix

[0012] In the scale attention module, we set is the nth eigenvector of the mth scale at the tth moment. There are M nth eigenvectors at the tth moment, n=1,2…,N, recorded as The vectors of different scales at the same time are input into the attention GRU module to learn the scale relationship dependency, and all the outputs are recorded as Finally, the output of each scale is fused into Calculate the N eigenvector sequence at the tth moment Take the average to get Z t , and finally for Z at T moments t Take the average and get

[0013] In the fusion module, all feature spectrum sequences with different scales are taken as input and firstly transformed by the convolution operator. The number of channels is doubled, and then it is divided into four feature sub-graphs by sampling with a step size of 2.

[0014] Then, the scores of each point on the four feature matrices can be calculated by the following formula to form a score matrix:

[0015]

[0016] Among them, log is the logarithmic function, abs is the absolute value operation;

[0017] Based on the score matrix calculated above Extract the eigenvalue points with the highest scores at the same position between feature subgraphs to form a new feature matrix Then it is fused with the feature spectrum of the next scale and updated as follows:

[0018]

[0019] Update and recurse step by step to obtain the feature matrix fused along the scale direction Calculate X at each moment t The score of each point in ′, and then the score matrix S is obtained t as follows:

[0020] S t =Sigmoid(log(abs(X t ′))),t=1,2,…,T

[0021] Based on the score matrix S calculated above t , extract the eigenvalue point with the highest score at the same position between the feature matrices at each moment to form the feature matrix Y′ that is fused along the time direction:

[0022] Then, the feature matrix Y′ is divided into N feature vectors, denoted as Y″, using the same method as in the feature encoding module. n ;

[0023] Calculate the average value of N eigenvectors to get Y fusion , and then with Y attention The added output O is used for subsequent classification tasks;

[0024] Step 2: Use the trained attention-based Bi-GRU model to classify and recognize vein images.

[0025] Furthermore, in the feature encoding module, let X 1:T is a multi-view finger vein image sequence X1,X2,…,X T , for the t-th image X t ∈R w×H , where W represents width and H represents height;

[0026] In the feature encoding module, a convolution operator with a convolution kernel size of 3×3 and 64 channels is used for X t After downsampling, the output characteristic spectrum is expressed as

[0027] Similarly, the next convolution and downsampling operation also uses a 3×3 convolution kernel, but the number of channels will be doubled, and the output feature spectrum can be expressed as After M convolution and downsampling operations, M feature spectra of different scales are obtained.

[0028] Furthermore, in the feature encoding module, a convolution pair with an output channel of d, a convolution kernel size of p×p, and a step size of p is used. Perform convolution operation to obtain N feature vectors, expressed as n=1,2,…,N, where N=((W / 2)×(H / 2)) / (p×p); N eigenvectors are combined into a matrix

[0029] For the subsequent m=2,3,…M, using output channel d and convolution kernel size (p / 2 m-1 )×(p / 2 m -1 ), the step size is (p / 2 m-1 ) is divided into blocks and stretched to obtain

[0030] Furthermore, the attention GRU module includes three gates: a reset gate, a selection gate, and an update gate;

[0031] The reset gate is responsible for determining which timing information of the previous moment needs to be retained, the selection gate is responsible for deciding which feature matrices of the current feature matrix interact with which feature matrices of the previous moment for timing information, and the update gate is used to determine the impact of the previous moment's timing information on the current timing information.

[0032] Furthermore, in the attention GRU module, As the input at the current moment, let is the output of the previous hidden state at the mth scale, Mapped into three matrices through a multi-layer perceptron and After the multi-layer perceptron is converted into three matrices K m , K′ m and V m ;

[0033] The selection of each gate is made by the sigmoid activation function, and the calculation process of each gate is as follows:

[0034]

[0035]

[0036]

[0037]

[0038]

[0039]

[0040] Where thres is the threshold value, the value range is 0≤thres≤1, σ is the sigmoid activation function, 0≤σ(·)≤1, and for and The relationship matrix, To reset the gate, and is the update gate, the output at the current moment Obtained by:

[0041]

[0042] Among them, Tanh is the hyperbolic tangent function, Concat is the concatenation function, is the hidden state at the current moment. Finally, the output at the current moment can be obtained through nonlinear mapping.

[0043]

[0044] Where: W′ is the coefficient matrix in the nonlinear mapping network here, and b′ is the bias in the nonlinear mapping here.

[0045] Furthermore, let the outputs of the forward process and the backward process at time t be and The outputs of the forward process and the backward process are linearly mapped to as follows:

[0046]

[0047] Where: W1, W2 are coefficient matrices in the corresponding linear mapping network, and g is the linear mapping function.

[0048] Furthermore, in the scale attention module, the output of each scale is fused into as follows:

[0049]

[0050] Where: W″ is the coefficient matrix in the nonlinear mapping network here, and b″ is the bias in the nonlinear mapping here.

[0051] Furthermore, in the fusion module, the calculation process of the feature subgraph is as follows:

[0052]

[0053] Among them, Conv is a convolution operator with a convolution kernel size of 1×1, and Split represents an interval sampling method with a step size of 2.

[0054] Furthermore, the eigenvalue points with the highest scores at the same position between the feature subgraphs are extracted to form a new feature matrix The process is as follows:

[0055]

[0056]

[0057] Furthermore, the eigenvalue points with the highest scores at the same position in the feature matrix at each moment are extracted to form the feature matrix Y that is fused along the time direction ′ The process is as follows:

[0058]

[0059]

[0060] Compared with the attention LSTM, the attention GRU introduced in the present invention can better extract the feature information between multi-view finger vein images, and is more suitable for multi-view finger vein small data sets. The scale attention module can effectively extract the scale feature information of the vein image sequence. The compact feature information (scale, time and space feature information) proposed by the fusion module can represent the vein image sequence. After the feature information extracted by the time attention module, scale attention module and fusion model in the model are fused, the multi-view finger vein image sequence can be effectively modeled. Experiments have proved that the model has higher recognition accuracy than the prior art. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] Figure 1 Schematic diagram of the structure of the Bi-GRU model based on the attention mechanism in an embodiment of the present invention.

[0062] Figure 2 Schematic diagram of the structure of the attention bidirectional GRU module in an embodiment of the present invention.

[0063] Figure 3 Schematic diagram of the structure of the attention GRU module in an embodiment of the present invention.

[0064] Figure 4 Schematic diagram of the scale fusion process in the scale fusion module in the embodiment of the present invention. DETAILED DESCRIPTION

[0065] The following is an introduction to the modeling method of the Bi-GRU model based on the attention mechanism in this example. The basic framework of the model is as follows Figure 1As shown in the figure, it is divided into a downsampling layer, a feature encoding module, a temporal attention module, a scale attention module and a fusion module.

[0066] Downsampling layer: Assume X 1:T is a multi-view finger vein image sequence X1,X2,…,X T For the t-th image X t ∈R W×H , where W represents width and H represents height. Then, the feature encoding module performs M convolution and downsampling operations on the image sequence to obtain M feature spectrum sequences of different scales. Specifically, the module uses a convolution operator with a convolution kernel size of 3×3 to convolve X t After downsampling, the output characteristic spectrum can be expressed as Similarly, the next convolution and downsampling operation also uses a 3×3 convolution kernel, but the number of channels will be doubled, and the output feature spectrum can be expressed as After M times of convolution and downsampling operations, M feature spectra of different scales can be obtained. m=1,2…M,where Represents the characteristic spectrum of the mth scale of the tth sequence.

[0067] Feature encoding module: includes a block module and an encoding layer; for the M-scale feature spectra of the t-th sequence, the block module splits each feature spectrum into N feature blocks, and the encoding layer stretches the feature blocks into feature vectors. For example, this module uses a convolution pair with an output channel of 384 (d=384), a convolution kernel size of 8×8 (p=8), and a step size of 8 (p=8). The result of the convolution operation is expressed as (n=1,2,…,N), where N=((W / 2)×(H / 2)) / (p×p) is the number of feature blocks. This module combines the output N feature vectors into a matrix For subsequent scales (m=2,3,4), this module uses p=4,2,1 to divide it into blocks (m=2,3,4). This module also combines feature matrices of different scales into one matrix Then combine the feature matrices of different time series into one matrix

[0068] Temporal attention module: The recurrent neural network GRU can be used to learn short-range feature information, and the GRU model has fewer parameters than the long short-term memory network LSTM, so the computational cost is lower. Neural networks based on the attention mechanism are used to learn long-range feature information, and this type of model has a strong representation ability and has been successfully applied in the fields of computer vision and natural language processing. For multi-view finger vein recognition, it is necessary to extract the temporal dependency between each view, and this temporal dependency must have a certain degree of universality and high semantics. Therefore, the temporal attention module in this example introduces the attention mechanism into the GRU and further enriches the gating information of the GRU so that it can learn a more robust temporal dependency, thereby obtaining the attention GRU (Attention Bi-GRU) module, and then expanding the attention GRU to a bidirectional GRU (Bi-GRU) network to obtain temporal dependencies in both forward and backward ways. The attention GRU module contains three gates: reset gate, selection gate, and update gate, which play a vital role in determining the information flow within the module. The reset gate is responsible for determining which timing information of the previous moment needs to be retained, the selection gate is responsible for deciding which feature matrices of the current feature matrix interact with which feature matrices of the previous moment for timing information, and the update gate is used to determine the impact of the previous moment's timing information on the current timing information.

[0069] The temporal attention module proposed in this example is used to extract the temporal dependencies between multi-view feature spectra and the temporal dependencies between multi-scale feature spectra. Figure 2 As shown in the figure, the temporal attention module contains two rows of attention GRU modules. The number of attention GRU modules in each row is equal to the number of feature matrices in the feature matrix sequence at a single scale, that is, the number of images in the multi-view finger vein image sequence. The attention GRU modules in each row are aligned in sequence with the feature matrices in the feature matrix sequence as input, but the attention GRU modules in each row are cascaded in sequence but in opposite directions, which are respectively called the forward process and the backward process.

[0070] In order to learn the temporal dependencies at a single scale, As the input of the current moment (the moment can be understood as the order of a single image in the image sequence), assume is the output of the previous hidden state at the mth scale, such as Figure 3 As shown, in the attention GRU module, Mapped into three matrices through a multi-layer perceptron (MLP) and After MLP, it is converted into three matrices K m , K′ m and V mThe selection of each gate is made by the Sigmoid activation function, and the calculation process is shown in formulas (1)-(6):

[0071]

[0072]

[0073]

[0074]

[0075]

[0076]

[0077] Where thres is the threshold value, and its value range is 0≤thres≤1. The threshold value needs to be manually tuned. The threshold value used in this example is 0.9. σ is the sigmoid activation function, 0≤σ(·)≤1. and for and The relationship matrix, To reset the gate, and is the update gate, the output at the current moment It can be obtained from formula (7) and formula (8):

[0078]

[0079]

[0080] Among them, Tanh is the hyperbolic tangent function, Concat is the concatenation function, is the hidden state at the current moment. Finally, the output at the current moment can be obtained through nonlinear mapping. According to formula (9),

[0081]

[0082] Where: W′ is the coefficient matrix in the nonlinear mapping network here, and b′ is the bias in the nonlinear mapping here.

[0083] The outputs of the forward process and the backward process at time t are and The outputs of the forward process and the backward process are concatenated and linearly mapped. According to formula (10), we can get

[0084]

[0085] Where: W1, W2 are coefficient matrices in the corresponding linear mapping network, g is the linear mapping function;

[0086] Then, by aggregating the outputs at T times, we can get Y according to formula (11): m :

[0087]

[0088] Finally, the feature matrices of M scales are aggregated and Y is obtained according to formula (11):

[0089] Y = Aggregate(Y 1 ,Y 2 ,…,Y M ), (12)

[0090] Aggregate represents the aggregation operation.

[0091] Scale attention module: In practical application scenarios, scale-invariant features are very important for vein recognition tasks, especially for non-contact vein recognition tasks, because there are many scale changes between vein images collected using non-contact methods. Therefore, after obtaining the temporal dependency between multi-view images, this example also calculates the dependency between images of different scales. Assume is the nth, n=1, 2, …, Nth feature vector of the mth, m=1, 2, …, Mth scale at the tth, t=1, 2, …, Tth time. There are a total of M nth feature vector sequences at the tth time, recorded as like Figure 1 As shown in the scale attention module, these vectors of different scales at the same time are input into the attention GRU module to learn the scale relationship dependency, and all the outputs are recorded as Finally, using nonlinear mapping, the output of each scale is fused according to formula (13)

[0092]

[0093] Where: W″ is the coefficient matrix in the nonlinear mapping network here, and b″ is the bias in the nonlinear mapping here.

[0094] In this way, the scale attention module can calculate the scale dependency between N feature vectors. The fusion operation module takes the average of these N feature vectors and obtains Z according to formula (14): t :

[0095]

[0096] Among them, Mean is the averaging method. Finally, the fusion operation module performs an average operation on the T feature vectors and obtains Y according to formula (15): attention :

[0097] Y attention =Mean(Z1,Z2,…,Z T ) (15)

[0098] Among them, Y attention ∈R 1×d is a one-dimensional feature vector with dimension d=384.

[0099] Fusion module: In order to obtain the time, scale and space feature information, the feature spectrum sequence obtained from the downsampling layer is input into the encoding layer, and the feature spectra of different scales at the same time are used to form a new feature spectrum sequence As the input of the scale fusion module; Figure 4 As shown, in the scale fusion module, is the feature spectrum of the mth scale at the tth moment. First, the number of its channels is doubled by the convolution operator, and then it is divided into four feature sub-graphs by interval sampling with a step size of 2. The calculation process is shown in formula (16):

[0100]

[0101] Among them, Conv is a convolution operator with a convolution kernel size of 1×1, and Split represents an interval sampling method with a step size of 2.

[0102] Then, the score of each point on the four feature subgraphs can be calculated by formula (17) to form a score matrix :

[0103]

[0104] Among them, log is the logarithmic function, and abs is the absolute value operation.

[0105] Based on the score matrix calculated above, the eigenvalue points with the highest scores at the same position between the feature subgraphs are extracted to form a new feature matrix The process can be expressed by formulas (18) and (19):

[0106]

[0107]

[0108] After calculation according to the above formula, It will include the eigenvalue points with the highest scores, and then merge them with the feature spectrum of the next scale according to formula (20):

[0109]

[0110] By updating and recursively step by step, the characteristic spectrum fused along the scale direction can be obtained.

[0111] In the time fusion module, X′ at each moment is calculated according to formula (21): t The score of each eigenvalue in the , and then get the score matrix S t :

[0112] S t =Sigmoid(log(abs(X t ′))),t=1,2,…,T (21)

[0113] Similar to the above process, the feature spectrum Y′ consisting of the eigenvalues ​​with the highest scores is obtained through formulas (22) and (23):

[0114]

[0115]

[0116] In this way, the feature spectrum Y′ is fused along the time direction, and then the number of channels is reduced to 384 through the convolution operation as in the feature encoding module and divided into N feature vectors, recorded as Y″ n ∈R 1×d , n=1,2…,N;d=384.

[0117] Next, we calculate the average value of N eigenvectors according to formula (24) to obtain Y fusion :

[0118]

[0119] Finally, the time and scale dependency and the fused feature information are fused through formula (25) to obtain the output O, which will be used for subsequent classification tasks:

[0120] O=Y attention +Y fusion (25)

[0121] The following multi-view finger vein dataset is used to evaluate the performance of the proposed method in this example. The data comes from the CTBUFull View Vein database (denoted as database A) and the LFMB-3DFB database (denoted as database B). Each database is divided into a training set and a test set. The training set is used to train various classifiers, and the test set is used to evaluate the recognition performance of the classifier. Database A has 235 fingers, and each finger is sampled 10 times. If each finger is regarded as a category, there are 235 categories in total. Six image sequences of a finger are used for training, and the remaining four sequences are used for testing. Therefore, the training set and test set include 1410 (235 fingers × 6 sequences) sequences and 940 (235 fingers × 4 sequences) sequences, respectively. Database B has 695 fingers, and each finger is sampled 10 times. Similarly, the training set and test set of database B have 3954 (695 fingers × 6 sequences) sequences and 2636 (695 fingers × 4 sequences) sequences, respectively.

[0122] In the experiment, the stochastic gradient descent method is used to optimize the model, the cross entropy function is used as the loss function of the model, the momentum factor is 0.9, the weight decay factor is 0.0001, and the learning rate is 0.01. In the training phase, the OneCycleLR learning rate decay method is used for training, and the batch size is 32. After 120 rounds of training, the obtained model will be used to predict the probability of each test sequence belonging to any category, and the category with the maximum probability is used as the predicted category.

[0123] The following compares some excellent recognition methods in the field of vein research, which can be divided into two-dimensional vein recognition algorithms and three-dimensional vein recognition algorithms. The two-dimensional vein recognition algorithms are: ArcVein, FV-CNN, FVRAS-NET and Lightweight-CNN; the three-dimensional vein recognition algorithms are: CNN+LSTM, MVCNN, Local Transformer, HCAN and MFV-FESNet. There are 1410 image sequences in the training set of database A, each of which consists of 12 images from 12 perspectives. Three and six images are extracted from the sequence of equally spaced views, respectively, to obtain two training subsets A1 and A2. Therefore, the sequences in training subsets A1 and A2 include 3 and 6 images, respectively. For database B, the training set includes 3954 image sequences, each of which contains 6 images from 6 perspectives. Similarly, 3 and 6 images are equally spaced from a sequence, and two training subsets B1 and B2 are obtained accordingly. All classifiers are trained on different sub-datasets and validated on the corresponding test sets of datasets A and B (the input test set image sequence length is equal to the sequence length of the training subset). Compared with all the methods considered in the experiment, the proposed method has the highest recognition accuracy and the lowest equal error rate (EER). Specifically, for database A, the proposed method achieved a recognition accuracy of 99.47% and an EER of 0.43%; for database B, the proposed method achieved a recognition accuracy of 99.86% and an EER of 0.12%. Through analysis, it can be concluded that: first, the attention GRU in the model can better extract the feature information between multi-view finger vein images than the attention LSTM, and is more suitable for multi-view finger vein small datasets. Second, the scale attention module can effectively extract the scale feature information of the vein image sequence. Third, the compact feature information (scale, temporal and spatial feature information) proposed by the fusion module can represent the vein image sequence. Fourth, after the feature information extracted by the temporal attention module, the scale attention module and the fusion model in the model are fused, the multi-view finger vein image sequence can be effectively modeled.

Claims

1. The Bi-GRU vein recognition method based on the attention mechanism is characterized by: include: Step 1: Use the following strategy to build a Bi-GRU model based on the attention mechanism: The model mainly consists of a downsampling layer, a feature encoding module, a temporal attention module, a scale attention module, and a fusion module; The downsampling layer maps the T images in the multi-view finger vein image sequence into feature spectra of multiple scales, where the same image is downsampled at different times to obtain feature spectra of M scales. The feature encoding module divides each feature spectrum into N feature blocks, and then converts each feature block into a feature vector to form a feature matrix of a single image at a single size; In the temporal attention module, the feature matrix of each image at a single scale is taken to form a sequence, and then the feature sequence at a single scale is obtained. The feature sequences at each scale are respectively input into the attention bidirectional GRU module, in which the temporal dependency between the feature spectra at the same scale is learned; The attention bidirectional GRU module includes two rows of attention GRU modules, the number of attention GRU modules in each row is equal to the number of feature spectra in the feature spectrum sequence at a single scale, that is, the number of images in the multi-view finger vein image sequence, and the attention GRU modules in each row are aligned with the feature spectra in the feature spectrum sequence as input in sequence, but the attention GRU modules in each row are cascaded in sequence but in opposite directions, which are respectively called the forward process and the backward process; The final output of the forward process and the backward process is concatenated and linearly mapped to obtain the feature matrix In the scale attention module, we set is the nth eigenvector of the mth scale at the tth moment. There are M nth, n=1,2...,N, eigenvectors at the tth moment, denoted as The vectors of different scales at the same time are input into the attention GRU module to learn the scale relationship dependency, and all the outputs are recorded as Finally, the output of each scale is fused into Calculate the N eigenvector sequence at the tth moment Take the average to get Z t , and finally for Z at T moments t Take the average to get Y attention ; In the fusion module, all feature spectrum sequences with different scales are taken as input and firstly transformed by the convolution operator. The number of channels is doubled, and then it is divided into four feature sub-graphs by interval sampling with a step size of 2. Then, the scores of each point on the four feature matrices can be calculated by the following formula to form a score matrix: Among them, log is the logarithmic function, abs is the absolute value operation; Based on the score matrix calculated above Extract the eigenvalue points with the highest scores at the same position between feature subgraphs to form a new feature matrix Then it is fused with the feature spectrum of the next scale and updated as follows: Update and recurse step by step to obtain the feature matrix fused along the scale direction Calculate X′ at each moment t The score of each point in the t as follows: S t =Sigmoid(log(abs(X′ t ))),t=1,2,...,T Based on the score matrix S calculated above t , extract the eigenvalue point with the highest score at the same position in the feature matrix at each moment to form the feature matrix Y′ that is fused along the time direction: Then, the feature matrix Y′ is divided into N feature vectors, denoted as Y″, using the same method as in the feature encoding module. n ; Calculate the average value of N eigenvectors to get Y fusion , and then with Y attention The added output O is used for subsequent classification tasks; Step 2: Use the trained attention-based Bi-GRU model to classify and recognize vein images.

2. The method according to claim 1, characterized in that In the feature encoding module, let X 1：T is a multi-view finger vein image sequence X1, X2, ..., X T , for the t-th image X t ∈R W×H , where W represents width and H represents height; In the feature encoding module, a convolution operator with a convolution kernel size of 3×3 and 64 channels is used for X t After downsampling, the output characteristic spectrum is expressed as Similarly, the next convolution and downsampling operation also uses a 3×3 convolution kernel, but the number of channels will be doubled, and the output feature spectrum can be expressed as After M convolution and downsampling operations, M feature spectra of different scales are obtained.

3. The method according to claim 2, characterized in that In the feature encoding module, a convolution pair with an output channel of d, a convolution kernel size of p×p, and a step size of p is used. Perform convolution operation to obtain N feature vectors, expressed as Where N = ((W / 2) × (H / 2)) / (p × p); N eigenvectors are combined into a matrix For the subsequent The output channel is d and the convolution kernel size is (p / 2 m-1 )×(p / 2 m-1 ), the step size is (p / 2 m-1 ) is divided into blocks and stretched, and the convolution is obtained 4. The method according to claim 1, characterized in that The attention GRU module contains three gates: reset gate, selection gate and update gate; The reset gate is responsible for determining which timing information of the previous moment needs to be retained, the selection gate is responsible for deciding which feature matrices of the current feature matrix interact with which feature matrices of the previous moment for timing information, and the update gate is used to determine the impact of the previous moment's timing information on the current timing information.

5. The method according to claim 4, characterized in that In the attention GRU module, take As the input at the current moment, let is the output of the previous hidden state at the mth scale, Mapped into three matrices through a multi-layer perceptron and After the multi-layer perceptron is converted into three matrices K m , K′ m and V m ; The selection of each gate is made by the Sigmoid activation function, and the calculation process of each gate is as follows: Where thres is the threshold value, the value range is 0≤thres≤1, σ is the sigmoid activation function, 0≤σ(·)≤1, and for and The relationship matrix, To reset the gate, and is the update gate, the output at the current moment Obtained by: Among them, Tanh is the hyperbolic tangent function, Concat is the concatenation function, is the hidden state at the current moment. Finally, the output at the current moment can be obtained through nonlinear mapping. Where: W′ is the coefficient matrix in the nonlinear mapping network here, and b′ is the bias in the nonlinear mapping here.

6. The method according to claim 1, characterized in that The outputs of the forward process and the backward process at time t are and The outputs of the forward process and the backward process are linearly mapped to as follows: Where: W1, W2 are coefficient matrices in the corresponding linear mapping network, and g is the linear mapping function.

7. The method according to claim 1, characterized in that The output of each scale is fused into as follows: Where: W″ is the coefficient matrix in the nonlinear mapping network here, and b″ is the bias in the nonlinear mapping here.

8. The method according to claim 1, characterized in that In the fusion module, the calculation process of the feature subgraph is as follows: Among them, Conv is a convolution operator with a convolution kernel size of 1×1, and Split represents an interval sampling method with a step size of 2.

9. The method according to claim 1, characterized in that: Extract the eigenvalue points with the highest scores at the same position between feature subgraphs to form a new feature matrix The process is as follows:

10. The method according to claim 1, characterized in that Extract the eigenvalue point with the highest score at the same position between the feature matrices at each moment to form the feature matrix Y that is fused along the time direction ′ The process is as follows:

Images