Feature extraction method and apparatus

Abstract

Provided in the present application are a feature extraction method and apparatus. In the present application, a plurality of finger vein images of the same finger which are captured at different angles are acquired, and then by means of the processing method in the present application, deformation of veins in each finger vein image, which deformation is caused by slight shaking of the finger, can be corrected on the basis of the plurality of finger vein images captured at the different angles, and thus finger vein features having relatively high accuracy are obtained, thereby facilitating an improvement in the accuracy of identity authentication performed by means of finger vein recognition.

Description

Feature extraction method and device

[0001] Cross-reference to related applications

[0002] The present application claims priority to the Chinese patent application No. 202410414988X, filed on April 8, 2024, and titled "A feature extraction method and device", the content of which is incorporated herein by reference in its entirety. TECHNICAL FIELD

[0003] The present application relates to the technical field of image processing, in particular to a feature extraction method and device. BACKGROUND

[0004] At present, biometric recognition technology is applied in many scenes, for example, biometric recognition technology is used to unlock door locks, intelligent storage boxes, etc., and vein feature recognition technology is also a biometric recognition technology applied in various scenes.

[0005] When performing vein feature recognition, the finger vein image of a finger needs to be collected through infrared imaging technology, so as to obtain the finger vein image of the finger, and then the finger vein feature of the finger is obtained, so as to perform identity authentication through the finger vein feature. Since the fingers of a person will vibrate slightly in a normal state, the fingers will rotate, translate and move, which causes the finger veins in the finger vein image to deform after the finger vein image is obtained through infrared imaging technology, so that the accuracy of the obtained finger static feature is relatively low, and thus the accuracy when performing identity authentication is reduced.

[0006] SUMMARY

[0007] Therefore, the embodiments of the present application provide a feature extraction method and device to improve the accuracy when performing identity authentication through finger veins.

[0008] In a first aspect, the embodiments of the present application provide a feature extraction method, which comprises:

[0009] After obtaining a plurality of finger vein images of a finger taken at different angles, the finger vein images are convoluted according to a preset order to obtain a label sequence corresponding to each feature block included in each finger vein image, wherein the label sequence corresponding to each feature block is composed of a one-dimensional vector corresponding to the feature block; the number of feature blocks is N, W is the width of the finger vein image, H is the height of the finger vein image, M is the height of the convolution kernel, and P is the width of the convolution kernel;

[0010] According to the position of each feature block included in the finger vein image in the finger vein image, the mark sequence corresponding to each feature block included in the finger vein image is matrix spliced to obtain a first matrix for representing the finger vein image;

[0011] According to the first matrix corresponding to each finger vein image, a local time relationship matrix Y local of each finger vein image at the same position is determined;

[0012] According to the first matrix corresponding to each finger vein image, a time relationship matrix Y LSTM of each finger vein image is determined;

[0013] According to the following formula, a first total time relationship matrix Y' between the finger vein images is obtained; Y'=Concat(Y LSTM ,Y local );

[0014] Where, Concat is a vector connection function;

[0015] According to the following formula, the total time relationship matrix Y' is nonlinearly mapped to obtain a second total time relationship matrix Y between the finger vein images:

[0016] Where, is a ReLU activation function, and W1 and W2 are weight matrices;

[0017] The second total time relationship matrix Y is input into a spatial attention module to obtain an output matrix containing time relationship dependence and spatial relationship dependence, and the output matrix is used as the features of the finger vein images taken at different angles.

[0018] Optionally, the determination of the local time relationship matrix Y local of each finger vein image at the same position according to the first matrix corresponding to each finger vein image comprises:

[0019] According to the first matrix corresponding to each finger vein image, the first matrix is segmented according to an LxL window to obtain S first sub-matrices of LxL size, 0<L<P;

[0020] The first sub-matrices on the same position window in each finger vein image are input into a local time attention module according to the preset order for linear mapping to obtain three mapping matrices Q s , K s and V s of the first sub-matrices;

[0021] According to the following formula, the local time relationship matrix times ;

[0022] wherein SM(·) is used for matrix multiplication operation on each row of the similarity matrix , d k is a normalization parameter, E s is a first preset increment, and T is a transpose operation of a matrix.

[0023] According to the positions of the windows, the local time relationship matrices of the window classes are matrix spliced to obtain the local time relationship matrix Y local .

[0024] Optionally, the time relationship matrix Y LSTM of each finger vein image is determined according to the first matrix corresponding to each finger vein image, and includes the following steps.

[0025] According to the preset order, the first matrix corresponding to each finger vein image is sequentially linearly mapped to obtain a query vector Q x , a key vector K x , and a value vector V x of each first matrix.

[0026] For each first matrix, a first value corresponding to the first matrix is calculated according to the following formula:

[0027] wherein Att is attention calculation, Softmax is a normalized exponential function, d k is a normalization parameter, B x is a second preset increment, and T is a transpose operation of a matrix.

[0028] A second value corresponding to the first matrix is calculated according to the following formula: Cross x =Att(Q x ,K h ,V h ).

[0029] A third value corresponding to the first matrix is calculated according to the following formula: Self h =Att(Q h ,K h ,V h ).

[0030] A fourth value corresponding to the first matrix is calculated according to the following formula: Cross h =Att(Q h ,K x ,V x ).

[0031] wherein Qh K is a query vector obtained by linear mapping of the first target matrix; h V is a key vector obtained by linear mapping of the first target matrix; h Q is a value vector obtained by linear mapping of the first target matrix, the hidden state corresponding to the previous first matrix of the first matrix in the preset order, when the first target matrix is the first first matrix in the preset order; h K and V are obtained after preset initialization of the first first matrix in the preset order; h and V are obtained after preset initialization of the first first matrix in the preset order; h

[0032] According to the following formula, the forgetting gate corresponding to the first matrix is calculated: f t =σ(linear(Concat(self x ,Cross h )));

[0033] Where σ is a sine function, linear is a linear mapping function, and Concat is a vector concatenation function.

[0034] According to the following formula, the input gate corresponding to the first matrix is calculated: i t =σ(linear(Concat(self x ,self h )));

[0035] According to the following formula, the fifth value corresponding to the first matrix is calculated:

[0036] According to the following formula, the storage unit corresponding to the first matrix is calculated:

[0037] Where C t-1 is the storage unit corresponding to the first target matrix, and C t-1 is the initialized storage unit when the first target matrix is the first first matrix in the preset order, and is a Hadamard product operator.

[0038] According to the following formula, the output gate corresponding to the first matrix is calculated: o t =σ(linear(Concat(Cross x ,self h )));

[0039] According to the following formula, the hidden state corresponding to the first matrix in the positive order is calculated: h t =o t ​tanh(C t );

[0040] wherein tanh is a hyperbolic tangent function;

[0041] determine a time relationship matrix Y LSTM of each finger vein image according to the hidden state corresponding to each first matrix in the normal order case

[0042] Optionally, the determining the time relationship matrix Y LSTM of each finger vein image according to the hidden state corresponding to each first matrix comprises:

[0043] perform linear mapping on the first matrix corresponding to each finger vein image in the reverse order of the preset order to obtain a query vector -Q x , a key vector -K x and a value vector -V x ;

[0044] For each first matrix, a first value corresponding to the first matrix is calculated according to the following formula:

[0045] A second value corresponding to the first matrix is calculated according to the following formula: -Cross x = Att(-Q x , -K h , -V h ) ;

[0046] A third value corresponding to the first matrix is calculated according to the following formula: -Self h = Att(-Q h , -K h , -V h ) ;

[0047] A fourth value corresponding to the first matrix is calculated according to the following formula: -Cross h = Att(-Q h , -K x , -V x ) ;

[0048] wherein -Q h is a query vector obtained by linear mapping on a second target matrix, -K h is a key vector obtained by linear mapping on the second target matrix, and -V ha value vector obtained by linear mapping the second target matrix, the second target matrix being a hidden state corresponding to a previous first matrix of the first matrix in a reverse order of the preset order, when the second target matrix is a first first matrix in the reverse order of the preset order, -Q h , -K h , and -V h are obtained after preset initialization of the first first matrix in the reverse order of the preset order;

[0049] The forget gate corresponding to the first matrix is calculated according to the following formula: -f t =σ(linear(Concat(-self x ,-Cross h )));

[0050] The input gate corresponding to the first matrix is calculated according to the following formula: -i t =σ(linear(Concat(-self x ,-self h )));

[0051] The fifth value corresponding to the first matrix is calculated according to the following formula:

[0052] The storage unit corresponding to the first matrix is calculated according to the following formula:

[0053] Wherein, -C t-1 is a storage unit corresponding to the second target matrix, when the second target matrix is the first first matrix in the reverse order of the preset order, -C t-1 is an initialized storage unit;

[0054] The output gate corresponding to the first matrix is calculated according to the following formula: -o t =σ(linear(Concat(-Cross x ,-self h )));

[0055] The hidden state corresponding to the first matrix in the reverse order is calculated according to the following formula: -h t =(-o t )*(tanh(-C t ));

[0056] The time relationship matrix Y LSTM of each finger vein image is determined according to the hidden state corresponding to each first matrix in the forward order and the hidden state corresponding to each first matrix in the reverse order.

[0057] Optionally, the time relationship matrix Y of each finger vein image is determined according to the hidden state corresponding to each first matrix in the normal sequence and the hidden state corresponding to each first matrix in the reverse sequence LSTM , comprising:

[0058] According to the preset order and the reverse order of the preset order, the hidden states corresponding to the first matrices on the same position are added to obtain a plurality of added matrices;

[0059] The obtained plurality of added matrices are spliced to obtain a spliced matrix, and the obtained spliced matrix is taken as the time relationship matrix Y LSTM .

[0060] In a second aspect, an embodiment of the present application provides a feature extraction device, and the device comprises:

[0061] A convolution processing unit is configured to, after obtaining a plurality of finger vein images of a finger taken at different angles, perform convolution processing on each finger vein image according to a preset order to obtain a label sequence corresponding to each feature block included in each finger vein image, wherein the label sequence corresponding to each feature block is composed of a one-dimensional vector corresponding to the feature block; the number of feature blocks is N, W is the width of the finger vein image, H is the height of the finger vein image, M is the height of the convolution kernel, and P is the width of the convolution kernel;

[0062] A splicing unit is configured to, for each finger vein image, perform matrix splicing on the label sequence corresponding to each feature block included in the finger vein image according to the position of each feature block in the finger vein image to obtain a first matrix for representing the finger vein image;

[0063] A first determination unit is configured to determine a local time relationship matrix Y local of each finger vein image at the same position according to the first matrix corresponding to each finger vein image;

[0064] A second determination unit is configured to determine a time relationship matrix Y LSTM of each finger vein image according to the first matrix corresponding to each finger vein image;

[0065] A calculation unit is configured to obtain a first total time relationship matrix Y' between each finger vein image according to the following formula; Y' = Concat(Y LSTM , Y local );

[0066] Wherein, Concat is a vector concatenation function;

[0067] mapping unit configured to perform nonlinear mapping on the total time relationship matrix Y' according to the following formula to obtain a second total time relationship matrix Y between the finger vein images:

[0068] wherein, is a ReLU activation function, and W1 and W2 are weight matrices;

[0069] an output unit configured to input the second total time relationship matrix Y into a spatial attention module to obtain an output matrix containing time relationship dependency and spatial relationship dependency, and use the output matrix as features of the finger vein images taken at different angles.

[0070] Optionally, when the first determination unit is configured to determine the local time relationship matrix Y local for each finger vein image according to the first matrix corresponding to the finger vein image,

[0071] the first matrix corresponding to each finger vein image is segmented according to an LxL window to obtain S first sub-matrices of LxL size, 0

[0072] the first sub-matrices on the same position window in the finger vein images are input into a local time attention module according to the preset order for linear mapping to obtain three mapping matrices Q s , K s , and V s corresponding to the first sub-matrices;

[0073] the local time relationship matrix time s of the window class corresponding to the window is obtained according to the following formula;

[0074] wherein, SM(·) is configured to perform matrix multiplication on each row of the similarity matrix , d k is a normalization parameter, E s is a first preset increment, and T is a transposition operation of a matrix;

[0075] the local time relationship matrices of the window classes are spliced according to the positions of the windows to obtain the local time relationship matrix Y local .

[0076] Optionally, when the second determination unit is configured to determine the time relationship matrix Y LSTM for each finger vein image according to the first matrix corresponding to the finger vein image,

[0077] According to the preset order, the first matrix corresponding to each finger vein image is sequentially subjected to linear mapping to obtain a query vector Q of each first matrix x , a key vector K x , and a value vector V x ;

[0078] For each first matrix, a first value corresponding to the first matrix is calculated according to the following formula:

[0079] Wherein, Att is attention calculation, Softmax is a normalized exponential function, d k is a normalization parameter, B x is a second preset increment, and T is a transpose operation of the matrix.

[0080] A second value corresponding to the first matrix is calculated according to the following formula: Cross x = Att(Q x , K h , V h );

[0081] A third value corresponding to the first matrix is calculated according to the following formula: Self h = Att(Q h , K h , V h );

[0082] A fourth value corresponding to the first matrix is calculated according to the following formula: Cross h = Att(Q h , K x , V x );

[0083] Wherein, Q h is a query vector obtained by linear mapping of a first target matrix, K h is a key vector obtained by linear mapping of the first target matrix, V h is a value vector obtained by linear mapping of the first target matrix, the first target matrix is a hidden state corresponding to a previous first matrix of the first matrix when sorted according to the preset order, when the first target matrix is the first first matrix in the preset order, Q h , K h , and V h are obtained after preset initialization of the first first matrix in the preset order.

[0084] A forgetting gate corresponding to the first matrix is calculated according to the following formula: f t = σ(linear(Concat(selfx Cross h )));

[0085] wherein, σ is a sine function, linear is a linear mapping function, and Concat is a vector concatenation function;

[0086] The input gate corresponding to the first matrix is calculated according to the following formula: t = σ (linear (Concat (self x , self h ))) ;

[0087] The fifth value corresponding to the first matrix is calculated according to the following formula:

[0088] The storage unit corresponding to the first matrix is calculated according to the following formula:

[0089] wherein, C t-1 is the storage unit corresponding to the first target matrix, and when the first target matrix is the first first matrix in the preset order, C t-1 is an initialized storage unit, and is a Hadamard product operator;

[0090] The output gate corresponding to the first matrix is calculated according to the following formula: t = σ (linear (Concat (Cross x , self h ))) ;

[0091] The hidden state corresponding to the first matrix in the normal order is calculated according to the following formula: t = o t * tanh (C t ) ;

[0092] wherein, tanh is a hyperbolic tangent function;

[0093] The time relationship matrix Y LSTM of the finger vein images is determined according to the hidden state corresponding to each first matrix in the normal order.

[0094] Optionally, when the determining unit is used to determine the time relationship matrix Y LSTM of the finger vein images according to the hidden state corresponding to each first matrix, it comprises:

[0095] According to the reverse order of the preset order, the first matrix corresponding to each finger vein image is sequentially linearly mapped to obtain the query vector-Q x and the key vector-K xand value vector -V x ;

[0096] For each first matrix, a first numerical value corresponding to the first matrix is calculated according to the following formula:

[0097] A second numerical value corresponding to the first matrix is calculated according to the following formula: -Cross x = Att(-Q x , -K h , -V h );

[0098] A third numerical value corresponding to the first matrix is calculated according to the following formula: -Self h = Att(-Q h , -K h , -V h );

[0099] A fourth numerical value corresponding to the first matrix is calculated according to the following formula: -Cross h = Att(-Q h , -K x , -V x );

[0100] Wherein, -Q h is a query vector obtained by linearly mapping a second target matrix, -K h is a key vector obtained by linearly mapping the second target matrix, -V h is a value vector obtained by linearly mapping the second target matrix, the second target matrix is a hidden state corresponding to a previous first matrix in the first matrix when the second target matrix is in the reverse order of the preset order, and -Q h , -K h and -V h are obtained by presetting and initializing the first matrix in the reverse order of the preset order when the second target matrix is the first matrix in the reverse order of the preset order.

[0101] A forgetting gate corresponding to the first matrix is calculated according to the following formula: -f t = σ(linear(Concat(-self x , -Cross h )));

[0102] An input gate corresponding to the first matrix is calculated according to the following formula: -i t = σ(linear(Concat(-self x , -self h )));

[0103] The fifth value corresponding to the first matrix is calculated according to the following formula:

[0104] The storage unit corresponding to the first matrix is calculated according to the following formula:

[0105] Wherein, -C t-1 is the storage unit corresponding to the second target matrix, when the second target matrix is the first matrix in the reverse order of the preset order, -C t-1 is the initialized storage unit;

[0106] The output gate corresponding to the first matrix is calculated according to the following formula: -o t =σ(linear(Concat(-Cross x ,-self h )));

[0107] The hidden state corresponding to the first matrix in the reverse order is calculated according to the following formula: -h t =(-o t )*(tanh(-C t ));

[0108] The time relationship matrix Y LSTM of each finger vein image is determined according to the hidden state corresponding to each first matrix in the forward order and the hidden state corresponding to each first matrix in the reverse order.

[0109] Optionally, when the second determining unit is used to determine the time relationship matrix Y LSTM of each finger vein image according to the hidden state corresponding to each first matrix in the forward order and the hidden state corresponding to each first matrix in the reverse order, the second determining unit comprises:

[0110] The hidden states corresponding to the first matrices in the same position are added according to the preset order and the reverse order of the preset order, to obtain a plurality of added matrices;

[0111] The obtained plurality of added matrices are spliced to obtain a spliced matrix, and the obtained spliced matrix is taken as the time relationship matrix Y LSTM .

[0112] The technical scheme provided by the embodiments of the present application can include the following beneficial effects:

[0113] In the present application, multiple finger vein images of the same finger are acquired at different angles, and then the deformation of the veins in each finger vein image due to slight shaking of the finger is corrected through the processing manner in the present application, so that the finger vein features with relatively high accuracy are obtained, and then the accuracy of identity authentication through finger vein recognition is improved.

[0114] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the following preferred embodiments are specifically described below, and the accompanying drawings are described in detail as follows. BRIEF DESCRIPTION OF DRAWINGS

[0115] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0116] Fig. 1 is a flow diagram of a feature extraction method provided by an embodiment of the present application;

[0117] Fig. 2 is a structural diagram of a feature extraction device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0118] In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the technical solutions in the embodiments of the present application will be described clearly and completely in the following by combining the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, but not all the embodiments. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of the present application.

[0119] The existing vein recognition technology uses a fixed single camera and a single light source to perform infrared imaging on a vein image, and then uses the finger vein image to achieve identity authentication. However, this imaging method has a problem: the projection of a three-dimensional finger vein on a two-dimensional image may be distorted due to the change in the position of the finger, which is referred to as the multi-pose problem of the finger. Two-dimensional finger vein recognition may cause the finger vein image to change due to the rotation or translation of the finger along the X-axis, Y-axis and Z-axis. If the finger is translated along the X-axis and Y-axis, the position of the finger region in the finger vein image will change; if the finger is translated along the Z-axis, the proportion of the finger region in the finger vein image will change; if the finger is rotated along the X-axis, Y-axis and Z-axis, the finger region in the finger vein image will change more severely. These three changes that occur when collecting the finger vein image are referred to as roll, pitch and yaw, respectively. Yaw may cause the finger to rotate in the two-dimensional finger vein image; pitch may cause the finger to deform in the two-dimensional finger vein image, thereby causing the veins in the two-dimensional finger vein image to be distorted; roll may change the projection view of the three-dimensional finger vein on the two-dimensional plane. Therefore, there is a large difference between the finger vein images of the same finger collected at different angles, which directly affects the recognition performance of the finger vein and reduces the accuracy of the finger vein recognition. In real life, it is difficult to ensure that the finger is placed in an ideal state, and the sampling process cannot be performed in an ideal state (i.e., the finger does not rotate or translate). The user's finger will inevitably rotate or translate, causing the existing vein recognition system to fail. Therefore, how to eliminate the influence of finger roll, pitch and yaw in the finger vein image and how to more effectively extract the vein features of the finger vein to improve the recognition performance have become one of the key problems in the research on finger vein recognition.

[0120] To solve the above problem, in the present application, multiple finger vein images of the same finger are collected at different angles, and then the deformation of the veins in each finger vein image due to the slight shaking of the finger is corrected by the processing method in the present application, so as to obtain finger vein features with relatively high accuracy, thereby facilitating the improvement of the accuracy of identity authentication by finger vein recognition.

[0121] The following is a detailed description of the embodiments of the present application.

[0122] FIG. 1 is a flowchart of a feature extraction method according to an embodiment of the present application. As shown in FIG. 1, the method includes the following steps:

[0123] In step 101, after obtaining a plurality of finger vein images of a finger taken at different angles, the convolution processing is performed on each finger vein image according to a preset order, to obtain a label sequence corresponding to each feature block included in each finger vein image, wherein the label sequence corresponding to each feature block is composed of a one-dimensional vector corresponding to the feature block, and the number of feature blocks is N, W is the width of the finger vein image, H is the height of the finger vein image, M is the height of the convolution kernel, and P is the width of the convolution kernel.

[0124] Specifically, when the finger of the user is collected by the infrared imaging technology, the finger can be photographed around to obtain a plurality of finger vein images taken at different angles. Preferably, the plurality of finger vein images can be spliced to obtain a complete three-dimensional finger vein image of the finger. After obtaining the plurality of finger vein images, the convolution processing can be performed on each finger vein image according to the collection order of the finger vein images or according to the preset order of the user's sorting of the plurality of finger vein images, to obtain a label sequence corresponding to each feature block included in each finger vein image. For example, the finger vein image of the user's right index finger is collected, and the collected finger vein images include finger vein image 1, finger vein image 2 and finger vein image 3. The convolution processing can be performed on the above three finger vein images in the order of finger vein image 1, finger vein image 2 and finger vein image 3. Taking finger vein image 1 as an example, the finger vein image 1 can be divided into a plurality of feature blocks, and a one-dimensional vector corresponding to each feature block is obtained as the label sequence of the feature block. After the convolution processing of the finger vein image, the number of feature blocks is related to the convolution and the finger vein image itself, wherein the number of feature blocks is N, W is the width of the finger vein image, H is the height of the finger vein image, M is the height of the convolution kernel, and P is the width of the convolution kernel.

[0125] In step 102, for each finger vein image, the label sequences corresponding to the feature blocks included in the finger vein image are matrix spliced according to the positions of the feature blocks in the finger vein image, to obtain a first matrix for representing the finger vein image.

[0126] Specifically, since each feature block is obtained by dividing the finger vein image, each feature block corresponds to a position in the finger vein image. Then, the label sequences corresponding to the feature blocks adjacent in position are matrix spliced according to the positions of the feature blocks in the finger vein image, to obtain the first matrix of the finger vein image. At this time, the first matrix of each finger vein image can be obtained.

[0127] In step 103, a local time relationship matrix Y of the finger vein images at the same position is determined according to the first matrices corresponding to the finger vein images.local .

[0128] Specifically, after obtaining the first matrix corresponding to each of the finger vein images, the local time relationship matrix Y of the finger vein images at the same position is determined by using the first matrix corresponding to each of the finger vein images, so as to perform local attention calculation. local .

[0129] Step 104, determining the time relationship matrix Y of the finger vein images according to the first matrix corresponding to each of the finger vein images. LSTM .

[0130] Step 105, obtaining the first total time relationship matrix Y' between the finger vein images according to the following formula one: Y'=Concat(Y LSTM , Y local ); (formula one)

[0131] Wherein, Concat is a vector connection function.

[0132] Step 106, performing nonlinear mapping on the total time relationship matrix Y' according to the following formula two to obtain the second total time relationship matrix Y between the finger vein images:

[0133] Wherein, is the ReLU activation function, and W1 and W2 are weight matrices.

[0134] Step 107, inputting the second total time relationship matrix Y into the spatial attention module to obtain an output matrix containing time relationship dependence and spatial relationship dependence, and taking the output matrix as the feature of the finger vein images photographed at different angles.

[0135] Specifically, through the processing manner in the present application, the deformation of the veins in each finger vein image due to slight shaking of the finger can be corrected, so that the finger vein feature with relatively high accuracy is obtained, and then the accuracy of identity authentication through finger vein recognition is improved.

[0136] In a feasible implementation, when step 103 is performed, the following steps can be implemented:

[0137] Step 201, for the first matrix corresponding to each finger vein image, the first matrix is segmented according to an LxL window to obtain S first sub-matrices with the size of LxL, 0

[0138] Step 202, inputting the first sub-matrix on the same position window in each finger vein image into the local time attention module according to the preset order for linear mapping to obtain three mapping matrices Q s , K s and V s .

[0139] Step 203, obtaining the local time relationship matrix of the window class corresponding to the window according to the following formula s .

[0140] Wherein, SM(·) is used for matrix multiplication operation on each row of the similarity matrix , d k is a normalization parameter, E s is a first preset increment, and T is a matrix transposition operation.

[0141] Step 204, performing matrix splicing on the local time relationship matrix of each window class according to the position of each window to obtain the local time relationship matrix Y local .

[0142] Specifically, taking the finger vein image 1 and the finger vein image 2 as examples, after obtaining the first matrix corresponding to the finger vein image 1 and the finger vein image 2 respectively, the first matrix can be divided by the given window size, each first matrix can be divided into a plurality of first sub-matrices, and then the first sub-matrices on the same position in the first matrix are input into the local time attention module according to the order of convolution processing of the first sub-matrices belonging to the finger vein image for linear mapping to obtain three mapping matrices Q s , K s and V s of the first sub-matrix. Taking the finger vein image 1, the finger vein image 2 and the finger vein image 3 as examples, the order of convolution processing of the finger vein image 1, the finger vein image 2 and the finger vein image 3 is the order of finger vein image 1→finger vein image 2→finger vein image 3 for convolution processing. After obtaining the first matrix (i.e., the first matrix 1, the first matrix 2 and the first matrix 3) corresponding to the finger vein image 1, the finger vein image 2 and the finger vein image 3 respectively, the first matrix 1, the first matrix 2 and the first matrix 3 are divided by the window of L×L respectively, and each first matrix can obtain a plurality of first sub-matrices. When the first sub-matrix 1 in the first matrix 1, the first sub-matrix 2 in the first matrix 2 and the first sub-matrix 3 in the first matrix 3 are first sub-matrices on the same position window, the above three first sub-matrices are input into the local time attention module according to the arrangement of first sub-matrix 1→first sub-matrix 2→first sub-matrix 3 for linear mapping to obtain three mapping matrices Q s , Ks and V s .

[0143] After obtaining the mapping matrix of the first submatrix corresponding to all windows, the local time relationship matrix time of the window class corresponding to each window is calculated according to the method of step 203 for the first submatrix corresponding to each window. s , after obtaining the local time relationship matrix time of the window class corresponding to all windows s After that, the local time relationship matrix time is obtained s The number of is S, that is, the same as the number of the first submatrices obtained after segmenting the first matrix of each finger vein image. Then, according to the position of each window in the first matrix, the local time relationship matrix of each window class is concatenated to obtain the local time relationship matrix Y local , where the local time relationship matrix Y local The matrix shape is the same as the first matrix.

[0144] In a feasible embodiment, when executing step 104, it can be achieved by the following steps:

[0145] Step 301: linearly map the first matrices corresponding to the finger vein images in sequence according to the preset order to obtain the query vector Q of each first matrix. x , key vector K x Sum value vector V x .

[0146] Step 302: For each first matrix, calculate the first value corresponding to the first matrix according to the following formula:

[0147] Among them, Att is the attention calculation, Softmax is the normalized exponential function, d k is the normalization parameter, B x is the second preset increment, and T is the matrix transpose operation.

[0148] Step 303: Calculate the second value corresponding to the first matrix according to the following formula: Cross x =Att(Q x ,K h ,V h ).

[0149] Step 304: Calculate the third value corresponding to the first matrix according to the following formula: Self h =Att(Q h ,K h ,V h ).

[0150] Step 305, calculating a fourth value corresponding to the first matrix according to the following formula: Cross h = Att(Q h , K x , V x );

[0151] Wherein, Q h is a query vector obtained by linear mapping of the first target matrix, K h is a key vector obtained by linear mapping of the first target matrix, and V h is a value vector obtained by linear mapping of the first target matrix, the first target matrix is a hidden state corresponding to a previous first matrix of the first matrix when sorted according to the preset order, when the first target matrix is the first first matrix in the preset order, Q h , K h and V h are obtained after preset initialization of the first first matrix in the preset order.

[0152] Step 306, calculating a forget gate corresponding to the first matrix according to the following formula: f t = σ(linear(Concat(self x , Cross h )));

[0153] Wherein, σ is a sine function, linear is a linear mapping function, and Concat is a vector connection function.

[0154] Step 307, calculating an input gate corresponding to the first matrix according to the following formula: i t = σ(linear(Concat(self x , self h )))。

[0155] Step 308, calculating a fifth value corresponding to the first matrix according to the following formula:

[0156] Step 309, calculating a storage unit corresponding to the first matrix according to the following formula:

[0157] Wherein, C t-1 is a storage unit corresponding to the first target matrix, when the first target matrix is the first first matrix in the preset order, C t-1 is an initialized storage unit, and is a Hadamard product operator.

[0158] Step 310, calculating the output gate corresponding to the first matrix according to the following formula: o t =σ(linear(Concat(Cross x ,self h )))).

[0159] Step 311, calculating the hidden state corresponding to the first matrix in the normal order according to the following formula: h t =o t *tanh(C t );

[0160] Wherein, tanh is the hyperbolic tangent function.

[0161] Step 312, determining the time relationship matrix Y LSTM of each finger vein image according to the hidden state corresponding to each first matrix in the normal order.

[0162] Specifically, taking the preset order of the finger vein image 1→the finger vein image 2→the finger vein image 3 as an example, when calculating the query vector Q x , the key vector K x and the value vector V x of the first matrix corresponding to the finger vein image 1, the query vector Q h , the key vector K h and the key vector V h of the first matrix corresponding to the finger vein image 1 are obtained by presetting the initialization of the first matrix corresponding to the finger vein image 1, then the query vector Q x , the key vector K x and the value vector V x of the first matrix corresponding to the finger vein image 1 are obtained by linear mapping the first matrix corresponding to the finger vein image 1, then the hidden state of the first matrix corresponding to the finger vein image 1 is obtained by using the technical solutions of steps 302-311, and the hidden state is taken as the time relationship matrix Y LSTM of the finger vein image 1, then the query vector Q h , the key vector K h and the key vector V h of the hidden state of the first matrix corresponding to the finger vein image 1 are obtained by linear mapping the hidden state of the first matrix corresponding to the finger vein image 1, and the query vector Q x , the key vector K x and the value vector V x of the first matrix corresponding to the finger vein image 2 are obtained by linear mapping the first matrix corresponding to the finger vein image 2.Then, the hidden state of the first matrix corresponding to the finger vein image 2 is obtained by using the technical solutions of steps 302-311, and when the hidden state of the first matrix corresponding to the finger vein image 3 is obtained, the hidden state of the first matrix corresponding to the finger vein image 2 is linearly mapped to obtain a query vector Q h , a key vector K h and a value vector V h of the hidden state, and the first matrix corresponding to the finger vein image 3 is linearly mapped to obtain a query vector Q x , a key vector K x and a value vector V x of the first matrix corresponding to the finger vein image 3.

[0163] In a feasible implementation, when step 312 is performed, the following steps can be implemented:

[0164] Step 401, linearly map the first matrix corresponding to each finger vein image in the reverse order of the preset order to obtain a query vector Q x , a key vector K x and a value vector V x of each first matrix.

[0165] Step 402, for each first matrix, calculate a first value corresponding to the first matrix according to the following formula:

[0166] Step 403, calculate a second value corresponding to the first matrix according to the following formula: -Cross x =Att(-Q x ,-K h ,-V h ).

[0167] Step 404, calculate a third value corresponding to the first matrix according to the following formula: -Self h =Att(-Q h ,-K h ,-V h ).

[0168] Step 405, calculate a fourth value corresponding to the first matrix according to the following formula: -Cross h =Att(-Q h ,-K x ,-V x ).

[0169] Wherein, -Q his a query vector obtained by linearly mapping the second target matrix, -K h is a key vector obtained by linearly mapping the second target matrix, -V h is a value vector obtained by linearly mapping the second target matrix, the second target matrix being a hidden state corresponding to a previous first matrix in the first matrix when the second target matrix is the first first matrix in the reverse order of the preset order, -Q h , -K h and -V h are obtained after preset initialization of the first first matrix in the reverse order of the preset order.

[0170] Step 406, calculating a forget gate corresponding to the first matrix according to the following formula: -f t = σ(linear(Concat(-self x , -Cross h ))).

[0171] Step 407, calculating an input gate corresponding to the first matrix according to the following formula: -i t = σ(linear(Concat(-self x , -self h ))).

[0172] Step 408, calculating a fifth numerical value corresponding to the first matrix according to the following formula:

[0173] Step 409, calculating a memory cell corresponding to the first matrix according to the following formula:

[0174] wherein -C t-1 is a memory cell corresponding to the second target matrix, -C t-1 is an initialized memory cell when the second target matrix is the first first matrix in the reverse order of the preset order.

[0175] Step 410, calculating an output gate corresponding to the first matrix according to the following formula: -o t = σ(linear(Concat(-Cross x , -self h ))).

[0176] Step 411, calculating a hidden state corresponding to the first matrix in the reverse order according to the following formula: -h t = (-o t ) * (tanh(-Ct ))).

[0177] Step 412, determining the time relationship matrix Y of each finger vein image according to the hidden state corresponding to each first matrix in the normal order and the hidden state corresponding to each first matrix in the reverse order. LSTM .

[0178] Specifically, taking the normal order of the preset order of the finger vein image 1→the finger vein image 2→the finger vein image 3 as an example, the reverse order of the above preset order is the order of the finger vein image 3→the finger vein image 2→the finger vein image 1, and the detailed description of steps 401-412 can refer to the related description of steps 301-312, and will not be described in detail here.

[0179] In a feasible implementation, when step 412 is performed, the following steps can be used to achieve it:

[0180] Step 501, adding the hidden states corresponding to the first matrices in the same position according to the preset order and the reverse order of the preset order to obtain a plurality of added matrices.

[0181] Step 502, splicing the plurality of added matrices to obtain a spliced matrix, and taking the spliced matrix as the time relationship matrix Y of the finger vein images. LSTM .

[0182] Specifically, when the finger vein images include the finger vein image 1, the finger vein image 2 and the finger vein image 3, and the normal order of the preset order is the finger vein image 1→the finger vein image 2→the finger vein image 3, three hidden states corresponding to the normal order can be obtained: the hidden state 1 of the first matrix corresponding to the finger vein image 1, the hidden state 2 of the first matrix corresponding to the finger vein image 2 and the hidden state 3 of the first matrix corresponding to the finger vein image 3, and three hidden states corresponding to the reverse order can also be obtained: the hidden state 4 of the first matrix corresponding to the finger vein image 3, the hidden state 5 of the first matrix corresponding to the finger vein image 2 and the hidden state 6 of the first matrix corresponding to the finger vein image 1, then the hidden state 1 and the hidden state 4 are added to obtain the added matrix 1, the hidden state 2 and the hidden state 5 are added to obtain the added matrix 2, and the hidden state 3 and the hidden state 6 are added to obtain the added matrix 3, then the added matrix 1, the added matrix 2 and the added matrix 3 are spliced to obtain the spliced matrix, and the spliced matrix is taken as the time relationship matrix Y of the finger vein image 1, the finger vein image 2 and the finger vein image 3. LSTM , wherein the time relationship matrix Y is the same as the matrix shape of the first matrix. LSTM .

[0183] FIG. 2 is a structural schematic diagram of a feature extraction device provided by an embodiment of the present application, as shown in FIG. 2, the device comprises:

[0184] the convolution processing unit 21 is configured to, after obtaining a plurality of finger vein images of a finger taken at different angles, perform convolution processing on each of the finger vein images in a preset order to obtain a label sequence corresponding to each feature block included in each of the finger vein images, wherein the label sequence corresponding to each feature block is composed of a one-dimensional vector corresponding to the feature block, and the number of feature blocks is N, W is the width of the finger vein image, H is the height of the finger vein image, M is the height of the convolution kernel, and P is the width of the convolution kernel;

[0185] the splicing unit 22 is configured to, for each of the finger vein images, perform matrix splicing on the label sequence corresponding to each of the feature blocks included in the finger vein image according to the positions of the feature blocks in the finger vein image to obtain a first matrix for representing the finger vein image;

[0186] the first determination unit 23 is configured to determine a local time relationship matrix Y local of the finger vein images at the same position according to the first matrices corresponding to the finger vein images;

[0187] the second determination unit 24 is configured to determine a time relationship matrix Y LSTM of the finger vein images according to the first matrices corresponding to the finger vein images;

[0188] the calculation unit 25 is configured to obtain a first total time relationship matrix Y' between the finger vein images according to the following formula: Y' = Concat(Y LSTM , Y local );

[0189] wherein Concat is a vector concatenation function;

[0190] the mapping unit 26 is configured to perform nonlinear mapping on the total time relationship matrix Y' according to the following formula to obtain a second total time relationship matrix Y between the finger vein images:

[0191] wherein is a ReLU activation function, and W1 and W2 are weight matrices;

[0192] the output unit 27 is configured to input the second total time relationship matrix Y into a spatial attention module to obtain an output matrix containing time relationship dependency and spatial relationship dependency, and perform multi-angle feature extraction on the output matrix.

[0193] In a feasible implementation, the first determination unit is configured to determine a local time relationship matrix Ylocal When including:

[0194] For the first matrix corresponding to each finger vein image, the first matrix is ​​divided according to the L×L window to obtain S first sub-matrices of L×L size, 0 <L<P;

[0195] The first sub-matrix on the same position window in each finger vein image is input into the local time attention module according to the preset order for linear mapping to obtain the three mapping matrices Q of the first sub-matrix s , K s and V s ;

[0196] According to the following formula, the local time relationship matrix time of the window class corresponding to the window is obtained s ;

[0197] Among them, SM(·) is used to calculate the similarity matrix Perform matrix multiplication on each row of d k is the normalization parameter, E s is the first preset increment, T is the transpose operation of the matrix;

[0198] According to the position of each window, the local time relationship matrix of each window class is concatenated to obtain the local time relationship matrix Y local .

[0199] In a feasible embodiment, the second determining unit is used to determine the time relationship matrix Y of each finger vein image according to the first matrix corresponding to each finger vein image. LSTM When including:

[0200] According to the preset order, linear mapping is performed on the first matrix corresponding to each finger vein image in turn to obtain the query vector Q of each first matrix x , key vector K x Sum value vector V x ;

[0201] For each first matrix, calculate the first value corresponding to the first matrix according to the following formula:

[0202] Among them, Att is the attention calculation, Softmax is the normalized exponential function, d k is the normalization parameter, B x is the second preset increment, T is the transpose operation of the matrix;

[0203] According to the following formula, the second value corresponding to the first matrix is ​​calculated: Cross x =Att(Qx ,K h ,V H );

[0204] The third value corresponding to the first matrix is calculated according to the following formula: Self h =Att(Q h ,K h ,V h );

[0205] The fourth value corresponding to the first matrix is calculated according to the following formula: Cross h =Att(Q h ,K x ,V x );

[0206] Wherein, Q h is a query vector obtained by linearly mapping a first target matrix, K h is a key vector obtained by linearly mapping the first target matrix, V h is a value vector obtained by linearly mapping the first target matrix, the first target matrix is a hidden state corresponding to a previous first matrix of the first matrix when sorted according to the preset order, when the first target matrix is the first first matrix in the preset order, Q h , K h and V h are obtained after preset initialization of the first first matrix in the preset order;

[0207] The forgetting gate corresponding to the first matrix is calculated according to the following formula: f t =σ(linear(Concat(self x ,Cross h )));

[0208] Wherein, σ is a sine function, linear is a linear mapping function, and Concat is a vector concatenation function;

[0209] The input gate corresponding to the first matrix is calculated according to the following formula: i t =σ(linear(Concat(self x ,self h )));

[0210] The fifth value corresponding to the first matrix is calculated according to the following formula:

[0211] The storage unit corresponding to the first matrix is calculated according to the following formula:

[0212] wherein C t-1 is the storage unit corresponding to the first target matrix, when the first target matrix is the first first matrix in the preset order, C t-1 is an initialized storage unit, and is a Hadamard product operator;

[0213] According to the following formula, the output gate corresponding to the first matrix is calculated: o t = σ(linear(Concat(Cross x , self h )));

[0214] According to the following formula, the hidden state corresponding to the first matrix in the forward order is calculated: h t = o t *tanh(C t );

[0215] wherein tanh is the hyperbolic tangent function;

[0216] According to the hidden state corresponding to each first matrix in the forward order, the time relationship matrix Y LSTM of each finger vein image is determined.

[0217] In a feasible implementation, when the determining unit is used to determine the time relationship matrix Y LSTM of each finger vein image according to the hidden state corresponding to each first matrix, it includes:

[0218] According to the reverse order of the preset order, the first matrix corresponding to each finger vein image is sequentially linearly mapped to obtain the query vector-Q x , the key vector-K x and the value vector-V x of each first matrix;

[0219] According to the following formula, the first value corresponding to each first matrix is calculated:

[0220] According to the following formula, the second value corresponding to the first matrix is calculated: -Cross x = Att(-Q x ,-K h ,-V h );

[0221] According to the following formula, the third value corresponding to the first matrix is calculated: -Self h = Att(-Q h ,-K h ,-V h );

[0222] The fourth value corresponding to the first matrix is calculated according to the following formula: -Cross h = Att(-Q h , -K x , -V x );

[0223] wherein -Q h is a query vector obtained by linearly mapping a second target matrix, -K h is a key vector obtained by linearly mapping the second target matrix, and -V h is a value vector obtained by linearly mapping the second target matrix, the second target matrix being a hidden state corresponding to a previous first matrix of the first matrix in descending order of the preset order, when the second target matrix is a first first matrix in descending order of the preset order, -Q h , -K h , and -V h being obtained after preset initialization of the first first matrix in descending order of the preset order;

[0224] The forgetting gate corresponding to the first matrix is calculated according to the following formula: -f t = σ(linear(Concat(-self x , -Cross h )));

[0225] The input gate corresponding to the first matrix is calculated according to the following formula: -i t = σ(linear(Concat(-self x , -self h )));

[0226] The fifth value corresponding to the first matrix is calculated according to the following formula:

[0227] The storage unit corresponding to the first matrix is calculated according to the following formula:

[0228] wherein -C t-1 is a storage unit corresponding to the second target matrix, when the second target matrix is a first first matrix in descending order of the preset order, -C t-1 being an initialized storage unit;

[0229] The output gate corresponding to the first matrix is calculated according to the following formula: -o t = σ(linear(Concat(-Cross x , -selfh )));

[0230] According to the following formula, the hidden state corresponding to the first matrix in the reverse order is calculated; t =(-o t )*(tanh(-C t ));

[0231] Determine the time relationship matrix Y of each finger vein image according to the hidden state corresponding to each first matrix in the positive order and the hidden state corresponding to each first matrix in the reverse order LSTM .

[0232] In a feasible embodiment, the second determining unit is used to determine the time relationship matrix Y of each finger vein image according to the hidden state corresponding to each first matrix in the positive order and the hidden state corresponding to each first matrix in the reverse order. LSTM When, including:

[0233] According to the preset order and the reverse order of the preset order, the hidden states corresponding to the first matrix at the same position are added to obtain a plurality of addition matrices;

[0234] The obtained multiple addition matrices are spliced ​​to obtain a spliced ​​matrix, and the obtained spliced ​​matrix is ​​used as the time relationship matrix Y LSTM .

[0235] For the relevant description of the content shown in FIG2 , please refer to the detailed description of the content shown in FIG1 , which will not be described in detail here.

[0236] In the embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interface, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0237] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0238] In addition, each of the functional units in the embodiments of the present application can be integrated in one processing unit, or each can exist as an independent physical unit, or two or more than two of them can be integrated in one physical unit.

[0239] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the parts of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0240] It should be noted that: similar reference numerals and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings, in addition, the terms "first", "second", "third" and the like are only used to distinguish description, and cannot be understood as indicating or implying relative importance.

[0241] Finally, it should be noted that: the above-described embodiments are merely specific embodiments of the present application, used to illustrate the technical solutions of the present application, and not to limit the same, the protection scope of the present application is not limited thereto, although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any skilled person in the art can modify or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present application, or make equivalent replacement to some of the technical features; and these modifications, changes or replacements do not make the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application. All should be covered in the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A feature extraction method, characterized in that: The method comprises: After obtaining multiple finger vein images of a finger taken at different angles, convolution processing is performed on each finger vein image in a preset order to obtain a label sequence corresponding to each feature block included in each finger vein image, wherein the label sequence corresponding to each feature block is composed of a one-dimensional vector corresponding to the feature block; the number of feature blocks is N, W is the width of the finger vein image, H is the height of the finger vein image, M is the height of the convolution kernel, and P is the width of the convolution kernel; For each finger vein image, performing matrix concatenation on the tag sequences corresponding to the feature blocks included in the finger vein image according to the positions of the feature blocks in the finger vein image, to obtain a first matrix for representing the finger vein image; According to the first matrix corresponding to each finger vein image, the local time relationship matrix Y of each finger vein image at the same position is determined local ; Determine the time relationship matrix Y of each finger vein image according to the first matrix corresponding to each finger vein image LSTM ; According to the following formula, the first total time relationship matrix Y′ between each finger vein image is obtained; Y′=Concat(Y LSTM ,AND local ); Among them, Concat is the vector connection function; According to the following formula, the total time relationship matrix Y′ is nonlinearly mapped to obtain the second total time relationship matrix Y between the finger vein images: in, is the ReLU activation function, W1 and W2 are weight matrices; The second total time relationship matrix Y is input into the spatial attention module to obtain an output matrix containing time relationship dependency and spatial relationship dependency, and the output matrix is ​​used as the features of multiple finger vein images taken at different angles.

2. The method according to claim 1, wherein The local time relationship matrix Y of each finger vein image at the same position is determined based on the first matrix corresponding to each finger vein image. local ,include: For the first matrix corresponding to each finger vein image, the first matrix is ​​divided according to the L×L window to obtain S first sub-matrices of L×L size, 0 <L<P; The first sub-matrix on the same position window in each finger vein image is input into the local time attention module according to the preset order for linear mapping to obtain the three mapping matrices Q of the first sub-matrix s , K s and V s ; According to the following formula, the local time relationship matrix time of the window class corresponding to the window is obtained s ; Among them, SM(·) is used to calculate the similarity matrix Perform matrix multiplication on each row of d k is the normalization parameter, E s is the first preset increment, T is the transpose operation of the matrix; According to the position of each window, the local time relationship matrix of each window class is concatenated to obtain the local time relationship matrix T local .

3. The method according to claim 1, wherein The time relationship matrix Y of each finger vein image is determined according to the first matrix corresponding to each finger vein image. LSTM ,include: According to the preset order, linear mapping is performed on the first matrix corresponding to each finger vein image in turn to obtain the query vector W of each first matrix x , key vector K x Sum value vector V x ; For each first matrix, calculate the first value corresponding to the first matrix according to the following formula: Among them, Att is the attention calculation, Softmax is the normalized exponential function, d k is the normalization parameter, B x is the second preset increment, T is the transpose operation of the matrix; The second value corresponding to the first matrix is ​​calculated according to the following formula: Cross x =That(Q x ,K h ,V h ); The third value corresponding to the first matrix is ​​calculated according to the following formula: Self h =That(Q h ,K h ,V h ); The fourth value corresponding to the first matrix is ​​calculated according to the following formula: Cross h =That(Q h ,K x ,V x ); Among them, Q h is the query vector obtained by linearly mapping the first target matrix, K h is the key vector obtained by linearly mapping the first target matrix, V h is a value vector obtained by linearly mapping the first target matrix, wherein the first target matrix is ​​a hidden state corresponding to the first matrix above the first matrix when sorted in the preset order. When the first target matrix is ​​the first first matrix in the preset order, Q h 、k h and V h is obtained by performing a preset initialization on the first first matrix in the preset order; According to the following formula, calculate the forget gate corresponding to the first matrix: f t =σ(linear(Concat(self x ,Cross h ))); Among them, σ is the sine function, linear is the linear mapping function, and Concat is the vector connection function; According to the following formula, the input gate corresponding to the first matrix is ​​calculated: i t =σ(linear(Concat(self x ,self h ))); The fifth value corresponding to the first matrix is ​​calculated according to the following formula: The storage unit corresponding to the first matrix is ​​calculated according to the following formula: Among them, C t-1 is the storage unit corresponding to the first target matrix. When the first target matrix is ​​the first first matrix in the preset order, C t-1 is the initialized storage unit, ⊙ is the Hadamard product operator; According to the following formula, the output gate corresponding to the first matrix is ​​calculated: 0 t =σ(linear(Concat(Cross x ,self h ))); According to the following formula, calculate the hidden state corresponding to the first matrix in the positive order; h t =o t *fishy(C) t ); Among them, tanh is the hyperbolic tangent function; According to the hidden state corresponding to each first matrix in the positive order, the time relationship matrix Y of each finger vein image is determined LSTM .

4. The method according to claim 3, wherein The time relationship matrix Y of each finger vein image is determined according to the hidden state corresponding to each first matrix LSTM ,include: According to the reverse order of the preset sequence, linear mapping is performed on the first matrix corresponding to each finger vein image in turn to obtain the query vector -Q of each first matrix x , key vector - K x Sum value vector - V x ; For each first matrix, calculate the first value corresponding to the first matrix according to the following formula: The second value corresponding to the first matrix is ​​calculated according to the following formula: -Cross x =At(-Q x ,-K h ,-V h ); The third value corresponding to the first matrix is ​​calculated according to the following formula: -Self h =At(-Q h ,-K h ,-V h ); The fourth value corresponding to the first matrix is ​​calculated according to the following formula: -Cross h =At(-Q h ,-K x ,-V x ); Among them, -Q h is the query vector obtained by linearly mapping the second target matrix, -K h is the key vector obtained by linearly mapping the second target matrix, -V h is the value vector obtained by linearly mapping the second target matrix, the second target matrix is ​​the hidden state corresponding to the previous first matrix of the first matrix when sorted in the reverse order of the preset order, when the second target matrix is ​​the first first matrix in the reverse order of the preset order, -Q h , -K h and -V h is obtained by performing a preset initialization on the first first matrix in the reverse order of the preset sequence; According to the following formula, calculate the forget gate corresponding to the first matrix: -f t =σ(linar(Concat(-self x ,-Cross h ))); According to the following formula, the input gate corresponding to the first matrix is ​​calculated: -i t =σ(linear(Concat(-self x ,-self h ))); The fifth value corresponding to the first matrix is ​​calculated according to the following formula: The storage unit corresponding to the first matrix is ​​calculated according to the following formula: Among them, -C t-1 is the storage unit corresponding to the second target matrix. When the second target matrix is ​​the first first matrix in the reverse order of the preset sequence, -C t-1 The storage unit is initialized; According to the following formula, the output gate corresponding to the first matrix is ​​calculated: -o t =σ(linear(Concat(-Cross x ,-self h ))); According to the following formula, calculate the hidden state corresponding to the first matrix in the reverse order; -h t =(-o t )*(tanh(-C t )); Determine the time relationship matrix Y of each finger vein image according to the hidden state corresponding to each first matrix in the positive order and the hidden state corresponding to each first matrix in the reverse order LSTM .

5. The method according to claim 4, wherein The time relationship matrix Y of each finger vein image is determined according to the hidden state corresponding to each first matrix in the positive order and the hidden state corresponding to each first matrix in the reverse order. LSTM ,include: According to the preset order and the reverse order of the preset order, the hidden states corresponding to the first matrix at the same position are added to obtain a plurality of addition matrices; The obtained multiple addition matrices are spliced ​​to obtain a spliced ​​matrix, and the obtained spliced ​​matrix is ​​used as the time relationship matrix Y LSTM .

6. A feature extraction device, characterized in that: The device comprises: The convolution processing unit is configured to, after acquiring multiple finger vein images of a finger taken at different angles, perform convolution processing on each finger vein image in a preset order to obtain a label sequence corresponding to each feature block included in each finger vein image, wherein the label sequence corresponding to each feature block is composed of a one-dimensional vector corresponding to the feature block; the number of feature blocks is N, W is the width of the finger vein image, H is the height of the finger vein image, M is the height of the convolution kernel, and P is the width of the convolution kernel; The splicing unit is used to perform matrix splicing on the tag sequences corresponding to the feature blocks included in each finger vein image according to the positions of the feature blocks included in the finger vein image in the finger vein image, so as to obtain a matrix representing the finger vein image. The first matrix of the image; The first determining unit is used to determine the local time relationship matrix Y of each finger vein image at the same position according to the first matrix corresponding to each finger vein image. local ; The second determining unit is used to determine the time relationship matrix Q of each finger vein image according to the first matrix corresponding to each finger vein image. LSTM ; A calculation unit, configured to obtain a first total time relationship matrix Q′ between the finger vein images according to the following formula; Y′=Concat(Y LSTM ,AND local ); Among them, Concat is the vector connection function; The mapping unit is configured to perform nonlinear mapping on the total time relationship matrix Y′ according to the following formula to obtain a second total time relationship matrix Y between the finger vein images: in, is the ReLU activation function, W1 and W2 are weight matrices; The output unit is used to input the second total time relationship matrix Y into the spatial attention module to obtain an output matrix containing time relationship dependency and spatial relationship dependency, and use the output matrix as the features of multiple finger vein images taken at different angles.

7. The device according to claim 6, characterized in that The first determining unit is used to determine the local time relationship matrix Y of each finger vein image at the same position according to the first matrix corresponding to each finger vein image. local When, including: For the first matrix corresponding to each finger vein image, the first matrix is ​​divided according to the L×L window to obtain S first sub-matrices of L×L size, 0 <L<P; The first sub-matrix on the same position window in each finger vein image is input into the local time attention module according to the preset order for linear mapping to obtain the three mapping matrices Q of the first sub-matrix s , K s and V s ; According to the following formula, the local time relationship matrix time of the window class corresponding to the window is obtained s ; Among them, SM(·) is used to calculate the similarity matrix Perform matrix multiplication on each row of d k is the normalization parameter, E s is the first preset increment, T is the transpose operation of the matrix; According to the position of each window, the local time relationship matrix of each window class is concatenated to obtain the local time relationship matrix Y local .

8. The device according to claim 6, wherein The second determining unit is used to determine the time relationship matrix Y of each finger vein image according to the first matrix corresponding to each finger vein image. lSTM When, including: According to the preset order, linear mapping is performed on the first matrix corresponding to each finger vein image in turn to obtain the query vector Q of each first matrix x , key vector K x Sum value vector V x ; For each first matrix, calculate the first value corresponding to the first matrix according to the following formula: Among them, Att is the attention calculation, Softmax is the normalized exponential function, d k is the normalization parameter, B x is the second preset increment, T is the transpose operation of the matrix; The second value corresponding to the first matrix is ​​calculated according to the following formula: Cross x =That(Q x ,K h ,V h ); The third value corresponding to the first matrix is ​​calculated according to the following formula: Self h =That(Q h ,K h ,V h ); The fourth value corresponding to the first matrix is ​​calculated according to the following formula: Cross h =That(Q h ,K x ,V x ); Among them, Q h is the query vector obtained by linearly mapping the first target matrix, K h is the key vector obtained by linearly mapping the first target matrix, V h is a value vector obtained by linearly mapping the first target matrix, wherein the first target matrix is ​​a hidden state corresponding to the first matrix above the first matrix when sorted in the preset order. When the first target matrix is ​​the first first matrix in the preset order, Q h , K h and V h is obtained by performing a preset initialization on the first first matrix in the preset order; According to the following formula, calculate the forget gate corresponding to the first matrix: f t =σ(linear(Concat(self x ,Cross h ))); Among them, σ is the sine function, linear is the linear mapping function, and Concat is the vector connection function; According to the following formula, the input gate corresponding to the first matrix is ​​calculated: i t =σ(linear(concat(self x ,self h ))); The fifth value corresponding to the first matrix is ​​calculated according to the following formula: The storage unit corresponding to the first matrix is ​​calculated according to the following formula: Among them, C t-1 is the storage unit corresponding to the first target matrix. When the first target matrix is ​​the first first matrix in the preset order, C t-1 is the initialized storage unit, ⊙ is the Hadamard product operator; According to the following formula, the output gate corresponding to the first matrix is ​​calculated: o t =σ(linear(Concat(Cross x ,self h ))); According to the following formula, calculate the hidden state corresponding to the first matrix in the positive order; h t =o t *fishy(C) t ); Among them, tanh is the hyperbolic tangent function; According to the hidden state corresponding to each first matrix in the positive order, the time relationship matrix Y of each finger vein image is determined LSTM .

9. The device according to claim 8, wherein The determining unit is used to determine the time relationship matrix Y of each finger vein image according to the hidden state corresponding to each first matrix LSTM When, including: According to the reverse order of the preset sequence, linear mapping is performed on the first matrix corresponding to each finger vein image in turn to obtain the query vector -Q of each first matrix x , key vector - K x Sum value vector - V x ; For each first matrix, calculate the first value corresponding to the first matrix according to the following formula: The second value corresponding to the first matrix is ​​calculated according to the following formula: -Cross x =At(-Q x ,-K h ,-V h ); The third value corresponding to the first matrix is ​​calculated according to the following formula: -Self h =At(-Q h ,-K h ,-V h ); The fourth value corresponding to the first matrix is ​​calculated according to the following formula: -Cross h =At(-Q h ,-K x ,-V x ); Among them, -Q h is the query vector obtained by linearly mapping the second target matrix, -K h To perform linear The key vector obtained by the mapping, -V h is the value vector obtained by linearly mapping the second target matrix, the second target matrix is ​​the hidden state corresponding to the previous first matrix of the first matrix when sorted in the reverse order of the preset order, when the second target matrix is ​​the first first matrix in the reverse order of the preset order, -Q h , -K h and -V h is obtained by performing a preset initialization on the first first matrix in the reverse order of the preset sequence; According to the following formula, calculate the forget gate corresponding to the first matrix: -f t =σ(linar(Concat(-self x ,-Cross h ))); According to the following formula, the input gate corresponding to the first matrix is ​​calculated: -i t =σ(linear(Concat(-self x ,-self h ))); The fifth value corresponding to the first matrix is ​​calculated according to the following formula: The storage unit corresponding to the first matrix is ​​calculated according to the following formula: Among them, -C t-1 is the storage unit corresponding to the second target matrix. When the second target matrix is ​​the first first matrix in the reverse order of the preset sequence, -C t-1 The storage unit is initialized; According to the following formula, the output gate corresponding to the first matrix is ​​calculated: -o t =σ(linear(Concat(-Cross x ,-self h ))); According to the following formula, calculate the hidden state corresponding to the first matrix in the reverse order; -h t =(-o t )*(tanh(-C t )); Determine the time relationship matrix Y of each finger vein image according to the hidden state corresponding to each first matrix in the positive order and the hidden state corresponding to each first matrix in the reverse order LSTM .

10. The device according to claim 9, wherein The second determining unit is used to determine the time relationship matrix Y of each finger vein image according to the hidden state corresponding to each first matrix in the positive order and the hidden state corresponding to each first matrix in the reverse order. LSTM When, including: According to the preset order and the reverse order of the preset order, the hidden states corresponding to the first matrix at the same position are added to obtain a plurality of addition matrices; The obtained multiple addition matrices are spliced ​​to obtain a spliced ​​matrix, and the obtained spliced ​​matrix is ​​used as the time relationship matrix Y LSTM .

Images