A graph neural network vein recognition method based on deep learning

Through the deep learning graph neural network method, combined with the venous encoder and the K nearest neighbor algorithm, the problem of low venous recognition rate in the prior art is solved, comprehensive recognition of venous recognition is achieved, and the recognition rate is improved.

CN117994815BActive Publication Date: 2024-11-26CHONGQING TECH & BUSINESS UNIV +2
View PDF 14 Cites 0 Cited by

Patent Information

Application Number
CN202410128871.5
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-01-29
Publication Date
2024-11-26
Estimated Expiration
2044-01-29

AI Technical Summary

Technical Problem

It is difficult for existing venous recognition algorithms to comprehensively learn the relationship between different samples during training, resulting in a low recognition rate.

Method used

Using a graph neural network method based on deep learning, image features are extracted through venous encoder, combined with K nearest neighbor algorithm to calculate Manhattan distance, determine the nearest neighbor image and establish a data set, use graph neural network for learning, and finally fuse the venous encoder and graph neural The network gets the predicted tag.

Benefits of technology

The venous recognition rate is improved, and more stable and effective characteristics are obtained through comprehensive learning of relationships between samples, and the accuracy of recognition is improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN117994815B_ABST
    Figure CN117994815B_ABST
Patent Text Reader

Abstract

The present invention provides a graph neural network vein recognition method based on deep learning, comprising: obtaining an image to be recognized and a training image, extracting features of the image to be recognized and the training image using a vein encoder, and obtaining recognition image features and training image features; calculating the Manhattan distance between the recognition image features and the training image features using the K nearest neighbor algorithm, and determining the K nearest neighbor images and corresponding labels closest to the image to be recognized according to the Manhattan distance, and establishing a data set; using a graph neural network to learn the feature information of the recognition image features and the K nearest neighbor images to obtain a recognition result; fusing the recognition image features obtained by the vein encoder and the recognition result obtained by the graph neural network to obtain a predicted label for vein recognition. The present invention can realize comprehensive classification and recognition of the relationship between different vein images, and improve the vein recognition rate.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of vein recognition technology, and in particular to a graph neural network vein recognition method based on deep learning. Background Art

[0002] Vein recognition is a biometric technology that identifies people based on the characteristic information of their palm veins. At present, vein recognition has attracted more and more attention due to its unique security and portability, and more and more vein recognition algorithms have been proposed.

[0003] However, during the training process, the vein recognition algorithm in the prior art can only learn the features of each sample individually for classification, and it is difficult to comprehensively learn the relationship between different samples for classification, resulting in a low recognition rate.

[0004] Therefore, there is an urgent need for a vein recognition method that can comprehensively learn and classify the relationships between different samples and improve the vein recognition rate. Summary of the invention

[0005] Based on this, it is necessary to provide a graph neural network vein recognition method based on deep learning to address the above technical problems.

[0006] A deep learning-based graph neural network vein recognition method comprises the following steps: obtaining an image to be recognized and a training image, extracting features of the image to be recognized and the training image by using a vein encoder, and obtaining recognition image features and training image features; calculating the Manhattan distance between the recognition image features and the training image features by using a K-nearest neighbor algorithm, and determining the M nearest neighbor images and corresponding labels closest to the image to be recognized according to the Manhattan distance, and establishing a data set; using a graph neural network to learn the feature information of the recognition image features and the M nearest neighbor images to obtain a recognition result, including: calculating an adjacency matrix A=[A i,j ] (M+1)×(M+1) , then the relationship between the two eigenvectors is:

[0007]

[0008] Where δ is the symmetric function of the neural network parameterization, is the learned weighted adjacency matrix;

[0009] δ(z i (t) , z j (t) )=MLP(abs(z i (t) -z j (t))),

[0010] Among them, δ(z i (t) , z j (t) )=MLP(abs(z i (t) -z j (t) )),δ(z i (t) , z i (t) )=0;

[0011] During calculation, the absolute difference between the two nodes is input into the multi-layer perceptron, the corresponding weight value is output, and normalized to obtain the adjacency matrix B. The next layer of the network is calculated using the graph neural network, and the recognition result is finally output. The formula is:

[0012]

[0013] In the formula, z p (t+1) represents the next layer of network, p represents the dimension of the feature vector, Gc represents the graph neural network, z (t) represents the feature expression of the tth layer; fusing the recognition image features obtained by the vein encoder and the recognition results obtained by the graph neural network to obtain the predicted label of vein recognition, including: combining the distribution of the vein encoder with the distribution of the graph neural network model to obtain the distribution of the target model, which is:

[0014] P=λP Encoder (y|x)+(1-λ)P GCN (y|x),

[0015] In the formula, λ represents the set parameter, P represents the target model, x represents the image to be recognized, and y represents the label of the image to be recognized; the recognition image features and the recognition results are fused through the target model to output the predicted label.

[0016] In one embodiment, the step of obtaining the image to be identified and the training image, extracting the features of the image to be identified and the training image using an encoder, and obtaining the features of the identification image and the training image includes: setting x1, x2, x3...x s and corresponding labels y1, y2, y3...y s is a sample in the data set, x0 and y0 represent the image to be identified and its label respectively, s is the number of samples, and the feature extraction model is represented by Φ w , W is the training weight, then the feature extraction formula in the image to be identified and the training image is:

[0017] zi =Φ w (x i ) i=1,2,3...s,

[0018] In the formula, z i represents the recognition image features or training image features, Φ w (·) represents the feature extractor, which is used to map the image from the high-dimensional feature space to the low-dimensional feature space.

[0019] In one embodiment, the vein encoder includes a convolution backbone, four Mix-Transformer modules and a classification layer; the convolution backbone is used to obtain an embedded feature map through an input image, including a 3×3 convolution layer with a stride of 2 and two 3×3 convolution layers with a stride of 1; the four Mix-Transformer modules are respectively set with corresponding downsampling rates, and the feature layer of the convolved input image is downsampled through stride convolution to obtain the features of the input image; wherein the Mix-Transformer module includes a CNN module, a channel attention module and a Transformer module; the classification layer is used to predict the probability of the input image entering each category.

[0020] In one embodiment, the CNN module includes an expansion layer, a deep convolution layer and a projection layer, and the formula is:

[0021] IConv(X)=Conv(F(Conv(X))),

[0022] F(X)=DWConv(X)+X,

[0023] In the channel attention module, for the input X∈R n×d After linear transformation, we get and Where n = H × W, d, d k ,d v are the dimensions of Q, K, and V respectively. The channel attention module uses the obtained channel attention to multiply the vector V in the attention branch to correct the channel weight. The formula is:

[0024]

[0025] In the formula, is a weight-corrected vector; the Transformer module includes two continuous convolutional layers and a global maximum pooling layer, the continuous convolutional layers include two convolutional layers of 3×3 and 1×1, and a normalization layer and an activation layer are provided between the convolutional layers, and the calculation formula of the channel attention is:

[0026] X=C3(X),

[0027] X=C1(MaxPool(X)+C1(C3(X))),

[0028] C1(X)=Relu(BN(Conv1(X))),

[0029] C3(X)=Relu(BN(Conv3(X))),

[0030] The global maximum pooling layer is used to concatenate feature vectors to obtain a comprehensive feature vector V; Q, K and the comprehensive feature vector V are combined and processed using a multi-head self-attention module to obtain the features of the input image.

[0031] In one embodiment, the method of calculating the Manhattan distance between the recognition image feature and the training image feature by the K nearest neighbor algorithm, and determining the M nearest neighbor images and corresponding labels of the image to be recognized according to the Manhattan distance, and establishing a data set includes: setting the recognition image feature and the training image feature to be [z0, z1, z2...z d ], where z m =[z m1 ,z m2 ,z m3 ...z md ] T and z u =[z u1 ,z u2 ,z u3 ,…z ud ] T Represent the feature vectors of the m-th image and the u-th image respectively; calculate the Manhattan distance between the feature vectors of the m-th image and the u-th image, the formula is:

[0032]

[0033] Wherein, L represents the Manhattan distance, and the smaller L is, the more similar it is to the vein image; the obtained Manhattan distance is used to determine the M nearest neighbor images and the corresponding labels of the image to be identified, and establish a data set.

[0034] Compared with the prior art, the advantages and beneficial effects of the present invention are as follows: by acquiring an image to be identified and a training image, a vein encoder is used to extract features of the image to be identified and the training image, and the features of the identified image and the training image are obtained, so that the relationship between samples can be mined, making the obtained features more stable and effective; the Manhattan distance between the features of the identified image and the features of the training image is calculated by the K nearest neighbor algorithm, and the M nearest neighbor images of the image to be identified and the corresponding labels are determined according to the Manhattan distance to establish a data set, so as to comprehensively learn the relevant information between different samples; a graph neural network is used to learn the features of the identified image and the feature information of the M nearest neighbor images to obtain a recognition result; the features of the identified image obtained by the vein encoder and the recognition result obtained by the graph neural network are fused to obtain a predicted label for vein recognition, thereby realizing comprehensive recognition of veins and improving the vein recognition rate. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] Figure 1 A schematic diagram of a process of a graph neural network vein recognition method based on deep learning in one embodiment;

[0036] Figure 2 A flowchart of a graph neural network vein recognition method based on deep learning in one embodiment;

[0037] Figure 3 Schematic diagram of the structure of a vein encoder in one embodiment. DETAILED DESCRIPTION

[0038] Before describing the specific embodiments of the present invention, the overall concept of the present invention is described as follows:

[0039] The present invention is mainly developed based on the vein recognition process. Currently, the vein recognition algorithm can only identify and classify sample features individually, resulting in a low recognition rate.

[0040] Therefore, the present invention proposes a graph neural network vein recognition method based on deep learning. By acquiring an image to be recognized and a training image, a vein encoder is used to extract the features of the image to be recognized and the training image, and the recognition image features and the training image features are obtained, so that the relationship between samples can be mined, so that the obtained features are more stable and effective; the Manhattan distance between the recognition image features and the training image features is calculated by the K nearest neighbor algorithm, and the M nearest neighbor images of the image to be recognized and the corresponding labels are determined according to the Manhattan distance to establish a data set, so as to comprehensively learn the relevant information between different samples; the graph neural network is used to learn the recognition image features and the feature information of the M nearest neighbor images to obtain the recognition result; the recognition image features obtained by the vein encoder and the recognition result obtained by the graph neural network are fused to obtain the predicted label of vein recognition, thereby realizing comprehensive recognition of veins and improving the vein recognition rate.

[0041] After introducing the overall concept of the present invention, in order to make the purpose, technical solution and advantages of the present invention more clear, the present invention is further described in detail by specific implementation methods in combination with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0042] In one embodiment, Figure 1-2 As shown, a deep learning-based graph neural network vein recognition method is provided, comprising the following steps:

[0043] Step S101, obtaining an image to be recognized and a training image, using a vein encoder to extract features of the image to be recognized and the training image, and obtaining recognition image features and training image features.

[0044] Specifically, traditional encoders usually focus more on the extraction of local and global features, but tend to ignore the problem of weak modeling ability in the channel dimension.

[0045] Since the dimension of the original vein image is high, it is not conducive to the subsequent KNN (K-NearestNeighbor, K nearest neighbor algorithm) clustering. Therefore, after obtaining the image to be identified and the training image, the vein encoder is used to extract the robust features of the image to be identified and the training image to obtain the recognition image features and the training image features.

[0046] Wherein, step S101 includes: let x1, x2, x3...x s and corresponding labels y1,y2,y3...y s is a sample in the data set, x0 and y0 represent the vein images to be classified and their labels respectively, s is the number of samples, and the feature extraction model is represented by Φ w , W is the training weight, then the feature extraction formula in the image to be identified and the training image is:

[0047] z i =Φ w (x i ) i=1,2,3...s,

[0048] In the formula, Φ w (·) represents a feature extractor, which is used to map the image from a high-dimensional feature space to a low-dimensional feature space to obtain recognition image features and training image features.

[0049] Specifically, the feature extractor can map the vein image from a high-dimensional feature space to a low-dimensional feature space, so that the mapped feature vector has good separability; the robust features of the image to be identified and the training image are extracted through the independently designed encoder to facilitate the subsequent K-nearest neighbor clustering.

[0050] Among them, the vein encoder includes a convolution backbone, four Mix-Transformer modules and a classification layer; the convolution backbone is used to obtain an embedded feature map through an input image, including a 3×3 convolution layer with a stride of 2 and two 3×3 convolution layers with a stride of 1; the four Mix-Transformer modules are respectively set with corresponding downsampling rates, and the feature layer of the convolved input image is downsampled through stride convolution to obtain the features of the input image; among them, the Mix-Transformer module includes a CNN module, a channel attention module and a Transformer module; the classification layer is used to predict the probability of the input image entering each category.

[0051] Specifically, the overall structure of the vein encoder is as follows Figure 3 As shown in (a), it consists of a convolutional layer, four Mix-Transformer modules and a classification layer. The image is input to the convolution backbone to obtain an embedded feature map; the embedded feature map is further forwarded to the Mix-Transformer module to learn vein feature representation; finally, the classification layer is used to predict the probability of the input image entering each category.

[0052] like Figure 3 As shown in (b), the input image enters the convolution backbone, which reduces the size of the input image by changing the number of output channels through a 3×3 convolution with a stride of 2, and obtains better local information through another two 3×3 convolutions with a stride of 1 to obtain an embedded feature map.

[0053] The embedded feature map passes through four Mix-Transformer modules with corresponding downsampling rates in turn, and the feature layer of the convolved input image is downsampled by stride convolution to obtain the features of the input image.

[0054] like Figure 3 As shown in (c) in the figure, the Mix-Transformer module has three branches, namely the CNN module, the channel attention module and the Transformer module. The window-based channel attention module is combined with the convolutional neural network module, and the acquired features are combined with the channel dimension, so that more attention can be paid to the extraction of local and global features, and the modeling ability in the channel dimension is enhanced. The complementary information between the three modalities is used to make the fusion feature more stable and effective.

[0055] Finally, the classification layer predicts the probability of the input image falling into each category based on the features of the input image.

[0056] Among them, the CNN module includes an expansion layer, a deep convolution layer, and a projection layer, and the formula is:

[0057] IConv(X)=Conv(F(Conv(X))),

[0058] F(X)=DWConv(X)+X,

[0059] In the channel attention module, for the input X∈R n×d After linear transformation, we get and Where n = H × W, d, d k ,d v are the dimensions of Q, K, and V respectively. The channel attention module uses the obtained channel attention to multiply the vector V in the attention branch to correct the channel weight. The formula is:

[0060]

[0061] In the formula, is the weight-corrected vector; the Transformer module includes two continuous convolutional layers and a global maximum pooling layer. The continuous convolutional layers include 3×3 and 1×1 convolutional layers, and normalization layers and activation layers are set between the convolutional layers. The calculation formula of channel attention is:

[0062] X=C3(X),

[0063] X=C1(MaxPool(X)+C1(C3(X))),

[0064] C1(X)=Relu(BN(Conv1(X))),

[0065] C3(X)=Relu(BN(Conv3(X))),

[0066] The global maximum pooling layer is used to concatenate feature vectors to obtain the integrated feature vector V; Q, K and the integrated feature vector V are combined and processed using a multi-head self-attention module to obtain the features of the input image.

[0067] Specifically, convolution is one of the most important parts of modern convolutional neural networks. The specific process of the convolution algorithm: First, set the convolution kernel Where k is the size of the convolution kernel, C in and C out are the input and output channel dimensions respectively; the feature vectors of the input and output are given H and W represent the height and width of the input. and As the feature vectors of F and G corresponding to pixels (i, j), the standard convolution can be set as:

[0068]

[0069] in, p,q∈{0,1,2...k-1} represents the kernel weight relative to the kernel position (p,q);

[0070] For the input feature vector, the dimension is expanded by convolution. The first layer expands the dimension by 4 times, and the second layer reduces the dimension by the same ratio.

[0071] IConv(X)=GELU(XW1+b1)W2+b2,

[0072] Where W1∈R d×4d ,W2∈R 4d×d They represent the weights of the two linear layers respectively, and b1 and b2 are bias terms.

[0073] The CNN module in this embodiment is similar to the inverse residual block, and is composed of an expansion layer, a deep convolution layer, and a projection layer, as follows:

[0074] IConv(X)=Conv(F(Conv(X))),

[0075] F(X)=DWConv(X)+X,

[0076] Deep convolution is used to enhance the extraction of local features, and the residual structure can promote the propagation ability of gradients.

[0077] The traditional method uses convolution and self-attention in sequence to increase the receptive field, but this method will result in little interweaving of the relationship between the two windows and between windows. Therefore, this embodiment uses deep convolution and self-attention in series to increase the receptive field.

[0078] In order to further improve the modeling ability of the channel dimension, the obtained channel attention is multiplied with the vector V in the attention branch to correct its channel weight, which solves the problem of weak modeling ability in the channel dimension and can achieve better feature learning. The channel attention module can use the importance of the channel for learning, thereby enhancing the texture features of the vein and improving the expression ability of the network.

[0079] like Figure 3 As shown in the figure, the Transformer module includes two continuous convolutional layers and a global maximum pooling layer. The continuous convolutional layers include two convolutional layers, 3×3 and 1×1, and normalization layers and activation layers are set between the convolutional layers.

[0080] By calculating the channel attention of the vector V in the attention branch, the input features of size H×W×C are integrated into a feature expression of 1×1×C using global pooling, which can be used to represent the global receptive field while ensuring that the output dimension matches the input dimension, and then the input image X∈R H×W×C After transformation, we get X∈R 1×1×C The obtained 1×1×C features are used as weights and multiplied with the vector V in the attention branch to update the vector V, thereby achieving better feature learning.

[0081] Step S102, calculating the Manhattan distance between the recognition image features and the training image features through the K nearest neighbor algorithm, and determining the M nearest neighbor images and corresponding labels to the image to be recognized according to the Manhattan distance, and establishing a data set.

[0082] Specifically, since the amount of computation required to predict labels using the relationship between the image to be identified and all samples in the data set is large, and the coefficients of the relationship matrix obtained will result in low label prediction accuracy. Therefore, the K nearest neighbor algorithm can be used to obtain M neighbor images of the image to be identified, and a data set is established based on the M nearest neighbor images closest to the image to be identified and the corresponding labels for subsequent label prediction. The K nearest neighbor algorithm is simple and easy to implement, and can comprehensively learn the relationship between different samples. Through the M nearest neighbor images of the image to be classified, the graph iteration algorithm is used to comprehensively learn the relevant information between different samples.

[0083] Wherein, step S102 includes: setting the image features to be identified and the training image features to be [z0, z1, z2...z d ], d represents the number of features, among which z m =[z m1 ,z m2 ,z m3 ...z md ] T and z u =[z u1 ,z u2 ,z u3 ,…z ud ] T Respectively represent the feature vectors of the m-th image and the u-th image; calculate the Manhattan distance between the feature vectors of the m-th image and the u-th image, the formula is:

[0084]

[0085] Where L represents the Manhattan distance. The smaller L is, the more similar it is to the vein image. mi and z uiRepresent the i-th feature vector of the m-th image and the u-th image respectively; use the obtained Manhattan distance to determine the M nearest neighbor images and corresponding labels to the image to be identified, and establish a data set.

[0086] Specifically, the KNN algorithm can obtain M neighboring images of the image to be classified, and the graph iteration algorithm can comprehensively learn the relevant information between different samples. Compared with the traditional model that can only learn the features of each sample separately, the KNN algorithm is simple and easy to implement, and can comprehensively learn the relationship between different samples.

[0087] Step S103, using a graph neural network to learn the features of the recognition image and the feature information of M neighboring images to obtain a recognition result.

[0088] Specifically, the feature vector of the image to be recognized obtained through the encoder is input into the graph neural network model. The initial vertices of the graph are K+1 (1 recognition image feature + M neighboring image features) feature vectors composed of the recognition image features and M neighboring images. The recognition result is obtained through graph neural network learning.

[0089] Wherein, step S103 comprises: according to the recognition image features and M neighboring images, calculate the adjacency matrix A=[A i,j ] (M+1)×(M+1) , then the two eigenvectors z i and z j The relationship between them is:

[0090]

[0091] Where δ is the symmetric function of the neural network parameterization, is the learned weighted adjacency matrix;

[0092] δ(z i (t) , z j (t) )=MLP(abs(z i (t) -z j (t) )),

[0093] Among them, δ(z i (t) , z j (t) )=MLP(abs(z i (t) -z j (t) )),δ(z i (t) , z i(t) )=0; during calculation, the absolute difference between the two nodes is input to the multilayer perceptron, the corresponding weight value is output, and normalized to obtain the adjacency matrix B, and the graph neural network is used to calculate the next layer of the network, and the final output is the recognition result. The formula is:

[0094]

[0095] In the formula, z p (t+1) represents the next layer of network, P represents the dimension of the feature vector, Gc represents the graph neural network, z (t) Represents the feature expression of the tth layer.

[0096] Specifically, when calculating the adjacency matrix, consider the multilayer perceptron stacked after the absolute difference between two vector nodes, input the absolute difference of the two nodes into the multilayer perceptron, output the corresponding termination, and perform a normalized exponential function on each row to ensure that the sum of the weights between each node and all other nodes is 1, and obtain the adjacency matrix B. Finally, the graph neural network is used to calculate the next layer of the network.

[0097] The graph convolution layer in this embodiment does not change the structure of the graph convolution model, but only changes the value of the feature vector on the node. The calculation of the adjacency matrix will update the connection weight between the two nodes. Therefore, the structure of the graph convolution model changes.

[0098] Step S104, fusing the recognition image features obtained by the vein encoder and the recognition results obtained by the graph neural network to obtain a predicted label for vein recognition.

[0099] Specifically, the recognition image features obtained by the vein encoder are fused with the recognition results obtained by the graph neural network to obtain the predicted labels for vein recognition. The vein recognition rate is improved through comprehensive learning and classification of the relationships between different samples.

[0100] Among them, step S104 includes: combining the distribution of the vein encoder with the distribution of the graph neural network model to obtain the distribution of the target model, which is:

[0101] P=λP Encoder (y|x)+(1-λ)P GCN (y|x),

[0102] In the formula, λ represents the parameter, P represents the target model, x represents the image to be identified, and y represents the label of the image to be identified. The image features and the recognition results are fused by the target model to output the predicted label.

[0103] In this embodiment, by obtaining an image to be identified and a training image, a vein encoder is used to extract features of the image to be identified and the training image, and the features of the identified image and the training image are obtained, so that the relationship between samples can be mined, so that the obtained features are more stable and effective; the Manhattan distance between the features of the identified image and the features of the training image is calculated by the K nearest neighbor algorithm, and the M nearest neighbor images of the image to be identified and the corresponding labels are determined according to the Manhattan distance to establish a data set to comprehensively learn the relevant information between different samples; a graph neural network is used to learn the features of the identified image and the feature information of the M nearest neighbor images to obtain a recognition result; the features of the identified image obtained by the vein encoder and the recognition result obtained by the graph neural network are fused to obtain a predicted label for vein recognition, thereby realizing comprehensive recognition of veins and improving the vein recognition rate.

[0104] A person skilled in the art can understand that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program, and the program can be stored in a computer-readable storage medium, and when the program is executed, it can include the processes of the embodiments of the above-mentioned methods. The storage medium can be a disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM), etc.

[0105] Obviously, those skilled in the art should understand that the modules or steps of the present invention described above can be implemented by a general-purpose computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, and optionally, they can be implemented by a program code executable by a computing device, so that they can be stored in a computer storage medium (ROM / RAM, magnetic disk, optical disk) and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than that here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Therefore, the present invention is not limited to any specific combination of hardware and software.

[0106] The above contents are further detailed descriptions of the present invention in combination with specific implementation methods, and it cannot be determined that the specific implementation of the present invention is limited to these descriptions. For ordinary technicians in the technical field to which the present invention belongs, several simple deductions or substitutions can be made without departing from the concept of the present invention, which should be regarded as falling within the scope of protection of the present invention.

Claims

1. A graph neural network vein recognition method based on deep learning, It is characterized in that The following steps are involved: Acquire an image to be recognized and a training image, and use a vein encoder to extract features of the image to be recognized and the training image to obtain recognition image features and training image features; The Manhattan distance between the recognition image feature and the training image feature is calculated by the K nearest neighbor algorithm, and the M nearest neighbor images and corresponding labels to the image to be recognized are determined according to the Manhattan distance to establish a data set; Using a graph neural network to learn the features of the recognition image and the feature information of M neighboring images to obtain a recognition result, including: calculating an adjacency matrix A=[A i,j ] (M+1)×(M+1) , then the relationship between the two eigenvectors is: Where δ is the symmetric function of the neural network parameterization, is the learned weighted adjacency matrix; δ(z i (t) ,With j (t) )=MLP(abs(z i (t) -With j (t) )), Among them, δ(z i (t) , z i (t) )=0; During calculation, the absolute difference between the two nodes is input into the multi-layer perceptron, the corresponding weight value is output, and normalized to obtain the adjacency matrix B. The next layer of the network is calculated using the graph neural network, and the recognition result is finally output. The formula is: In the formula, z p (t+1) represents the next layer of network, p represents the dimension of the feature vector, Gc represents the graph neural network, z (t) represents the feature expression of the tth layer; The recognition image features obtained by the vein encoder and the recognition results obtained by the graph neural network are integrated to obtain the prediction label of vein recognition, including: combining the distribution of the vein encoder with the distribution of the graph neural network model to obtain the distribution of the target model, which is: P=λP Encoder (y|x)+(1-λ)P GCN (y|x), In the formula, λ represents the setting parameter, P represents the target model, x represents the image to be identified, and y represents the label of the image to be identified; The identified image features and the identification results are fused through the target model to output a predicted label.

2. According to the deep learning-based graph neural network vein recognition method of claim 1, It is characterized in that The step of obtaining the image to be identified and the training image, and extracting features of the image to be identified and the training image using an encoder to obtain features of the identified image and the training image, comprises: Let x 1 ,x 2 ,x 3 ...x s and the corresponding label y 1 ,y 2 ,y 3 ...y s is a sample in the dataset, x 0 and 0 Represent the image to be identified and its label respectively, s is the number of samples, and the feature extraction model is represented by Φ w , W is the training weight, then the feature extraction formula in the image to be identified and the training image is: z i =Φ w (x i ) i=1,2,3...s, In the formula, z i represents the recognition image features or training image features, Φ w (·) represents the feature extractor, which is used to map the image from the high-dimensional feature space to the low-dimensional feature space.

3. According to the deep learning-based graph neural network vein recognition method of claim 2, It is characterized in that The vein encoder includes a convolutional backbone, four Mix-Transformer modules and a classification layer; The convolution backbone is used to obtain an embedded feature map through an input image, and includes a 3×3 convolution layer with a stride of 2 and two 3×3 convolution layers with a stride of 1; The four Mix-Transformer modules are respectively provided with corresponding downsampling rates, and the feature layer of the convolved input image is downsampled by stride convolution to obtain the features of the input image; wherein the Mix-Transformer module includes a CNN module, a channel attention module and a Transformer module; The classification layer is used to predict the probability of the input image falling into each category.

4. According to claim 3, the graph neural network vein recognition method based on deep learning, It is characterized in that The CNN module includes an expansion layer, a depth convolution layer and a projection layer, and the formula is: IConv(X)=Conv(F(Conv(X))), F(X)=DWConv(X)+X, In the channel attention module, for the input X∈R n×d After linear transformation, we get and Where n = H × W, d, d k ,d v are the dimensions of Q, K, and V respectively. The channel attention module uses the obtained channel attention to multiply the vector V in the attention branch to correct the channel weight. The formula is: In the formula, is the weight-corrected vector; The Transformer module includes two continuous convolutional layers and a global maximum pooling layer. The continuous convolutional layers include two convolutional layers of 3×3 and 1×1, and a normalization layer and an activation layer are provided between the convolutional layers. The calculation formula of the channel attention is: X=C3(X), X=C1(MaxPool(X)+C1(C3(X))), C1(X)=Relu(BN(Conv1(X))), C3(X)=Relu(BN(Conv3(X))), The global maximum pooling layer is used to concatenate feature vectors to obtain a comprehensive feature vector V; Combining Q, K and the comprehensive feature vector V, a multi-head self-attention module is used for processing to obtain the features of the input image.

5. According to claim 4, the graph neural network vein recognition method based on deep learning, It is characterized in that The method of calculating the Manhattan distance between the recognition image features and the training image features by the K nearest neighbor algorithm, and determining the M nearest neighbor images and corresponding labels to the image to be recognized according to the Manhattan distance, and establishing a data set includes: Let the recognition image features and training image features be [z 0 ,z 1 ,z 2 ...z d ], where z m =[z m1 ,z m2 ,z m3 ...z md ] T and z u =[z u1 ,z u2 ,z u3 ,…z ud ] T Represent the feature vectors of the m-th image and the u-th image respectively; Calculate the Manhattan distance between the feature vectors of the mth image and the uth image. The formula is: Where L represents the Manhattan distance. The smaller L is, the more similar it is to the vein image. The obtained Manhattan distance is used to determine the M nearest neighbor images and corresponding labels to the image to be identified, and a data set is established.

Citation Information

Patent Citations

  • Finger vein identification method and system based on convolutional neural network

    CN106529468A

  • Palm vein feature extraction method based on a graph network

    CN109740407A

  • Multi-modal identity authentication method based on vein similar image knowledge migration network

    CN112241680A

  • Image classification identification method and device based on adaptive dynamic convolutional network, and computer equipment

    CN114445664A

  • Intelligent target identification method based on multi-modal characteristic fusion

    CN115565050A