Cross-terminal finger vein recognition technology based on graph neural network

Abstract

The invention provides a cross-terminal finger vein recognition technology based on a graph neural network. Firstly, an image is segmented into a plurality of blocks, the blocks are regarded as nodes, and a graph is constructed by connecting nearest neighbors. And meanwhile, through a multi-head self-attention mechanism and a pyramid Mama module, the image processing and feature extraction process is optimized. The preliminary features are extracted through an improved Stem module, and the perception of a model on spatial information is enhanced by combining position embedding. A multi-head self-attention mechanism highlights related portions of the vein and suppresses unrelated context information. Through a dense spatial pyramid pool (DSPP), the model can capture detail information of different scales, pyramid fusion Mama (PFM) is responsible for effective information flow and fusion among multi-scale features, and the feature integration process is optimized. The method not only improves the accuracy and robustness of finger vein recognition, but also is suitable for seamless recognition conversion among different devices, and is expected to be widely applied to the fields of finance, security and the like.

Description

Technical Field

[0001] The present invention belongs to the field of biometric identification technology, and in particular relates to a cross-terminal finger vein recognition method implemented using a graph neural network. Background Art

[0002] Finger vein recognition technology is one of the advanced methods for identity authentication based on individual biometrics. Its principle is to capture the vein pattern inside the finger using optical or infrared imaging technology. These vein patterns are highly unique to each individual. Because they are located inside the human body, they are not easily affected by the external environment and are difficult to forge or copy. Compared with other biometric recognition technologies such as fingerprints or irises, finger vein recognition provides higher privacy protection while ensuring security because vein images are not easily observed from the outside. Therefore, this technology has been widely used in application scenarios that require a high level of security, such as financial transactions, high-security access control systems, and national security. Although the application of finger vein recognition technology on a single device is relatively mature, it faces major challenges in multi-device or cross-platform environments. The main problem is that there may be significant differences in the optical imaging system and image acquisition environment between different devices. These differences will cause changes in the vein images obtained by the same user on different devices, thereby affecting the accuracy and consistency of the recognition algorithm. In addition, with the development of technology and the upgrading of equipment, ensuring recognition compatibility between new and old devices is also a technical challenge. Therefore, developing a finger vein recognition technology that can adapt to the differences between different devices and provide stable recognition performance is the key to achieving its application in a wider range of scenarios.

[0003] In recent years, with the development of deep learning technology, graph neural networks, as an emerging variant of graph neural networks, have begun to be studied for solving structured problems in related tasks. Unlike standard graph neural networks, Vision GNN is designed specifically for visual data and can process graph structures with image blocks as nodes, and use the spatial relationship between nodes to enhance the model's ability to parse complex image data. The introduction of this technology provides new possibilities for solving the problem of cross-terminal consistency in finger vein recognition.

[0004] This paper introduces a Vision GNN-based method to process finger vein images, optimizes image preprocessing, feature extraction, and cross-device recognition processes, enhances the generalization ability of the model across different devices, improves the accuracy and robustness of recognition, and makes it more suitable for multi-device environments. Summary of the invention

[0005] The present invention proposes a cross-terminal finger vein recognition method based on graph neural network. By introducing a visual graph neural network, efficient feature extraction and matching are achieved, which solves the problems of low cross-terminal recognition accuracy and poor robustness in the prior art and has broad application prospects.

[0006] Finger vein images were collected using three different devices, and the regions of interest (ROI) were extracted from the collected images. This step involves using image processing techniques to identify and crop the image portion containing vein information. Then normalization was performed to ensure that the input data had a uniform scale and contrast, suitable for subsequent graph neural network processing.

[0007] The data collected by each device is randomly divided into training sets and test sets, ensuring that in each experiment, data from two devices is used for training and data from another device is used for testing. This cross-validation method is used for a total of six sets of experiments to comprehensively evaluate the cross-device performance and generalization ability of the model.

[0008] The entire image is divided into multiple patches, each of which is considered a node. Based on these image patches (i.e. nodes), a graph is constructed by connecting their nearest neighbors. Each node in the graph represents a part of the image, and the edges between nodes represent the proximity relationship between these parts. These graph structures make it easy to capture and utilize spatial relationships in finger vein images.

[0009] Preliminary image features are extracted through the improved Stem module, and the model's perception of spatial information is enhanced through position embedding technology.

[0010] The graph is processed using a graph convolution layer to update the features of each node by aggregating the information of neighboring nodes. After graph convolution, a multi-head self-attention mechanism is used to highlight features related to vein recognition and suppress irrelevant background information.

[0011] Dense spatial pyramid pooling (DSPP) is used to capture detailed information at different scales.

[0012] The Pyramid Fusion Mamba Module (PFM) is used to achieve effective information flow and fusion between features of different scales and optimize the feature integration process.

[0013] Linear layers are applied before and after graph convolution to project node features into the same domain and increase feature diversity.

[0014] A feed-forward network is applied on each node to further transform the node features to enhance the expressiveness of the model and avoid over-smoothing.

[0015] Finally, after passing through two fully connected layers, the final prediction result is obtained by softmax classification.

[0016] A cross-terminal finger vein recognition method based on graph neural network in the present invention has the following advantages: Cross-device compatibility: Through training and testing on multiple devices, the present invention significantly improves the cross-device compatibility and adaptability of the model, making it suitable for a variety of different acquisition environments.

[0017] Improve recognition accuracy: Utilizing the powerful image analysis capabilities of the ViG model, combined with multi-head attention and multi-scale feature fusion technology, the recognition accuracy of finger veins is significantly improved.

[0018] Enhanced model robustness: Through dense feature fusion and spatial pyramid technology, the model can effectively capture multi-scale features from micro to macro, enhancing the robustness of the recognition process. BRIEF DESCRIPTION OF THE DRAWINGS Figure 1 This is a flowchart of a cross-terminal finger vein recognition technology based on graph neural network. Figure 2 It is a specific module diagram. Figure 3 Schematic diagram of finger vein ROI images on three different devices. DETAILED DESCRIPTION

[0019] The method will be described in detail below with reference to the accompanying drawings. It should be noted that, in general, the method can be applied to finger vein recognition with different image qualities and different application scenarios.

[0020] This application provides a cross-device finger vein recognition method based on graph neural network (ViG), which converts complex finger vein images into efficiently processed graph data through advanced graph structure representation. This method uses multi-head self-attention mechanism and pyramid fusion Mamba (PFM) module to adaptively extract and fuse multi-scale vein features, thereby significantly improving the accuracy of finger vein recognition and the generalization ability of the system.

[0021] Figure 1 A flowchart of a cross-device finger vein recognition method based on a graph neural network of the present application is shown.

[0022] This application proposes a cross-device finger vein recognition method based on graph neural network. The specific steps are as follows: Finger vein images were collected using three different devices, and the regions of interest (ROI) were extracted from the collected images. Figure 2 Schematic diagram of finger vein ROI images on three different devices.

[0023] In the preprocessing stage, the region of interest (ROI) image of the finger vein is first normalized to adjust the original pixel value range [0, 255] to the interval [0, 1]. The specific operation is to subtract 0.5 from the pixel value of each channel and then divide it by 0.5.

[0024] In this way, the mean of the ROI image is (0.5, 0.5, 0.5) and the standard deviation (std) is (0.5, 0.5, 0.5), ensuring the consistency and standardization of the data.

[0025] Next, in order to adapt to the input requirements of the neural network model, the normalized image size is uniformly adjusted to (3, 224, 224), that is, a 3-channel 224x224 pixel image.

[0026] Before training begins, first configure the data preprocessing and enhancement parameters according to the specific requirements of the training task. This includes setting the input size of the image, whether to enable data enhancement (such as random erasing, mixing, etc.), the image cropping percentage, and color jitter parameters.

[0027] The finger vein image of size H×W×3 is subdivided into N blocks. Each block is converted into a feature vector x_i∈R^D, forming a set X=[x_1,x_2,…,x_N ], where D represents the feature dimension and i ranges from 1 to N. These feature vectors are regarded as a set of unordered nodes, represented as V={v_1,v_2,…,v_N}. For each node v_i, its K nearest neighbors N(v_i ) are determined, and a directed edge e_ji is established for each v_j from v_j∈N(v_i ) to v_i. Through this process, a graph G= (V,E) is constructed, where E contains all such edges. The graph construction process is called G=G(X).

[0028] Graph-level processing. Generally starts with feature x∈R^(N×D). For the feature, a graph is constructed: G=G(X). The graph convolution layer can exchange information between nodes by aggregating the features of neighboring nodes. Specifically, the operation of graph convolution is as follows: G^'=F(G,W)= Update(Aggregate(G,W_agg),W_update), Where W_agg and W_update are the learnable weights for the aggregation and update operations, respectively. The aggregation operation computes the representation of a node by aggregating the features of neighboring nodes, while the update operation further merges these aggregated features.

[0029] in yes The neighboring node set of , using the maximum relative graph convolution method. Graph-level processing can be expressed as: A multi-head attention mechanism is used after the graph convolution. A mask matrix is ​​created based on the important areas of the vein image. ,in When it is 1, it indicates a node For Node It is important in vein feature recognition; when it is 0, it is unimportant and should be ignored.

[0030] Calculate the attention score for each pair of nodes i and j in the graph: e_ij=((W_q x_i )^T (W_k x_j )) / √(d_k ) Where W_q and W_k are the linear transformation matrices of the query and key, and d_k is the dimension of the key vector, which is used to scale the scores to avoid excessively large gradient values.

[0031] Applying the mask to the attention scores, the scores corresponding to nodes not marked as important are set to very small negative numbers (e.g. -infinity), which are close to zero in the softmax function:

[0032] Normalize the scores using softmaxl and calculate the final attention weights: Update the features of each node using the attention weight: in is the linear transformation matrix of the values. Using different parameter sets ( Perform the above steps multiple times (for example, ), each header captures different information: Concatenate the outputs of all heads and integrate them using another linear transformation: The extracted feature map is used to capture the details of the image at different scales using dense spatial pyramid pooling (DSPP). Multiple pooling operations are used to process the feature map, each with a different kernel size and step size to capture different spatial information: in and They are The pooling kernel size and stride of each scale.

[0033] Each pooled feature map Pi is upsampled to match the size of the original feature map: U_i=Upsample(P_i,H,W) All upsampled feature maps are connected with the original feature map X in the depth direction: S="Concatenate" (X,U_1,U_2,…,U_K ) The Mamba module is used to further process the multi-scale feature map S, using its selective filtering mechanism to reduce semantic redundancy: Y=Mamba(S) The Mamba module can effectively compress homogeneous features and extract core semantic information through a selective scanning mechanism.

[0034] Finally, the nonlinear expression capability is enhanced through the activation function (such as ReLU), and the final feature map is output: Z = ReLU(Y) Z is the feature map after DSPP and Mamba processing.

[0035] Linear layers are applied before and after graph convolution to project node features into the same domain Construct a Grapher module, given the input feature x∈R^(N×D), and project the node features into the same domain. The Grapher module can be expressed as Y=σ("GraphConv"(XW_in))W_out+X Where Y∈R^(N×D), W_in " and W_out " " are the weights of the fully connected layer, σ is the activation function, such as ReLU and GeLU, and the bias term " is omitted.

[0036] In order to further enhance the feature conversion capability and alleviate the over-smoothing phenomenon, a feed-forward network (FFN) is used at each node. The FFN module is a simple multi-layer perceptron consisting of two fully connected layers: Z = σ(YW_1 ) W_2+Y where Z∈R^(N×D), W_1 and W_2 are the weights of the fully connected layer, and the bias term is omitted. The hidden dimension of FFN is usually larger than D.

[0037] In the Grapher and FFN modules, batch normalization is applied after each fully connected layer or graph convolutional layer. The stack of Grapher modules and FFN modules constitutes the ViG block.

[0038] After multiple layers of feature extraction and fusion, the feature map maps the extracted high-dimensional features to category predictions through a fully connected layer.

[0039] The final fully connected layer outputs nodes that match the number of categories. The output of each node is activated by softmax and represents the model's predicted probability for each category.

Claims

1. A cross-device finger vein recognition method based on graph neural network, the method comprising the following steps: Finger vein images were collected using three different devices, and regions of interest (ROI) were extracted from the collected images; Normalize the extracted ROI image and adjust the image size to a uniform (3, 224, 224) pixels; The data collected by each device is randomly assigned to training sets and test sets, and the cross-device performance and generalization ability of the model are evaluated through cross-validation; The normalized image is divided into multiple small blocks, each of which is regarded as a node, and a graph structure is constructed by connecting the nearest neighbor nodes; The improved Stem module is used to extract preliminary image features, and the position embedding technology is used to enhance the model's ability to perceive spatial information. Use graph convolutional layers to process the graph and update the features of each node by aggregating information from neighboring nodes; A multi-head self-attention mechanism is applied to highlight features related to vein recognition and suppress irrelevant background information; Capture details at different scales through dense spatial pyramid pooling (DSPP); Use the pyramid fusion Mamba module (PFM) to achieve effective information flow and fusion between features of different scales; Apply linear layers before and after graph convolution to project node features into the same domain and increase feature diversity; Apply a feed-forward network on each node to further transform node features and enhance the expressiveness of the model; After passing through two fully connected layers, the final prediction result is obtained by softmax classification.

2. The cross-device finger vein recognition method based on graph neural network according to claim 1 is characterized in that: The optical imaging systems and image acquisition environments of the multiple devices for acquiring finger vein images are different.

3. The cross-device finger vein recognition method based on graph neural network according to claim 1 is characterized in that: The preprocessing step also includes adjusting the brightness and contrast of the image to improve the clarity of the vein features.

4. The method according to claim 1, wherein the multi-head self-attention mechanism includes creating a mask matrix based on important areas of the vein image, through which irrelevant background information is suppressed and vein features are emphasized.

5. The method according to claim 1, characterized in that: The operation of the dense spatial pyramid pooling (DSPP) is as follows: Use multiple pooling operations to process the feature map, each with a different kernel size and stride to capture different spatial information; Each pooled feature map is upsampled to match the size of the original feature map, and all upsampled feature maps are depth-wise concatenated with the original feature map.

Images