Multi-scale mamba based two-view finger vein recognition method and system

Abstract

The application discloses a two-view finger vein recognition method and system based on multi-scale Mamba, and relates to the technical field of biometric recognition. The method comprises the following steps: using a multi-scale Mamba feature extraction module to perform feature extraction on a finger vein image of each view respectively to obtain multi-scale features; enhancing the multi-scale features in each view to obtain enhanced features of the two views respectively; performing feature interaction on the enhanced features of the two views by using a Mamba feature interaction model to obtain interaction features of the two views; and splicing the interaction features of the two views and predicting a finger vein image category. The application solves the information loss and posture sensitivity problems existing in single-view recognition, and further improves the recognition performance and robustness of the recognition system.

Description

Technical Field

[0001] This invention relates to the field of biometric recognition technology, and in particular to a two-view finger vein recognition method and system based on multi-scale Mamba. Background Technology

[0002] The statements in this section are merely background information related to the present invention and do not necessarily constitute prior art.

[0003] In recent years, biometric identification technology has been widely used in daily life due to its advantages in security and convenience. Among them, facial recognition and fingerprint recognition are widely used because of their mature technology, low equipment cost, and ease of operation. However, these identification technologies based on external physiological characteristics are susceptible to physical damage and environmental factors, and are also prone to forgery. Finger veins are hidden under the skin of the finger, naturally resistant to wear and tear, not easily altered by changes in skin surface condition, and difficult to steal or copy. Therefore, finger vein recognition technology has developed rapidly and has gradually become a research hotspot in the field of biometric identification.

[0004] Most existing finger vein recognition methods rely on single-view images, failing to fully exploit the complementarity between images from different perspectives. Finger veins are distributed in a three-dimensional network within the finger; single-view imaging can only capture partial information about the veins, inevitably leading to information loss. Furthermore, changes in finger posture during acquisition, such as rotation and translation, can negatively impact recognition performance. Therefore, researching two-view finger vein recognition technology, and fully exploring and fusing the features of finger vein images from both perspectives, will further improve the recognition system's performance and robustness. Summary of the Invention

[0005] To address the shortcomings of existing technologies, the present invention aims to provide a two-view finger vein recognition method and system based on multi-scale Mamba, which solves the problems of information loss and posture sensitivity in single-view recognition, and further improves the recognition performance and robustness of the recognition system.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solution: The first aspect of this invention provides a two-view finger vein recognition method based on multi-scale Mamba, comprising the following steps: The multi-scale Mamba feature extraction module was used to extract features from the finger vein images from each viewpoint to obtain the multi-scale features of each viewpoint image. Multi-scale features within each viewpoint are enhanced to obtain enhanced features for both viewpoints. The Mamba feature interaction model is used to perform feature interaction on the enhanced features of the two perspectives to obtain the interaction features of the two perspectives. The interaction features from two perspectives are concatenated to predict the category of finger vein images.

[0007] Furthermore, the specific steps for feature extraction from finger vein images at each viewpoint are as follows; Shallow feature extraction is performed on finger vein images from each viewpoint; Then, the multi-scale Mamba module is used for multi-scale feature extraction.

[0008] Furthermore, the multi-scale Mamba module uses a multi-scale spiral scanning method to convert shallow features into sequences.

[0009] Furthermore, the specific steps for enhancing the multi-scale features within each viewpoint are as follows: The cross-attention mechanism is used to enhance the multi-scale features within each viewpoint, and the output of cross-attention at each scale is summed bit by bit to obtain the enhanced features for each viewpoint.

[0010] Furthermore, when enhancing a feature at a certain scale within the viewpoint, the query matrix for cross-attention is calculated from that scale feature, while the key matrix and value matrix are calculated from the fused features of the three scale features within the viewpoint.

[0011] Furthermore, the specific steps for feature interaction between the enhanced features of the two perspectives using the Mamba interaction model are as follows: The Mamba interaction module includes a cross-view fusion branch and two single-view processing branches. The cross-view fusion branch adds and fuses the features of the two views to perform preliminary cross-view information integration and obtain cross-view features. The two single-view processing branches process the enhanced features of the two views respectively to obtain single-view features.

[0012] By performing Mamba-based interaction between cross-view features and single-view features, interactive features from both perspectives are obtained.

[0013] Furthermore, the interaction features from the two perspectives are stitched together to predict the category of the finger vein image.

[0014] A second aspect of the present invention provides a two-view finger vein recognition system based on multi-scale Mamba, comprising: The multi-scale Mamba feature extraction module is configured to extract features from 0-degree and 45-degree finger vein images respectively, and obtain multi-scale features for each viewpoint image. The in-view multi-scale feature enhancement module is configured to enhance the multi-scale features within each view, resulting in enhanced features for both views. The two-view Mamba feature interaction module is configured to use the Mamba model to perform feature interaction on the enhanced features of the two views, and obtain the interaction features of the two views. The identity recognition module is configured to stitch together the interaction features from two perspectives and predict the category of the finger vein image.

[0015] A third aspect of the present invention provides a computer-readable storage medium storing a computer program adapted to be loaded by a processor and to execute steps in the two-view finger vein recognition method based on multi-scale Mamba as described in the first aspect of the present invention.

[0016] A fourth aspect of the present invention provides a computer device comprising: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the two-view finger vein recognition method based on multi-scale Mamba as described in the first aspect of the present invention.

[0017] The above one or more technical solutions have the following beneficial effects: This invention discloses a two-view finger vein recognition method and system based on multi-scale Mamba. First, multi-scale features are extracted from single-view finger vein images using multi-scale Mamba, and cross-attention is used to enhance the features at each scale through multi-scale fusion. Then, in the two-view feature interaction module, feature interaction between the two views is performed to capture the discriminative features of finger veins from both perspectives, solving the problems of information loss and pose sensitivity in single-view recognition. Finally, the features from both views are fused for finger vein recognition.

[0018] This invention includes a multi-scale Mamba feature extraction module, a viewpoint-based multi-scale feature enhancement module, a two-viewpoint Mamba feature interaction module, and an identity recognition module. First, features of the finger vein image are extracted using several convolutional layers and a multi-scale Mamba module. Specifically, the multi-scale Mamba module employs a three-scale spiral two-dimensional scanning strategy. Next, within a single viewpoint, cross-attention is used to enhance features at each scale through multi-scale fusion. Subsequently, feature interaction between the two viewpoints is performed in the two-viewpoint feature interaction module. Finally, the features from both viewpoints are fused for finger vein recognition.

[0019] Considering the limited discriminative information in single-view images, this invention investigates feature extraction, enhancement, interaction, and fusion of two-view images to improve the performance of finger vein recognition. Existing neural network-based finger vein recognition methods mostly use convolutional networks to extract features, which struggles to capture long-range dependencies in images. This invention uses a multi-scale Mamba module to extract image features, leveraging the advantages of selective state-space models in long sequence modeling to fully exploit finger vein features. Furthermore, this invention designs a multi-scale feature enhancement module to enhance features at each scale through multi-scale fusion, and a two-view Mamba feature interaction module to fully exploit the complementarity of features from both perspectives, providing discriminative features for subsequent recognition steps.

[0020] Advantages of additional aspects of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1 This is a framework diagram of the two-view finger vein recognition method based on multi-scale Mamba in Embodiment 1 of the present invention.

[0023] Figure 2 This is a flowchart of the two-view finger vein recognition method based on multi-scale Mamba in Embodiment 2 of the present invention; Detailed Implementation It should be noted that the following detailed descriptions are exemplary and intended to provide further illustration of the invention. Unless otherwise specified, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0024] It should be noted that the terminology used herein is for the purpose of describing particular embodiments only and is not intended to limit the scope of exemplary embodiments according to the invention. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. Furthermore, it should be understood that when the terms "comprising" and / or "including" are used in this specification, they indicate the presence of features, steps, operations, devices, components, and / or combinations thereof. The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0025] Example 1: Embodiment 1 of the present invention provides a two-view finger vein recognition method based on multi-scale Mamba, such as... Figure 1 As shown, it includes the following steps: Step 1: Use the multi-scale Mamba feature extraction module to extract features from the finger vein images from each viewpoint.

[0026] In this embodiment, two viewing angles are selected: a 0-degree viewpoint and a 45-degree viewpoint. For the 0-degree and 45-degree finger vein images, feature extraction is performed using the multi-scale Mamba feature extraction module to obtain multi-scale features.

[0027] In one specific implementation, this embodiment uses several convolutional layers and multi-scale Mamba to extract finger vein features. First, several convolutional layers are used to extract shallow features. Then, a multi-scale Mamba module is used for multi-scale feature extraction to capture long-range spatial dependencies. Considering that the main veins in the finger are relatively large while the branch veins are relatively small, a multi-scale scanning strategy is designed to extract features at different scales in the finger vein image. Furthermore, compared to row scanning and column scanning, spiral scanning starts from the central feature block and scans outward layer by layer in a clockwise direction, maintaining the spatial continuity of the veins to a certain extent in the generated sequence. Therefore, a multi-scale spiral scanning strategy is used in the multi-scale Mamba feature extraction module.

[0028] The specific steps are as follows: Step 1.1: Perform shallow feature extraction on the finger vein images at 0 degrees and 45 degrees respectively.

[0029] Specifically, given two-view finger vein images , where j represents the view index. For each view image, shallow feature extraction is performed using three convolutional layers.

[0030] Step 1.2: For the shallow features of each viewpoint image, perform multi-scale feature extraction using the multi-scale Mamba module.

[0031] Specifically, the multi-scale Mamba module utilizes linear layers, depthwise separable convolutional layers, SiLU activation function layers, and the S6 module to process shallow features. The multi-scale Mamba module uses a multi-scale spiral scanning method to convert shallow features into sequences.

[0032] More specifically, in the multi-scale Mamba scanning strategy, shallow features are divided into segments of size [missing information]. The feature blocks are transformed into a sequence using a multi-scale spiral scanning method. Scale 1 is a global scan, treating the shallow features as a single region; Scale 2 is a local scan, dividing the shallow features into four equal-sized regions; Scale 3 is also a local scan, dividing the shallow features into nine equal-sized regions. Each scale scan traverses the regions in an S-shape, from left to right, then from right to left after a line break. Within each region, a spiral scan is performed, starting from the central feature block and expanding outwards clockwise layer by layer until all feature blocks within that region have been traversed. For each scale scan, all feature blocks are arranged into a sequence according to the access order and fed into a selective state-space model (S6 module) for efficient feature extraction.

[0033] Step 2: Enhance the multi-scale features within each viewpoint to obtain the enhanced features for both viewpoints.

[0034] In one specific implementation, this embodiment uses cross-attention to enhance features at each scale through multi-scale fusion, resulting in enhanced features for each viewpoint. First, a query matrix is ​​calculated for the deep features at each scale within the viewpoint, yielding query matrices for the three scales. Next, the features at the three scales within the viewpoint are fused, and the key and value matrices of the fused features are calculated. Then, attention weights are calculated, and the value matrices are weighted. Finally, the outputs of the cross-attention at each scale are summed bit-by-bit to obtain the enhanced features within the viewpoint.

[0035] Step 2.1: Calculate the query matrix for deep features at each scale within the viewpoint.

[0036] Specifically, the deep features at three scales within viewpoint j are derived from... The representation, where i represents the scale index and j represents the view index, is shown in Formula 1: (1).

[0037] in, It is a learnable weight matrix. It is the query matrix of the i-th scale within viewpoint j.

[0038] Step 2.2: Fuse features from three scales within the fusion perspective, and calculate the key matrix and value matrix of the fused features.

[0039] Specifically, when fusing the three-scale features within each viewpoint, the feature sequences at the three scales are folded into feature matrices at the three scales, and the fused feature matrix is ​​obtained by summing the three feature matrices bit by bit. Based on this, the fused feature matrix is ​​expanded into a sequence to calculate the key matrix and value matrix of the fused features. The above steps are shown in Equations 2 and 3: (2), (3).

[0040] in, V and V are the key matrix and value matrix of the fused features, respectively. and It is a learnable weight matrix. and These are folding and unfolding operations.

[0041] Step 2.3: Calculate the attention weights and weight the value matrix.

[0042] Specifically, the above calculation is shown in Formula 4: (4).

[0043] in, It is a scale index. It is a perspective index. It is the dimension of K, and the output of cross-attention is Atte. i,j It is the weighted matrix of the value matrix at the i-th scale from the j-th perspective.

[0044] Step 2.4: Sum the outputs of cross-attention at each scale bitwise to obtain the enhanced features for the two perspectives.

[0045] Specifically, the above steps are shown in Formula 5: (5).

[0046] in, This is an enhanced feature for viewpoint j.

[0047] Step 3: Use the Mamba feature interaction model to perform feature interaction on the enhanced features of the two perspectives to obtain the interaction features of the two perspectives.

[0048] In one specific implementation, features from two perspectives are input into a two-view Mamba interaction module for feature interaction. Through efficient sequence modeling and cross-view feature fusion, complementary information between perspectives is captured. The Mamba interaction model includes a cross-view fusion branch and two single-view processing branches. The cross-view fusion branch adds and fuses the enhanced features from the two perspectives, while the two single-view processing branches process the enhanced features from the two perspectives respectively.

[0049] Step 3.1: Perform preliminary cross-view information integration on the enhanced features of the two perspectives to obtain cross-view features.

[0050] Specifically, the cross-view fusion branch adds and fuses the features from view one and view two, achieving initial cross-view information integration. The fused features are then processed sequentially through a linear layer and a SiLU activation function layer, serving as the output of the cross-view fusion branch, as shown in Equation 6: (6).

[0051] in, Indicates cross-perspective features, It is an activation function. It is a linear layer.

[0052] Step 3.2: Perform single-view feature processing on the enhanced features of the two perspectives to obtain single-view features.

[0053] Specifically, the two single-view processing branches process the enhancement features of the two views respectively. The processing of features from view one will be used as an example. After the features from view one are input into the single-view processing branch, they are processed sequentially through a linear layer, a depthwise separable convolutional layer, and a SiLU activation function layer, and then input into the selective state space module, i.e., the S6 module, as shown in Equation 7: (7).

[0054] in, It is a single-view feature. It is a depth-separable convolutional layer.

[0055] Step 3.3: Interact the cross-view features and the single-view features to obtain the interactive features of the two views.

[0056] Specifically, the output of the cross-view fusion branch is multiplied by the outputs of the two single-view processing branches to obtain the interactive single-view features, as shown in Equation 8: (8).

[0057] in, Let j be the interactive feature of viewpoint j.

[0058] Step 4: Concatenate the interaction features from the two perspectives and predict the category of the finger vein image.

[0059] In one specific implementation, the interaction features from the two viewpoints are concatenated and input into two fully connected layers for linear transformation to predict the category of the finger vein image. This step uses the cross-entropy loss function to optimize the network.

[0060] Example 2: Embodiment 2 of the present invention provides a two-view finger vein recognition system based on multi-scale Mamba, such as... Figure 2 As shown, it includes: The multi-scale Mamba feature extraction module is configured to extract features from 0-degree and 45-degree finger vein images respectively, and obtain multi-scale features for each viewpoint image. The in-view multi-scale feature enhancement module is configured to enhance the multi-scale features within each view, resulting in enhanced features for both views. The two-view Mamba feature interaction module is configured to use the Mamba model to perform feature interaction on the enhanced features of the two views, and obtain the interaction features of the two views. The identity recognition module is configured to stitch together the interaction features from two perspectives and predict the category of the finger vein image.

[0061] Example 3: Embodiment 3 of the present invention provides a computer-readable storage medium storing a computer program adapted for loading by a processor and executing the steps of the two-view finger vein recognition method based on multi-scale Mamba as described in Embodiment 1 of the present invention.

[0062] Example 4: Embodiment 4 of the present invention provides a computer device, the device comprising: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program, which, when executed by the processor, implements the steps of the two-view finger vein recognition method based on multi-scale Mamba as described in Embodiment 1 of the present invention.

[0063] The steps and methods involved in Examples 2, 3 and 4 above correspond to those in Example 1. For specific implementation details, please refer to the relevant description section of Example 1.

[0064] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed in this application can be implemented in electronic hardware or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application. In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in or transmitted through a computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired or wireless means. The computer-readable storage medium can be any available medium that a computer can access or a data processing device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium, an optical medium, or a semiconductor medium, etc. The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A two-view finger vein recognition method based on multi-scale Mamba, characterized in that, Includes the following steps: The multi-scale Mamba feature extraction module was used to extract features from the finger vein images from each viewpoint to obtain the multi-scale features of each viewpoint image. Multi-scale features within each viewpoint are enhanced to obtain enhanced features for both viewpoints. The Mamba feature interaction model is used to perform feature interaction on the enhanced features of the two perspectives to obtain the interaction features of the two perspectives. The two-view Mamba feature interaction model includes a cross-view fusion branch and two single-view processing branches. The cross-view fusion branch is used to add and fuse the features of the two views to obtain cross-view features. Two single-view processing branches are used to process the enhanced features of the two views respectively, resulting in single-view features; Interacting cross-view features and single-view features yields interactive features from both perspectives. Specifically, the output of the cross-view fusion branch is multiplied with the outputs of the two single-view processing branches to obtain the interactive single-view features. The interaction features from two perspectives are concatenated to predict the category of finger vein images; The specific steps for enhancing multi-scale features within each viewpoint are as follows: The cross-attention mechanism is used to enhance the features of each scale through multi-scale fusion, and the output of cross-attention at each scale is summed bit by bit to obtain the enhanced features of each viewpoint. When enhancing a feature at a certain scale within the viewpoint, the query matrix for cross-attention is calculated from that scale feature, while the key matrix and value matrix are calculated from the fused feature of the three scale features within the viewpoint.

2. The two-view finger vein recognition method based on multi-scale Mamba as described in claim 1, characterized in that, The specific steps for feature extraction from finger vein images at each viewpoint are as follows: Shallow feature extraction is performed on finger vein images from each viewpoint; Then, multi-scale feature extraction is performed using the multi-scale Mamba feature extraction module.

3. The two-view finger vein recognition method based on multi-scale Mamba as described in claim 2, characterized in that, The multi-scale Mamba feature extraction module uses a multi-scale spiral scanning method to convert shallow features into sequences.

4. A two-view finger vein recognition system based on multi-scale Mamba, characterized in that, include: The multi-scale Mamba feature extraction module is configured to extract features from 0-degree and 45-degree finger vein images respectively, and obtain multi-scale features for each viewpoint image. The in-view multi-scale feature enhancement module is configured to enhance the multi-scale features within each view, resulting in enhanced features for both views. The two-view Mamba feature interaction module is configured to use the Mamba feature interaction model to perform feature interaction on the enhanced features of the two views, and obtain the interaction features of the two views. The two-view Mamba feature interaction model includes a cross-view fusion branch and two single-view processing branches. The cross-view fusion branch is used to add and fuse the features of the two views to obtain cross-view features. Two single-view processing branches are used to process the enhanced features of the two views respectively, resulting in single-view features; Interacting cross-view features and single-view features yields interactive features from both perspectives. Specifically, the output of the cross-view fusion branch is multiplied with the outputs of the two single-view processing branches to obtain the interactive single-view features. The identity recognition module is configured to stitch together the interaction features from two perspectives and predict the category of the finger vein image; The specific steps for enhancing multi-scale features within each viewpoint are as follows: The cross-attention mechanism is used to enhance the features of each scale through multi-scale fusion, and the output of cross-attention at each scale is summed bit by bit to obtain the enhanced features of each viewpoint. When enhancing a feature at a certain scale within the viewpoint, the query matrix for cross-attention is calculated from that scale feature, while the key matrix and value matrix are calculated from the fused feature of the three scale features within the viewpoint.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program adapted to be loaded by a processor and executed as described in any one of claims 1-3, the two-view finger vein recognition method based on multi-scale Mamba.

6. A computer device, characterized in that, include: A processor, adapted to execute computer programs; A computer-readable storage medium storing a computer program that, when executed by the processor, implements the two-view finger vein recognition method based on multi-scale Mamba as described in any one of claims 1-3.

Images