A method and system for identity recognition based on fusion of fingerprint and palmprint features

Abstract

This invention provides a method and system for identity recognition based on fingerprint and palmprint feature fusion, relating to the field of biometric recognition technology. The method includes: acquiring biometric features based on modality separation, wherein the biometric features include original fingerprint features and corresponding original palmprint features; extracting features from the biometric features to obtain key features; performing feature fusion processing on the key features, combining cross-modal attention and residual connections to obtain fused features; calculating modality confidence weights based on the fused features to obtain a weight vector; performing weighted prediction based on the weight vector and the fused features to obtain a category probability distribution result; constructing an identity recognition model based on the category probability distribution result; and recognizing the identity to be detected using the identity recognition model to obtain the corresponding identity recognition result. This invention solves the problem of modality contribution imbalance in the feature fusion stage of multimodal biometric recognition.

Description

Technical Field

[0001] This invention relates to the field of biometric identification technology, and more specifically, to an identity recognition method and system based on the fusion of fingerprint and palm print features. Background Technology

[0002] With the in-depth development of biometric recognition technology, fingerprint or palmprint recognition systems based on a single modality have gradually exposed performance bottlenecks, exhibiting inherent limitations in key indicators such as noise resistance, inter-class similarity, intra-class differences, universality, anti-counterfeiting capabilities, and recognition accuracy. Multimodal biometric recognition technology, by integrating the fused biometric information of fingerprints and palmprints, constructs a more robust identity authentication system. This technology system uses multi-dimensional biometrics as the matching basis, fully utilizing the complementary characteristics and redundant information between different modalities, effectively alleviating the limitations of single-modal recognition, and demonstrating significant advantages in anti-counterfeiting verification, liveness detection, and accuracy optimization.

[0003] In a multimodal learning framework, different modalities contribute significantly to the final decision-making process. This difference in contribution not only weakens the representational power of secondary modalities but also leads to inconsistent convergence speeds among modalities, ultimately limiting the overall performance of the model. Traditional architectures based on fully connected layers and convolutional neural networks have significant limitations in dynamically balancing these modal differences. Their fixed parameter configurations are ill-suited to the dynamic changes in multimodal data distribution, specifically manifested in the lack of a collaborative optimization mechanism for the complementarity and competition among modalities during the feature fusion stage. This inherent flaw in the architectural design directly affects the generalization ability and decision robustness of multimodal systems in complex real-world scenarios.

[0004] Fingerprint or palmprint recognition, relying on a single biometric feature, suffers from significant security and reliability limitations in complex application scenarios. While multimodal recognition based on fingerprint and palmprint feature fusion improves system performance through information complementarity mechanisms, it still faces the core challenge of modal contribution imbalance during the feature fusion stage, becoming a key technical bottleneck restricting the engineering application of multimodal recognition systems. Summary of the Invention

[0005] The purpose of this invention is to provide an identity recognition method and system based on fingerprint and palmprint feature fusion to improve the aforementioned problems. To achieve the above objective, the technical solution adopted by this invention is as follows: Firstly, this application provides an identity recognition method based on the fusion of fingerprint and palmprint features, including: Biometric features are obtained based on modality separation, and the biometric features include original fingerprint features and corresponding original palm print features; Feature extraction is performed on the biometrics to obtain key features, which include fingerprint modality key features and palmprint modality key features; The key features are subjected to feature fusion processing, and cross-modal attention and residual connection are combined to obtain fused features, which include a first fused feature and a second fused feature. Based on the fusion features, the modal confidence weights are calculated to obtain the weight vector; Based on the weight vector and the fused features, a weighted prediction is performed to obtain the category probability distribution result; An identity recognition model is constructed based on the category probability distribution results, and the identity to be detected is identified through the identity recognition model to obtain the corresponding identity recognition result.

[0006] Secondly, this application also provides an identity recognition system based on the fusion of fingerprint and palmprint features, including: A classification unit is used to obtain biometric features based on modality separation, wherein the biometric features include original fingerprint features and corresponding original palm print features; The feature extraction unit is used to extract features from the biometric features to obtain key features, including fingerprint modality key features and palmprint modality key features; A feature fusion unit is used to perform feature fusion processing on the key features, and combine cross-modal attention and residual connection to obtain fused features, the fused features including a first fused feature and a second fused feature; The calculation unit is used to calculate the modality confidence weights based on the fusion features to obtain a weight vector; A weighted unit is used to perform weighted prediction based on the weight vector and the fused features to obtain the category probability distribution result; The prediction unit is used to construct an identity recognition model based on the category probability distribution results, and to identify the identity to be detected through the identity recognition model to obtain the corresponding identity recognition result.

[0007] The beneficial effects of this invention are as follows: (1) This invention ensures that fingerprint and palm print features play their respective roles in the fusion process by dynamically balancing and optimizing them, thus effectively solving the common modal imbalance problem in multimodal fusion and improving the accuracy of identity recognition. (2) Secondly, by independently adjusting the learning objectives and optimization process of each modality, the problem of inconsistent convergence rates of different modalities during the training phase is alleviated. This independent optimization strategy improves training efficiency and model performance, enabling the identity recognition system to reach a stable recognition state more quickly. In addition, this invention demonstrates high fusion efficiency and robustness under extreme noise conditions, and can maintain stable performance in complex and noisy environments, making it suitable for continuous authentication in real-world scenarios; (3) At the same time, this invention enables the model to flexibly adapt to different tasks and environments through category-level weighting and adaptive decision fusion mechanisms, thereby improving the model's versatility and adaptability. It can provide stable and efficient identity recognition services regardless of the application scenario or the diverse user groups it serves.

[0008] Other features and advantages of the invention will be set forth in the following description, and will be apparent in part from the description, or may be learned by practicing embodiments of the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures particularly pointed out in the written description, claims, and drawings. Attached Figure Description

[0009] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0010] Figure 1 This is a schematic diagram of the identity recognition method based on fingerprint and palmprint feature fusion as described in an embodiment of the present invention; Figure 2 This is a schematic diagram of the identity recognition model described in an embodiment of the present invention. Detailed Implementation

[0011] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0012] It should be noted that similar reference numerals and letters in the following figures indicate similar items; therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures. Furthermore, in the description of this invention, terms such as "first," "second," etc., are used only to distinguish descriptions and should not be construed as indicating or implying relative importance.

[0013] Example 1: This embodiment provides an identity recognition method based on the fusion of fingerprint and palm print features.

[0014] See Figure 1 The figure shows that the method includes steps S1, S2, S3, S4, S5 and S6.

[0015] Step S1: Obtain biometric features based on modality separation, wherein the biometric features include original fingerprint features and corresponding original palm print features; In this embodiment, the collected biometric images, including fingerprints and palm prints, are input into the modality separation module for preliminary processing. The modality separation module includes two independent preprocessing modules, which extract the original fingerprint features and the corresponding original palm print features, respectively, to ensure the independent expression space of different modal features.

[0016] Step S2: Extract features from the biometrics to obtain key features, including fingerprint modality key features and palmprint modality key features; Step S2 includes: Step S21: Construct fingerprint feature extraction branch and palmprint feature extraction branch based on lightweight ConvNeXt network; Step S22: Extract features from the original fingerprint features through the fingerprint feature extraction branch to obtain key fingerprint modal features; Step S23: Extract features from the original palm print features through the palm print feature extraction branch to obtain key palm print modal features.

[0017] In this embodiment, a lightweight ConvNeXt network is used to construct fingerprint feature extraction branches and palmprint feature extraction branches, respectively. By introducing large-kernel convolutional layers and adopting a depthwise hierarchical design, feature extraction performance is significantly improved while ensuring time efficiency.

[0018] Specifically, both the fingerprint feature extraction branch and the palm print feature extraction branch adopt a lightweight ConvNeXt network, which integrates large kernel convolutional kernels, inverted bottleneck structure and Transformer-style training paradigm to construct a multi-scale modeling mechanism from local details to global semantics.

[0019] For the fingerprint feature extraction branch and the palm print feature extraction branch, the feature extraction includes three stages: shallow feature extraction stage, mid-level feature fusion stage and deep feature abstraction stage.

[0020] In the shallow feature extraction stage, a 7×7 depthwise separable convolution is used to replace the traditional 3×3 convolution kernel. Under the premise of controllable parameter quantity, the receptive field of a single layer is expanded to a 49-pixel region, effectively capturing long-range spatial dependencies in the image.

[0021] In the mid-level feature fusion stage, a grouped convolution strategy is used to compress parameters (reducing the number of parameters to 1 / N, where N is the number of groups), while a channel attention mechanism is integrated to achieve dynamic calibration of feature channels. The deep convolution following each convolutional module further enhances cross-channel feature interaction.

[0022] In the deep feature abstraction stage, a pyramid-shaped multi-scale feature representation system is constructed through progressive downsampling (convolutional layers with a stride of 2) and an increasing channel number strategy. Simultaneously, a lightweight self-attention module is embedded in the top-level feature map, achieving effective fusion of local features and global context through a three-step operation: spatial dimension compression, attention weight generation, and feature recalibration.

[0023] Step S3: Perform feature fusion processing on the key features, and combine cross-modal attention and residual connection to obtain fused features, which include a first fused feature and a second fused feature; Step S3 includes: Step S31: Construct the first fusion layer and the second fusion layer; Step S32: Based on the first fusion layer, the original palm print features are introduced into the feature space of the fingerprint modality key features through residual connections, and weighted attention is calculated by combining cross-modal attention to obtain the first fusion feature; Step S33: Based on the second fusion layer, the original fingerprint features are introduced into the feature space of the palm print modality key features through residual connections, and weighted attention is calculated by combining cross-modal attention to obtain the second fusion features.

[0024] In this embodiment, the core idea of ​​the cross-modal attention fusion mechanism lies in constructing a residual feature-driven cross-modal interaction framework to achieve deep feature fusion of fingerprint and palmprint modalities. This mechanism uses original image features and high-level semantic features extracted by the ConvNeXt network as bimodal inputs. Through a feature decoupling strategy, the single-modal representation is decomposed into modality-shared features (modality-invariant terms) and modality-specific features (residual terms). The residual terms act as a bridge for cross-modal interaction, carrying modality-specific information; while the modality-invariant terms retain the shared underlying structure across modalities. Based on this, a self-attention mechanism is introduced to jointly model the bimodal residual features. By calculating the long-distance dependencies between cross-modal features, dynamic attention weights are generated. This weight allocation mechanism adaptively strengthens complementary modal features while suppressing redundant noise. Finally, through residual connections, the fused features and original features are information-complementary, achieving progressive optimization of multimodal representations.

[0025] Specifically, a bilateral residual feature fusion mechanism is constructed through the first fusion layer and the second fusion layer, and its formula is as follows: In the formula, Indicates the first fusion feature, Indicates the second fusion feature, Indicates the original features of the fingerprint. Indicates the original characteristics of palm prints. Indicates the first residual characteristic. Indicates the second residual characteristic. Indicates key features of fingerprint modality, This represents the key features of palmprint modality. express transpose, This represents the attention weight matrix of the original fingerprint features to the key features of the palm print modality. This represents the attention weight matrix of the original palm print features to the key features of the fingerprint modality. Dimensions representing key features of palmprint modality. Dimensions representing key features of fingerprint modalities This represents the softmax function.

[0026] Step S4: Calculate the modal confidence weights based on the fusion features to obtain the weight vector; In this embodiment, an adaptive decision fusion strategy is used to calculate the weight vector. The contribution of multimodal features to the final decision is dynamically quantified. Using cross-modal fusion features as input, a modality confidence evaluation mechanism is constructed through a decision-level fusion paradigm to avoid overfitting to a single modality.

[0027] Step S4 includes: Step S41: Calculate the average value of the first fusion feature and the second fusion feature along the time dimension to obtain the corresponding first compressed feature and second compressed feature; Step S42: Concatenate the first compressed feature and the second compressed feature to obtain the concatenated feature; Step S43: Calculate the modal confidence weights of the spliced ​​features using the softmax function, and obtain the weight vector by combining the weight matrix projection and bias term adjustment.

[0028] In this embodiment, the formula for calculating the weight vector is: In the formula, Represents the weight vector. This represents the softmax function. Represents the weight matrix. Indicates the bias term. This indicates the operation of calculating the average value along the time dimension. express and splicing operation, Indicates the first fusion feature, Indicates the second fusion feature, , Represents the real number field. Indicates the target dimension of the projection. and These represent the feature dimensions of the fingerprint modality and the palmprint modality, respectively.

[0029] Understandably, by calculating the average value along the time dimension, the fused features can be dynamically weighted, thereby achieving more adaptive and robust decision-making.

[0030] Step S5: Perform weighted prediction based on the weight vector and the fused features to obtain the category probability distribution results; In this embodiment, a category-level weighting strategy is adopted for weighted prediction. The integrated cross-modal fusion features and the prediction results of single modality (i.e., the probability distribution results under different modalities) are used as the joint input of the weighted prediction module. By aggregating features by category, the category differences of modality contribution are explicitly modeled, and the weight allocation is ensured to be consistent with the task objective.

[0031] Finally, through adaptive decision fusion and category-level weight correction, the output is a dynamically weighted multimodal fusion prediction result (i.e., the category probability distribution result, which includes the confidence score of the input sample belonging to each preset category).

[0032] Step S5 includes: Step S51: Pass the first fused feature through two fully connected layers to obtain the probability distribution result under the fingerprint modality; Step S52: Pass the second fused feature through two fully connected layers to obtain the probability distribution result under the palmprint modality; Step S53: Match and weight the probability distribution results of the fingerprint mode and the palm print mode using the weight vector to obtain the category probability distribution results.

[0033] In this embodiment, the formula for calculating the category probability distribution result is as follows: In the formula, Represents hidden layer features. Represents the ReLU activation function. This indicates a fusion feature, namely the first fusion feature or the second fusion feature. This represents the probability distribution result. This represents the softmax function. and These represent the weight matrices of the first and second fully connected layers, respectively. and Let represent the bias vectors of the first fully connected layer and the second fully connected layer, respectively. This represents the category probability distribution results. Represents the weight vector. This represents the probability distribution results under fingerprint modalities. This represents the probability distribution results under the palmprint modality. This represents the Hadamard product.

[0034] in, Responsible for mapping the fused features to the hidden layer space, This maps the hidden layer features to the category space.

[0035] Step S6: Construct an identity recognition model based on the category probability distribution results, and use the identity recognition model to identify the identity to be detected, thereby obtaining the corresponding identity recognition result.

[0036] Step S6 involves the following steps in constructing the identity recognition model: Step S61: Construct a modality separation module, which extracts biometric features from the palm image; Step S62: Construct a feature extraction module that takes biometric features as input and key features as output. The feature extraction module includes a fingerprint feature extraction branch and a palm print feature extraction branch. Step S63: Construct a bilateral residual feature fusion mechanism with biometric features and key features as inputs and fused features as outputs. The bilateral residual feature fusion mechanism includes a first fusion layer and a second fusion layer. Step S64: Construct a weight generation module that takes fused features as input and weight vectors as output; Step S65: Construct a weighted prediction module that takes fused features and weight vectors as input and class probability distribution results as output; Step S66: The identity recognition model is constructed through the modality separation module, feature extraction module, bilateral residual feature fusion mechanism, weight generation module and weighted prediction module.

[0037] In this embodiment, the identity recognition model is as follows: Figure 2 As shown, before identifying the identity to be detected, the identity recognition model needs to be trained. Specifically, the fingerprint feature images used for training are from the FVC2002-DB3_A database, containing 800 images, and the palmprint feature images used for training are from the Tongji University palmprint database, containing 12,000 ROI images. A stochastic gradient descent (SGD) optimizer is used, with a total of 200 epochs. The learning rate is scheduled using a cosine decay strategy, with an initial value of 0.01, linearly decaying to 0.001 at the end of training. To balance the model's fitting ability and generalization performance, the loss function is label smoothed cross-entropy (LS-CE). By introducing a label distribution softening mechanism (smoothing coefficient = 0.1), the risk of overfitting is effectively mitigated and the model's robustness to noisy labels is enhanced.

[0038] Example 2: This embodiment provides an identity recognition system based on fingerprint and palmprint feature fusion, the system comprising: A classification unit is used to obtain biometric features based on modality separation, wherein the biometric features include original fingerprint features and corresponding original palm print features; The feature extraction unit is used to extract features from the biometric features to obtain key features, including fingerprint modality key features and palmprint modality key features; A feature fusion unit is used to perform feature fusion processing on the key features, and combine cross-modal attention and residual connection to obtain fused features, the fused features including a first fused feature and a second fused feature; The calculation unit is used to calculate the modality confidence weights based on the fusion features to obtain a weight vector; A weighted unit is used to perform weighted prediction based on the weight vector and the fused features to obtain the category probability distribution result; The prediction unit is used to construct an identity recognition model based on the category probability distribution results, and to identify the identity to be detected through the identity recognition model to obtain the corresponding identity recognition result.

[0039] The feature fusion unit includes: The first building subunit is used to build the first fusion layer and the second fusion layer; The first fusion subunit is used to, based on the first fusion layer, introduce the original palm print features into the feature space of the fingerprint modality key features through residual connection, and perform weighted attention calculation by combining cross-modal attention to obtain the first fusion feature; The second fusion subunit is used to, based on the second fusion layer, introduce the original fingerprint features into the feature space of the palm print modality key features through residual connections, and perform weighted attention calculation by combining cross-modal attention to obtain the second fusion feature.

[0040] The computing unit includes: The first calculation subunit is used to calculate the average value of the first fusion feature and the second fusion feature along the time dimension to obtain the corresponding first compressed feature and second compressed feature. The splicing subunit is used to splice the first compressed feature and the second compressed feature to obtain the spliced ​​feature; The second calculation subunit is used to calculate the modal confidence weights of the spliced ​​features using the softmax function, and then combine the weight matrix projection and bias term adjustment to obtain the weight vector.

[0041] The weighting unit includes: The first processing subunit is used to pass the first fused feature through two fully connected layers to obtain the probability distribution result under the fingerprint modality; The second processing subunit is used to pass the second fused feature through two fully connected layers to obtain the probability distribution result under the palmprint modality; The weighted subunit is used to match and weight the probability distribution results of the fingerprint modality and the palmprint modality through the weight vector to obtain the category probability distribution results.

[0042] It should be noted that the specific methods by which each module performs operations in the system described in the above embodiments have been described in detail in the embodiments related to the method, and will not be elaborated here.

[0043] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

[0044] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An identity recognition method based on fingerprint and palmprint feature fusion, characterized in that, include: Biometric features are obtained based on modality separation, and the biometric features include original fingerprint features and corresponding original palm print features; Feature extraction is performed on the biometrics to obtain key features, which include fingerprint modality key features and palmprint modality key features; The key features are subjected to feature fusion processing, and cross-modal attention and residual connection are combined to obtain fused features, which include a first fused feature and a second fused feature. Based on the fusion features, the modal confidence weights are calculated to obtain the weight vector; Based on the weight vector and the fused features, a weighted prediction is performed to obtain the category probability distribution result; An identity recognition model is constructed based on the category probability distribution results, and the identity to be detected is identified through the identity recognition model to obtain the corresponding identity recognition result.

2. The identity recognition method based on fingerprint and palmprint feature fusion according to claim 1, characterized in that... The step of extracting key features from the biometric image includes: Fingerprint feature extraction and palmprint feature extraction branches are constructed based on a lightweight ConvNeXt network; The fingerprint feature extraction branch extracts features from the original fingerprint features to obtain key fingerprint modality features; The palmprint feature extraction branch extracts features from the original palmprint features to obtain key palmprint modal features.

3. The identity recognition method based on fingerprint and palmprint feature fusion according to claim 1, characterized in that... The feature fusion process performed on the key features, combining cross-modal attention and residual connections to obtain fused features, includes: Construct the first fusion layer and the second fusion layer; Based on the first fusion layer, the original palm print features are introduced into the feature space of the fingerprint modality key features through residual connections, and weighted attention is calculated by combining cross-modal attention to obtain the first fusion feature; Based on the second fusion layer, the original fingerprint features are introduced into the feature space of the palm print modality key features through residual connections, and weighted attention is calculated by combining cross-modal attention to obtain the second fusion features.

4. The identity recognition method based on fingerprint and palmprint feature fusion according to claim 1, characterized in that... The step of calculating modal confidence weights based on the fusion features to obtain a weight vector includes: The first fusion feature and the second fusion feature are averaged along the time dimension to obtain the corresponding first compressed feature and second compressed feature. The first compressed feature and the second compressed feature are concatenated to obtain the concatenated feature; The spliced ​​features are used to calculate modal confidence weights using the softmax function, and the weight vector is obtained by combining the weight matrix projection and bias term adjustment.

5. The identity recognition method based on fingerprint and palmprint feature fusion according to claim 1, characterized in that... The weighted prediction based on the weight vector and the fused features to obtain the category probability distribution result includes: The first fused feature is passed through two fully connected layers to obtain the probability distribution result under the fingerprint modality; The second fusion feature is passed through two fully connected layers to obtain the probability distribution result under the palmprint modality; The probability distribution results of fingerprint mode and palmprint mode are matched and weighted by the weight vector to obtain the category probability distribution results.

6. The identity recognition method based on fingerprint and palmprint feature fusion according to claim 1, characterized in that... The steps for constructing the identity recognition model are as follows: A modality separation module is constructed, which extracts biometric features from a palm image; A feature extraction module is constructed that takes biometric features as input and key features as output. The feature extraction module includes a fingerprint feature extraction branch and a palmprint feature extraction branch. A bilateral residual feature fusion mechanism is constructed, which takes biological features and key features as inputs and fused features as outputs. The bilateral residual feature fusion mechanism includes a first fusion layer and a second fusion layer. Construct a weight generation module that takes fused features as input and weight vectors as output; Construct a weighted prediction module that takes fused features and weight vectors as input and class probability distribution results as output; An identity recognition model is constructed by a modality separation module, a feature extraction module, a bilateral residual feature fusion mechanism, a weight generation module, and a weighted prediction module.

7. An identity recognition system based on the fusion of fingerprint and palmprint features, characterized in that, include: A classification unit is used to obtain biometric features based on modality separation, wherein the biometric features include original fingerprint features and corresponding original palm print features; The feature extraction unit is used to extract features from the biometric features to obtain key features, including fingerprint modality key features and palmprint modality key features; A feature fusion unit is used to perform feature fusion processing on the key features, and combine cross-modal attention and residual connection to obtain fused features, the fused features including a first fused feature and a second fused feature; The calculation unit is used to calculate the modality confidence weights based on the fusion features to obtain a weight vector; A weighted unit is used to perform weighted prediction based on the weight vector and the fused features to obtain the category probability distribution result; The prediction unit is used to construct an identity recognition model based on the category probability distribution results, and to identify the identity to be detected through the identity recognition model to obtain the corresponding identity recognition result.

8. The identity recognition system based on fingerprint and palmprint feature fusion according to claim 7, characterized in that, The feature fusion unit includes: The first building subunit is used to build the first fusion layer and the second fusion layer; The first fusion subunit is used to, based on the first fusion layer, introduce the original palm print features into the feature space of the fingerprint modality key features through residual connection, and perform weighted attention calculation by combining cross-modal attention to obtain the first fusion feature; The second fusion subunit is used to, based on the second fusion layer, introduce the original fingerprint features into the feature space of the palm print modality key features through residual connections, and perform weighted attention calculation by combining cross-modal attention to obtain the second fusion feature.

9. The identity recognition system based on fingerprint and palmprint feature fusion according to claim 7, characterized in that, The computing unit includes: The first calculation subunit is used to calculate the average value of the first fusion feature and the second fusion feature along the time dimension to obtain the corresponding first compressed feature and second compressed feature. The splicing subunit is used to splice the first compressed feature and the second compressed feature to obtain the spliced ​​feature; The second calculation subunit is used to calculate the modal confidence weights of the spliced ​​features using the softmax function, and then combine the weight matrix projection and bias term adjustment to obtain the weight vector.

10. The identity recognition system based on fingerprint and palmprint feature fusion according to claim 7, characterized in that, The weighting unit includes: The first processing subunit is used to pass the first fused feature through two fully connected layers to obtain the probability distribution result under the fingerprint modality; The second processing subunit is used to pass the second fused feature through two fully connected layers to obtain the probability distribution result under the palmprint modality; The weighted subunit is used to match and weight the probability distribution results of the fingerprint modality and the palmprint modality through the weight vector to obtain the category probability distribution results.

Images