Palm print and palm vein bimodal identification method and electronic equipment

Abstract

The embodiment of the invention relates to the technical field of biological recognition, and discloses a palm print and palm vein bimodal recognition method and electronic equipment. According to the method, the quality weighted fusion is performed on the plurality of palm print images of the same palm, the fused palm print feature vector with higher robustness is generated, and the process can adaptively improve the contribution of the high-quality image and suppress the interference of the low-quality image; synchronously acquiring corresponding palm vein images and extracting feature vectors of the palm vein images; and a progressive identification strategy is adopted, rapid preliminary screening is performed by using the fused palmprint features so as to reduce the identification range, and then a preliminary screening result is accurately authenticated in combination with the palmprint vein features with higher anti-falsification. According to the method, the quality sensitivity defect of palmprint recognition is effectively overcome through a quality adaptive fusion mechanism, the complementary advantage of bimodal is fully played, and the accuracy, stability and safety of identity recognition in a complex actual scene are remarkably improved.

Description

Technical Field

[0001] This application relates to the field of biometrics, and in particular to a method and electronic device for palm print and palm vein dual-modal recognition. Background Technology

[0002] Biometric technology is increasingly being applied in fields such as security, finance, and access control. Palmprint recognition and palm vein recognition, as two important biometric identification methods, each have their advantages and disadvantages. Palmprint features are easy to collect and rich in information, but as an epidermal feature, they are easily affected by changes in lighting, dirt or wear on the palm surface, leading to large fluctuations in image quality and unstable feature extraction. Furthermore, palmprints also pose security risks of being copied and forged. Palm vein features are located subcutaneously, possessing natural liveness and anti-counterfeiting properties, but their image acquisition is more limited by equipment, and when training data is insufficient, the recognition accuracy and generalization ability of a single model are limited.

[0003] In related technologies in this field, there are studies on fusing palmprint and palm vein recognition to combine the advantages of both. However, common fusion methods are mostly simple feature stitching or decision-level fusion, failing to fully consider the core pain point of uncontrollable palmprint image quality in practical application scenarios. When the palmprint image quality is low, the extracted features are already unreliable. If they are still fused equally with high-quality palm vein features, noise will be introduced, reducing the overall system's recognition performance and stability. Therefore, how to design a fusion recognition method that can adapt to fluctuations in palmprint image quality, intelligently utilize bimodal information, and improve recognition accuracy while ensuring security has become an urgent technical problem to be solved in this field. Summary of the Invention

[0004] The purpose of this application is to provide a palmprint and palm vein dual-modal recognition method and electronic device, which effectively overcomes the quality sensitivity defect of palmprint recognition through a quality adaptive fusion mechanism, fully leverages the complementary advantages of dual-modality, and significantly improves the accuracy, stability and security of identity recognition in complex real-world scenarios.

[0005] To address the aforementioned technical problems, embodiments of this application provide a dual-modal palmprint and palm vein recognition method, comprising: obtaining a fused palmprint feature vector by quality-weighted fusion based on multiple palmprint images of the same hand; acquiring a palm vein image corresponding to the palmprint image and extracting a palm vein feature vector; performing preliminary screening by comparing the fused palmprint feature vector with the feature vector of a registered palmprint; and performing final recognition based on the preliminary screening result and the comparison result of the palm vein feature vector and the feature vector of a registered palm vein.

[0006] Embodiments of this application also provide an electronic device, including: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the above-described palm print and palm vein bimodal recognition method.

[0007] In this embodiment, multiple palmprint images of the same hand are quality-weightedly fused to generate a more robust fused palmprint feature vector. This process adaptively enhances the contribution of high-quality images and suppresses interference from low-quality images. Simultaneously, corresponding palm vein images are acquired and their feature vectors are extracted. A progressive recognition strategy is adopted: first, the fused palmprint features are used for rapid initial screening to narrow the recognition range; then, the palm vein features, which offer stronger anti-spoofing capabilities, are combined to accurately authenticate the initial screening results. This invention effectively overcomes the quality sensitivity defect of palmprint recognition through a quality-adaptive fusion mechanism, fully leverages the complementary advantages of dual modalities, and significantly improves the accuracy, stability, and security of identity recognition in complex real-world scenarios. Attached Figure Description

[0008] One or more embodiments are illustrated by way of example with reference numerals in the accompanying drawings. These illustrations do not constitute a limitation on the embodiments. Elements with the same reference numerals in the drawings are denoted as similar elements. Unless otherwise stated, the figures in the drawings are not to be limited by scale.

[0009] Figure 1 This is a flowchart of a palm print and palm vein dual-modal recognition method according to an embodiment of this application; Figure 2 This is a schematic diagram of the input-output relationship of palm print and palm vein training features according to an embodiment of this application; Figure 3 This is a flowchart of palm print and palm vein dual-modal training and operation according to one embodiment of this application; Figure 4 This is a schematic diagram of the structure of an electronic device according to another embodiment of this application. Detailed Implementation

[0010] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the various embodiments of this application will be described in detail below with reference to the accompanying drawings. However, those skilled in the art will understand that many technical details have been presented in the various embodiments of this application to enable readers to better understand this application. However, even without these technical details and various changes and modifications based on the following embodiments, the technical solutions claimed in this application can be implemented. The division of the various embodiments below is for the convenience of description and should not constitute any limitation on the specific implementation of this application. The various embodiments can be combined with and referenced by each other without contradiction.

[0011] One embodiment of this application relates to a palmprint and palm vein bimodal recognition method, which can be applied to a palmprint and palm vein bimodal recognition device. The palmprint recognition device can be a computer device such as a mobile phone or computer with palmprint recognition hardware, or an electronic device composed of electronic components such as chips. In this embodiment, multiple palmprint images of the same hand are quality-weightedly fused to generate a more robust fused palmprint feature vector. This process adaptively enhances the contribution of high-quality images and suppresses interference from low-quality images. Simultaneously, corresponding palm vein images are acquired and their feature vectors are extracted. A progressive recognition strategy is adopted: first, the fused palmprint features are used for rapid initial screening to narrow the recognition range; then, the palm vein features with stronger anti-spoofing capabilities are combined to accurately authenticate the initial screening results. This invention effectively overcomes the quality sensitivity defect of palmprint recognition through a quality adaptive fusion mechanism, fully leverages the complementary advantages of bimodality, and significantly improves the accuracy, stability, and security of identity recognition in complex real-world scenarios. The implementation details of the palmprint and palm vein bimodal recognition method of this embodiment are described below. The following content is only for ease of understanding and is not essential for implementing this solution.

[0012] like Figure 1 As shown, in step 101, the palmprint and palm vein dual-modal recognition device obtains a fused palmprint feature vector based on multiple palmprint images of the same hand through quality-weighted fusion. The quality-weighted fusion mechanism adaptively processes palmprint images of different qualities, prioritizing the use of high-quality image information and suppressing interference from low-quality images.

[0013] In one example, step 101 could be: processing multiple palmprint images using a quality fusion feature extraction network. This network includes parallel feature extraction branches with shared parameters, and a quality estimation branch configured for each feature extraction branch. Each palmprint image is input into a different feature extraction branch to extract its feature vector, and the corresponding quality estimation branch outputs the quality weights of the palmprint images. The normalized quality weights of the palmprint images are then used to weight the feature vectors of the corresponding palmprint images, and the feature vectors of all weighted palmprint images are summed to obtain the fused palmprint feature vector. This improves the robustness and recognition accuracy of feature extraction, making it particularly suitable for real-world scenarios involving changes in lighting and stains.

[0014] By inputting the multiple palmprint images obtained above into the trained quality fusion palmprint feature extraction model, the palmprint feature vector can be obtained. Specifically, the training method for the quality fusion palmprint feature extraction model used for palmprint recognition can be as follows: Step 201: Obtain the training dataset of palm print and palm vein images, and perform data augmentation processing on them respectively.

[0015] Step 201-a: Perform data augmentation on the palmprint image, including random image processing such as horizontal flipping, brightness adjustment, contrast adjustment, and random viewing angle transformation.

[0016] Step 201-b: Random data augmentation methods for palm vein images include, for example: horizontal flipping, brightness adjustment, contrast adjustment, Gaussian blur adjustment, and contrast adjustment.

[0017] Step 201-c: Introduce data augmentation methods that degrade quality, such as random processing of palmprint images including Gaussian blur, compressed blur, resolution reduction, distortion, etc.

[0018] Step 202: Input the processed image data into the feature extraction network for training.

[0019] Step 202-a: The palmprint feature extraction network is a quality fusion feature extraction network, and its structure is as follows: Figure 2 As shown, multiple data-augmented palmprint images are input into the network. Considering the efficiency of the network model, a fixed number of images are selected as input images (e.g., 3 images).

[0020] Step 202-b: Input palmprint image 1 into a deep learning network (e.g., select a commonly used residual network structure or visual transformation network, etc.), and the deep learning network outputs quality weights. , and the feature output vector of palmprint image 1 Quality weights after regularization and activation processing For eigenvectors After weighting, the quality-weighted feature output is obtained. The quality-weighted feature outputs of palmprint image 2 and palmprint image 3 were obtained in the same way. and Finally, the feature vector output of the quality fusion is: .

[0021] Step 202-c: The deep learning networks input to palmprint images 2 and 3 have the same structure as the deep learning network for palmprint image 1 and share network parameters.

[0022] In one example, the aforementioned quality fusion feature extraction network is trained using a combined loss function, which includes at least: an angular margin loss function to enhance the discriminative power of inter-class features; and an unsupervised energy minimization loss function to constrain the redundancy of network weight parameters. This enhances the discriminative power of features from different classes while preventing overfitting caused by network parameter redundancy, thus improving the model's generalization ability and feature representation capability. Specifically: Step 203: Based on the network parameters and network outputs mentioned above, combined with the labels of the dataset, the network can be trained using a loss function.

[0023] Step 204: The network loss function adopts a combination of "additive angular margin loss, such as the additive angular margin loss function (ArcFace) and unsupervised energy minimization loss". Specifically, the additive angular margin loss is used to enhance the inter-class discriminative power of palmprint feature vectors (e.g., to make the distribution of palmprint main lines and vein branches of different palms farther apart in the feature space); the newly added unsupervised energy minimization loss is used to optimize the network's own weight parameters, solving the problem of incomplete extraction or overfitting of palmprint details (such as bifurcations around the palmprint main lines) due to network weight redundancy. The two are balanced by the weight coefficient (λ) to optimize "classification accuracy" and "feature diversity".

[0024] Step 205: The unsupervised energy minimization loss is directly applied to the weight parameters of the feature extraction network (the network weight parameters are mainly the parameters of the convolutional layer and the fully connected layer). By quantifying the "redundancy" of the weights, the network is forced to learn more comprehensive palmprint features.

[0025] In one example, the above unsupervised energy minimization loss function can be calculated as follows: normalize the weight parameters of a specified layer in the network; calculate the cosine or Euclidean distance between any two weight vectors within the same layer; calculate the energy loss of the same layer based on the cosine or Euclidean distance, with a larger energy loss for closer cosine or Euclidean distances to penalize feature extraction redundancy; sum the energy losses calculated from different layers after applying different constraint coefficients to obtain the total unsupervised energy loss. This improves the comprehensiveness and effectiveness of feature extraction, especially significantly enhancing the extraction of palm print details (such as bifurcation lines). The specific operation can be: Step 205-a: Normalize the weight parameters of each layer of the network and map them to the unit hypersphere space to uniformly measure the directional differences of different weight vectors; Step 205-b: Calculate the distance between any two weight vectors in the same layer: If the weight vectors are too close (meaning the network tends to repeatedly extract certain features, such as focusing only on the main line of the palm print while ignoring more subtle bifurcations), then apply a stronger penalty (the closer the distance, the stronger the penalty). Step 205-c: Summarize the penalty values ​​of all layers to be constrained to form the energy loss of that layer, ensuring that the network simultaneously focuses on the multi-dimensional features of the palm print (main lines and details, trunk and bifurcation).

[0026] Step 206: Setting different coefficients for energy loss in different layers of the network makes parameter control easier to adjust.

[0027] In one example, different constraint coefficients are applied to the energy losses calculated for different layers. For instance, when the quality fusion feature extraction network is a convolutional neural network, a first coefficient is applied to the energy loss of the final feature extraction layer, and a second coefficient is applied to the energy losses of other layers, with the first coefficient being greater than the second. When the quality fusion feature extraction network is a Vision Transformer network, a third coefficient is applied to the energy loss of the Transformer encoder layer, and a fourth coefficient is applied to the energy loss of the classification head layer, with the fourth coefficient being greater than the third. Applying differentiated constraint coefficients to different layers based on the network structure enables fine-tuning of deep and shallow features, improving network training efficiency and final recognition performance. Specifically: Step 206-a: If a Convolutional Neural Network (CNN) series is used as the feature extraction network, parameters are set to control the "weights of the final feature extraction layer" and the "weights of other layers". The final feature extraction layer extracts the deep information of the network and outputs the fused feature vector of the palm print, which has the greatest impact on the recognition result. Therefore, the constraint is controlled by setting an independent coefficient (beta). Other layers extract the shallow and middle-level feature information of the palm print. Appropriate constraints are applied by the coefficient (alpha) to ensure the comprehensiveness of the basic features.

[0028] Step 206-b: If the Vision Transformer (ViT) series is used as the feature extraction network, parameters are set for the initial embedding layer, the Transformer encoder layer, and the classification head layer. Considering computational cost control and to retain more shallow palmprint information, the convolution parameters of the initial embedding layer (the first layer for palmprint input) are not included in the loss calculation. The parameters of the Transformer encoder layer are constrained by setting independent coefficients (alpha). The classification head layer, being the layer before the output of the fused palmprint feature vector, has a significant impact on the result; therefore, independent coefficients (beta) are set for the classification head layer.

[0029] Step 207: During network model training, the parameters of the network model are updated with the goal of minimizing the combined loss function. Ultimately, a high-quality fusion palmprint feature extraction model is obtained.

[0030] It is worth noting that the model training process can also indirectly reflect the model's usage and operating principles. Therefore, some processes involved in the above training process can be implemented in conjunction with the above method embodiments. The relevant technical details mentioned in the above method embodiments are still valid in the training process implementation, and will not be repeated here to reduce repetition. Correspondingly, the relevant technical details mentioned in the training embodiments can also be applied to the above method embodiments.

[0031] After model training, palmprint images need to be acquired, as their quality is inextricably linked to the recognition results. In one example, before obtaining the fused palmprint feature vector through quality-weighted fusion in step 101, multiple palm images of the same hand are acquired; the palm images are then horizontally corrected using palm keypoint detection; and a fixed-size region of interest (ROI) is cropped based on the length of the line connecting the keypoints, resulting in the palmprint image. Through palm keypoint detection, horizontal correction, and ROI cropping, image standardization is achieved, reducing the impact of changes in acquisition angle and position on recognition, and improving the consistency of feature extraction and system stability. Specifically: Step 301: A visible light (Red, Green, Blue, RGB) camera acquires multiple consecutive palm images (e.g., 3 consecutive images, consistent with the settings during model training), and an infrared (IR) camera acquires multiple corresponding IR palm images. The palm should be facing upwards during acquisition, thus obtaining RGB palm images and IR palm images.

[0032] Step 301-a: Perform palm key point detection on the RGB palm image to obtain key point K1 between the index finger and the palm and key point K2 between the little finger and the palm. Based on the angle between the line connecting K1 and K2 and the horizontal line, perform horizontal correction on the palm image to obtain the corrected palm image.

[0033] Step 301-b: Use the length of the line connecting K1 and K2 of the corrected palm as the side length Pw of the square region of the palm. Take the square region with side length Pw, which has K1 and K2 as one side, as the Region of Interest (ROI) of the palm. Cropping the ROI yields the ROI image, which is the palmprint image.

[0034] In step 102, the palmprint and palm vein dual-modal recognition device acquires the palm vein image corresponding to the palmprint image and extracts the palm vein feature vector. This combines two biometric features—palmprint and palm vein—for recognition, improving accuracy and anti-counterfeiting capabilities. Furthermore, compared to methods in related technologies that require acquiring each palmprint image and each palm vein image individually, this embodiment eliminates the need to acquire the palm vein image corresponding to each palmprint image. It only requires automatically selecting the palmprint image with the highest quality weight after quality-weighted fusion, and then acquiring the palm vein image corresponding to that image, saving workload and improving efficiency.

[0035] In one example, obtaining the palm vein image corresponding to the palmprint image in step 102 can be achieved by: selecting the palmprint image with the highest quality based on the quality weights of each palmprint image obtained during the quality-weighted fusion process; and obtaining the palm vein image captured by an infrared camera at the same acquisition time as the palmprint image with the highest quality. By selecting the palmprint image with the highest quality and obtaining its corresponding palm vein image at the same time, image alignment and feature consistency are achieved, avoiding feature misalignment caused by differences in acquisition time, and improving the matching accuracy of subsequent multimodal fusion recognition. Specifically: Based on the quality weight of the palm print image obtained in step 202-b above, the palm vein image corresponding to the palm print image with the highest quality weight is selected. Using the same processing method, steps 301-a and 302-b are repeated, and the processed image is set as an IR palm image to obtain the palm vein image.

[0036] The palm vein feature extraction network can be a single-input deep learning feature extraction network. Considering that the amount of palm vein training data is not as large as that of palm print training data, a commonly used CNN network such as the Residual Network (ResNet) network structure is selected as the feature extraction network. The augmented palm vein image is input into the network for training to obtain a trained palm vein feature extraction model.

[0037] Similarly, by inputting the palm vein image into the trained palm vein feature extraction model, the palm vein feature vector can be obtained.

[0038] In step 103, the palm print and palm vein dual-modal recognition device performs preliminary screening by comparing the fused palm print feature vector with the feature vector of the registered palm print, and performs final recognition based on the preliminary screening results and the comparison results of the palm vein feature vector and the feature vector of the registered palm vein.

[0039] In one example, step 103, which involves comparing the fused palmprint feature vector with the feature vectors of registered palmprints for preliminary screening, can be achieved by: calculating the palmprint similarity between the fused palmprint feature vector and the feature vectors of all registered palmprints; selecting registered user identifiers whose palmprint similarity exceeds a first preset threshold as candidate recognition results, and setting the upper limit of the number of candidate recognition results to N, where N is an integer greater than or equal to 1. Screening candidate users by using a similarity threshold and limiting the number of candidates effectively reduces the computational load of subsequent comparisons, improving the real-time performance and efficiency of the recognition system. Specifically: The obtained palmprint feature vector is compared one by one with multiple registered palmprint features to determine potential user identifiers. The similarity between the palmprint feature vector and the registered palmprint features (such as the commonly used cosine similarity) is calculated. When the similarity is greater than a threshold... The threshold range is [0, 1], and the corresponding registered user identifier is used as the potential user identifier of the current user being identified.

[0040] For example, the maximum number of potential user identifiers can be set to 3. If it exceeds 3, the 3 user identifiers with the highest similarity will be selected. If the number of potential user identifiers is 0, it will return that no user was identified; otherwise, the above hand image acquisition process will be repeated.

[0041] In one example, step 103 above, based on the preliminary screening results and the comparison between the palm vein feature vector and the feature vectors of registered palm veins, can be used for final identification. This can be achieved by: calculating the palm vein similarity between each palm vein feature vector and the feature vector of the registered palm vein corresponding to each candidate identification result; and determining the candidate identification result with the highest palm vein similarity exceeding a second preset threshold as the final identified user. This secondary palm vein verification, based on the initial palmprint screening, further eliminates false matches, significantly improving the security and accuracy of the identification system, making it particularly suitable for high-security scenarios such as access control and payment. Specifically: Based on the registered palm vein feature vectors corresponding to potential user identifiers, a similarity comparison is performed one by one with the palm vein feature vectors obtained in step 205. The vector with the highest similarity and a similarity higher than a set threshold is selected. The user identifier (within the threshold range [0, 1]) is returned. If a user meets the condition, the user identifier is returned and the identification is successful. If no user meets the condition, the user is returned as not identified.

[0042] In a complete example, such as Figure 3As shown, the palmprint and palm vein dual-modal recognition device acquires training datasets of palmprint and palm vein images and performs corresponding data augmentation processing on each. The processed training images are then input into a feature extraction network for training. The palmprint feature extraction network is a quality fusion feature extraction network that uses a combined loss function, adapts to the energy loss of palmprint features, and employs a loss hierarchy constraint strategy to adaptively learn the diverse palmprint features under different quality levels, thus training a palmprint feature extraction model. Since the quality stability of palm vein images acquired by an infrared (IR) camera is better than that of palmprint images acquired by a natural light (RGB) camera, a commonly used feature extraction network is used to train a palm vein feature extraction model. Then, multiple consecutive visible light (RGB) images of the same palm are acquired, and the acquired palm images are horizontally corrected to obtain corrected palm images. The region of interest (ROI) of the palm is cropped to obtain the palmprint image. These multiple consecutive palmprint images are then input into the pre-trained quality fusion palmprint feature extraction model to obtain the palmprint feature vector. The palm vein IR image acquired at the time corresponding to the palm image with the highest palmprint weight is selected, and the region of interest is extracted in the same way. This extracted region of interest is then input into a pre-trained palm vein feature extraction model to obtain a palm vein feature vector. The palmprint feature is compared with multiple registered and stored palmprint features. User identifiers with similarity greater than a set threshold are used as potential user identifiers for the palmprint being processed. Based on the registered palm vein features corresponding to the potential user identifiers, the current palm vein feature is compared with the current palm vein feature to finally determine the current user identifier.

[0043] In this embodiment, multiple palmprint images of the same hand are quality-weightedly fused to generate a more robust fused palmprint feature vector. This process adaptively enhances the contribution of high-quality images and suppresses interference from low-quality images. Simultaneously, corresponding palm vein images are acquired and their feature vectors are extracted. A progressive recognition strategy is employed: first, the fused palmprint features are used for rapid initial screening to narrow the recognition range; then, the palm vein features, which offer stronger anti-spoofing capabilities, are combined to accurately authenticate the initial screening results. This invention effectively overcomes the quality sensitivity defect of palmprint recognition through a quality-adaptive fusion mechanism, fully leverages the complementary advantages of dual modalities, and significantly improves the accuracy, stability, and security of identity recognition in complex real-world scenarios.

[0044] The steps described above are for clarity only. In practice, they can be combined into one step or some steps can be split into multiple steps. As long as they include the same logical relationship, they are all within the scope of protection of this application. Adding insignificant modifications or introducing insignificant designs to the algorithm or process, but without changing the core design of the algorithm and process, are also within the scope of protection of this application.

[0045] It is worth mentioning that all modules involved in this embodiment are logical modules. In practical applications, a logical unit can be a physical unit, a part of a physical unit, or a combination of multiple physical units. Furthermore, to highlight the innovative aspects of this application, this embodiment does not introduce units that are not closely related to solving the technical problem proposed in this application; however, this does not mean that other units are absent from this embodiment.

[0046] Another embodiment of this application relates to an electronic device, such as... Figure 4 As shown, it includes at least one processor 501; and a memory 502 communicatively connected to the at least one processor; wherein the memory 502 stores instructions that can be executed by the at least one processor 501, the instructions being executed by the at least one processor 501 to enable the at least one processor 501 to perform the palm print and palm vein bimodal recognition method as described above.

[0047] The memory 502 and processor 501 are connected via a bus, which can include any number of interconnecting buses and bridges. The bus connects various circuits of one or more processors 501 and memory 502 together. The bus can also connect various other circuits, such as peripheral devices, voltage regulators, and power management circuits, which are well known in the art and therefore will not be described further herein. A bus interface provides an interface between the bus and the transceiver. The transceiver can be a single element or multiple elements, such as multiple receivers and transmitters, providing a unit for communicating with various other devices over a transmission medium. Data processed by processor 501 is transmitted over a wireless medium via an antenna, which further receives data and transmits it to processor 501.

[0048] Processor 501 is responsible for managing the bus and general processing, and can also provide various functions, including timing, peripheral interfaces, voltage regulation, power management, and other control functions. Memory 502 can be used to store data used by processor 501 during operation.

[0049] Those skilled in the art will understand that the above embodiments are specific implementations of this application, and in practical applications, various changes can be made in form and detail without departing from the spirit and scope of this application.

Claims

1. A method for dual-modal recognition of palm prints and palm veins, characterized in that, include: Based on multiple palmprint images of the same hand, a fused palmprint feature vector is obtained through quality-weighted fusion. Obtain the palm vein image corresponding to the palm print image, and extract the palm vein feature vector; The fused palmprint feature vector is compared with the feature vector of the registered palmprint for preliminary screening, and the final identification is performed based on the comparison results of the palm vein feature vector and the feature vector of the registered palm vein, according to the preliminary screening results.

2. The palm print and palm vein dual-modal recognition method according to claim 1, characterized in that, The fused palmprint feature vector, obtained by quality-weighted fusion of multiple palmprint images of the same hand, includes: Multiple palmprint images are processed using a quality fusion feature extraction network, which includes feature extraction branches configured in parallel and sharing parameters, and a quality estimation branch configured for each feature extraction branch. Each palmprint image is input into a different feature extraction branch to extract the feature vector of each palmprint image, and the corresponding quality estimation branch outputs the quality weight of the palmprint image. The feature vectors of the corresponding palm print images are weighted using the quality weights of the normalized palm print images, and the feature vectors of all the weighted palm print images are summed to obtain the fused palm print feature vector.

3. The palm print and palm vein dual-modal recognition method according to claim 1, characterized in that, The step of obtaining the palm vein image corresponding to the palm print image includes: Based on the quality weights of each palmprint image obtained during the quality-weighted fusion process, the palmprint image with the highest quality is selected. Acquire palm vein images taken by an infrared camera at the same acquisition time as the highest quality palm print image.

4. The palm print and palm vein dual-modal recognition method according to claim 2, characterized in that, The quality fusion feature extraction network is trained using a combined loss function, which includes at least the following: Angle-space loss function used to enhance inter-class feature discriminativeness; and An unsupervised energy minimization loss function used to constrain the redundancy of network weight parameters.

5. The palm print and palm vein dual-modal recognition method according to claim 4, characterized in that, The calculation method of the unsupervised energy minimization loss function includes: The weight parameters of the specified layer in the network are normalized. Calculate the cosine or Euclidean distance between any two weight vectors within the same layer; The energy loss of the same layer is calculated based on the cosine distance or Euclidean distance. The closer the cosine distance or Euclidean distance, the greater the energy loss, in order to penalize the redundancy of feature extraction. The total unsupervised energy loss is obtained by summing the energy losses calculated at different layers after applying different constraint coefficients.

6. The palm print and palm vein dual-modal recognition method according to claim 5, characterized in that, Applying different constraint coefficients to the energy loss calculated for different layers includes: When the quality fusion feature extraction network is a convolutional neural network, a first coefficient is applied to the energy loss of the end layer of feature extraction, and a second coefficient is applied to the energy loss of other layers, wherein the first coefficient is greater than the second coefficient; When the quality fusion feature extraction network is a Vision Transformer network, a third coefficient is applied to the energy loss of the Transformer encoder layer, and a fourth coefficient is applied to the energy loss of the classification head layer, wherein the fourth coefficient is greater than the third coefficient.

7. The palm print and palm vein dual-modal recognition method according to claim 1, characterized in that, The preliminary screening using the fused palmprint feature vector and the feature vector of a registered palmprint includes: Calculate the palmprint similarity between the fused palmprint feature vector and the feature vectors of all registered palmprints; Registered user identifiers whose palmprint similarity exceeds a first preset threshold are used as candidate recognition results, and the upper limit of the number of candidate recognition results is set to N, where N is an integer greater than or equal to 1.

8. The palm print and palm vein dual-modal recognition method according to claim 7, characterized in that, The final identification, based on the preliminary screening results and the comparison results between the palm vein feature vector and the feature vectors of registered palm veins, includes: Calculate the palm vein similarity between the palm vein feature vector and the feature vector of the registered palm vein corresponding to each candidate recognition result; The candidate recognition result with the highest palm vein similarity and exceeding the second preset threshold is determined as the final recognized user.

9. The palm print and palm vein dual-modal recognition method according to claim 1, characterized in that, The method further includes: Before obtaining the fused palmprint feature vector through quality-weighted fusion, multiple palm images of the same hand are acquired; The palm image is horizontally corrected by detecting key points on the palm. Using the length of the line connecting the key points of the palm as a reference, a fixed-size region of interest on the palm is cut out to obtain the palm print image.

10. An electronic device, characterized in that, include: At least one processor; as well as, A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, which, when executed by the at least one processor, enables the at least one processor to perform the palm print and palm vein bimodal recognition method as described in any one of claims 1 to 9.

Images