A full-hand vein biometric method and apparatus for prosthesis attack prevention

Abstract

This invention discloses a method and device for full hand vein biometric recognition to prevent spoofing attacks, relating to the fields of full hand vein recognition, biometric information recognition, and identity authentication. The method includes: Step 1, image acquisition and detection: acquiring a frontal hand image covering the palm and finger areas, ensuring the presence of a palm and a real living person in the frontal hand image; Step 2, ROI extraction: extracting the palm vein ROI and the multi-finger ROI; Step 3, feature extraction: extracting palm vein features and features of the four fingers, and generating corresponding feature hash codes; Step 4, feature matching: first matching the palm vein features, then matching the four finger features; similarity comparison is achieved by calculating the Hamming distance between the feature hash codes; Step 5, score fusion: calculating the matching scores for the palm vein features and the four finger features respectively, then performing Tanh normalization on the matching scores, and finally fusing the normalized matching scores to obtain the final recognition result.

Description

Technical Field

[0001] This invention relates to the fields of whole hand vein recognition, biometric information recognition, and identity authentication, and particularly to a whole hand vein biometric recognition method and device that prevents spoofing attacks. Background Technology

[0002] With the rapid development of information technology, people have a more urgent need for security and convenience. Biometric technology is gradually entering people's lives and work, and is widely used in important fields such as financial payments, intelligent security, and information security. Palm vein recognition is a biometric technology that uses the uniqueness of palm veins for identity verification. It requires capturing palm vein images under near-infrared light illumination using a camera, then preprocessing the images, extracting vein features using a palm vein feature extraction algorithm, and finally comparing the collected vein features with palm vein data in a database to identify the individual. Palm vein recognition technology has strong anti-interference capabilities and can achieve accurate user identification even in complex environments.

[0003] Existing palm vein recognition systems typically use a single image for feature extraction and recognition, making them vulnerable to spoofing attacks. This is especially true when attackers provide biometric sensors with forged copies of genuine user biometric samples, causing the system to mistakenly identify the attacker as a legitimate user and gain unauthorized access, leading to privacy breaches, financial losses, and security vulnerabilities. Common types of spoofing attacks include: 1) 2D forgery attack on printed palm vein images: After obtaining images of other people's palm veins, attackers print the images on paper or other materials using a high-precision printer, and then bring the printed images close to the recognition device in an attempt to deceive the system; 2) Displaying images on display devices: Using display devices such as mobile phones and tablets to play pre-recorded or synthesized palm vein videos or images to deceive the recognition system; 3) 3D forgery attack: Using 3D printing technology, a hand mold with a three-dimensional structure is created based on the acquired palm vein data, including simulating skin texture and internal vein structure. This mold is similar to a real hand in appearance and to some extent in optical properties, increasing the success rate of the attack; 4) Making prostheses with materials such as silicone: Using materials such as silicone and latex, hand prostheses are made by hand or by using molds, and structures that simulate veins are embedded in them, such as injecting liquids with specific optical properties or adding vein-like patterns.

[0004] When facing sensor-level spoofing attacks, single-modal biometric solutions, such as palm vein recognition, are relatively vulnerable. Multimodal biometric systems, which integrate multiple biometric information, significantly improve their resistance to attacks and offer better performance in terms of biometric recognition capabilities and population coverage. However, multimodal biometric systems also have two main drawbacks: first, the need to use multiple biometric sensors to collect biometric samples from different modalities significantly increases the overall system cost; second, during the authentication (or registration) process, collecting biometric samples from different modalities requires users to invest more time and effort, making the image acquisition process less user-friendly and reducing the user experience.

[0005] Therefore, those skilled in the art are dedicated to developing a new multimodal biometric method and device to prevent spoofing attacks, addressing the issues of increased costs and poor user experience caused by multiple biometric sensors in existing technologies. Summary of the Invention

[0006] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is how to design a multimodal biometric identification method, device and apparatus to prevent spoofing attacks, and solve the problems of increased cost and poor user experience caused by multiple biometric sensors in the prior art.

[0007] To achieve the above objectives, the present invention provides a full-hand vein biometric identification method to prevent spoofing attacks, the method comprising the following steps: Step 1: Image acquisition and detection. Acquire a frontal hand image covering the palm and finger areas, and ensure that the frontal hand image contains the palm and a real living body. Step 2: ROI extraction, performing extraction of palm vein ROI and multi-finger ROI; Step 3: Feature extraction. Extract the palm vein features and the features of the four fingers, and generate the corresponding feature hash codes. Step 4: Feature matching. First, the palm vein features are matched, and then the features of the four fingers are matched. During feature matching, the similarity is compared by calculating the Hamming distance between the feature hash codes. Step 5: Score fusion. Calculate the matching scores for the palm vein feature and the four finger features respectively, then perform Tanh normalization on the matching scores. Finally, fuse the normalized matching scores using a confidence-weighted sum rule to obtain the final recognition result.

[0008] Further, step 1 includes the following sub-steps: Step 1.1: Acquire the frontal hand image covering the palm and finger areas; Step 1.2: Hand detection. A neural network model is used to detect whether a hand exists in the frontal hand image acquired in Step 1.1. Step 1.3: Liveness detection. A liveness detection model is used to detect whether a real live body exists in the frontal hand image acquired in Step 1.1. If the detection result is negative, Step 1.1 to Step 1.3 are repeated until it is determined that a palm and a real live body exist in the frontal hand image.

[0009] Furthermore, the multi-finger ROI in step 2 refers to the ROI extraction performed on the remaining four fingers after excluding the thumb.

[0010] Furthermore, step 2 includes the following sub-steps: Step 2.1: Collect several vein images containing the palm and fingers, mark the ROI boundaries of the palm veins and digital veins by marking 27 key points on the vein images, and perform data augmentation; Step 2.2: Construct a two-stage ROI extraction model consisting of a Mask R-CNN network. In the first stage, the image is scanned and target regions that may contain a hand are generated. In the second stage, the target regions are classified and bounding boxes for the hand, multiple fingers, and masks are generated respectively. The ROI extraction model uses a weighted summation method to combine the keypoint regression loss, bounding box regression loss, and category classification loss into a total loss function. Step 2.3: Perform the ROI extraction. Input the frontal hand image into the ROI extraction model for prediction, detect the key points and borders of the palm and each finger in the image, and extract the palm vein ROI and the multi-finger ROI.

[0011] Furthermore, step 3 includes the following sub-steps: Step 3.1: Construct a deep hash network. The input layer of the deep hash network receives the image data of the palm vein ROI and the multi-finger ROI. The feature extraction layer uses a lightweight convolutional neural network, and a hash layer is added at the end of the feature extraction layer. The hash layer is composed of fully connected layers and can map real value outputs to binary values. The output is a binary hash code, and the output dimension is equal to the pre-set hash code length. Step 3.2: Design the loss function of the deep hash network, which is a weighted sum of quantization loss, similarity loss, classification loss and hash loss; Step 3.3: Train the deep hash network and input the image data of the palm vein ROI and the multi-finger ROI into the trained deep hash network to obtain the palm vein features, the features of the four fingers, and the feature hash codes.

[0012] Further, in step 4, the Hamming distance is used to measure the degree of difference between two strings of equal length, and it calculates the number of different characters at corresponding positions; for two hash codes x and y of length n, the Hamming distance... The calculation formula is: in, and These represent the i-th character of hash codes x and y, respectively.

[0013] Furthermore, the matching score in step 5 is obtained by taking the reciprocal of the Hamming distance; the smaller the Hamming distance, the higher the matching score. The corresponding formula is: in, This refers to the Hamming distance. It is an extremely small constant.

[0014] Furthermore, the formula for the Tanh function used in the Tanh normalization process in step 5 is as follows: Where x is the input value, is the matching score to be normalized, e is the natural constant, and the range of y is (-1, 1).

[0015] Furthermore, in step 5, a weight is assigned to each of the five matching scores for the palm vein feature and the four finger features. The confidence levels represent different modal features. The palm vein is given the highest prior confidence, followed by the middle finger, index finger, ring finger, and little finger in descending order. The final fusion score is then calculated. ,in, The matching score is the normalized result of the palm vein features. The matching score is the result of normalizing the features of the four fingers.

[0016] The present invention also provides a full hand vein biometric identification device for preventing spoofing attacks, including the full hand vein biometric identification method for preventing spoofing attacks as described in any one of the preceding claims.

[0017] The present invention provides a method and device for full hand vein biometric identification to prevent spoofing attacks, which has at least the following technical effects: 1. The technical solution provided by this invention, by acquiring full hand vein images and fusing multimodal biometric features of palm and finger veins, increases the complexity and uniqueness of biometric features, greatly improving the system's resistance to spoofing attacks. Compared with traditional single-modal palm vein recognition systems, attackers find it difficult to forge fake samples containing multiple vein features, thus effectively ensuring the system's security.

[0018] 2. The technical solution provided by this invention uses a single sensor to acquire images, reducing the investment in hardware equipment and lowering the overall system cost. At the same time, it avoids the calibration and maintenance costs associated with multiple sensors, thus improving the system's cost-effectiveness.

[0019] 3. The technical solution provided by this invention integrates the features of palm veins and finger veins, and adopts a variety of optimization techniques such as attention mechanism and reasonable loss function design, which improves the accuracy and reliability of feature extraction and recognition, and can achieve accurate recognition even in complex environments.

[0020] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0021] Figure 1 This is a flowchart of a preferred embodiment of the present invention; Figure 2 yes Figure 1 The flowchart of the image acquisition and detection section in the illustrated embodiment is shown. Figure 3 yes Figure 1 A schematic diagram of 27 key points in the ROI extraction section of the illustrated embodiment; Figure 4 yes Figure 1 A schematic diagram of the polygonal boundary of the ROI extraction portion in the illustrated embodiment. Figure 5 yes Figure 1 A schematic diagram of the deep hash network for the ROI extraction part in the illustrated embodiment. Detailed Implementation

[0022] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0023] To achieve low-cost resistance to spoofing attacks and improve user-friendliness, it is necessary to reduce the number of sensors used for image acquisition. For hand images, palm prints, palmar veins, and hand geometric features can be extracted from images acquired using a single sensor. Palmar veins and digital veins differ in structure, distribution, and characteristics. Palmar veins have a wide distribution and contain a relatively large and complex vascular network, reflecting the overall blood circulation characteristics of the hand, while digital veins are more delicate, with unique textures and branches in different parts of the fingers. Combining multiple ROI images can integrate the feature information of both, providing richer and more comprehensive biometric data. Therefore, this invention acquires frontal hand images by covering the palm and finger areas, extracts palmar and digital veins separately, and further extracts vascular biometric features from the palm and finger areas, achieving multimodal biometric recognition of palmar and digital veins using the same device. Furthermore, a liveness detection module is designed to improve the ability to prevent spoofing attacks. Devices using this method can be applied to smart locks, financial payments, access control, and attendance systems.

[0024] Example 1 like Figure 1 As shown, this embodiment of the invention provides a full-hand vein biometric identification method to prevent spoofing attacks, including the following steps: Step 1: Image acquisition and detection. Acquire a frontal hand image covering the palm and finger areas, and ensure that the frontal hand image contains the palm and a real living body; Step 2: ROI extraction, performing extraction of palm vein ROI and multi-finger ROI; Step 3: Feature extraction. Extract the palm vein features and the features of the four fingers, and generate the corresponding feature hash codes. Step 4: Feature matching. First, match the palm vein features, then match the features of the four fingers. During feature matching, the similarity is compared by calculating the Hamming distance between the feature hash codes. Step 5: Score fusion. Calculate the matching scores for palm vein features and four finger features respectively, then perform Tanh normalization on the matching scores. Finally, fuse the normalized matching scores using a confidence-weighted sum rule to obtain the final recognition result.

[0025] Example 2 like Figure 2 As shown, based on Example 1, step 1 includes the following sub-steps: Step 1.1: Acquire a frontal image of the hand covering the palm and fingers. A suitable image acquisition device, equipped with a CMOS sensor and a camera, is used to ensure clear capture of the entire hand's veins. Simultaneously, an illumination array of alternating 660nm and 940nm infrared LEDs is used to evenly illuminate the hand, reducing the impact of shadows and reflections on image quality. A dual-band bandpass filter is employed to selectively transmit light at the specific wavelengths of 660nm and 940nm, blocking other interfering wavelengths, thus ensuring that the CMOS camera receives the pure light signal of the required wavelength, improving image quality and accuracy. During acquisition, the user places their hand naturally in the acquisition area, ensuring complete coverage of the palm and fingers and correct posture. The acquisition device acquires hand images quickly and stably, ensuring image clarity and integrity. Palm vein images at 660nm and 940nm are captured alternately using a single CMOS sensor, camera, and dual-band bandpass filter. The acquired hand vein images are preprocessed using Contrast-Limited Adaptive Histogram Equalization (CLAHE) technology to enhance image contrast and make the vein patterns clearer. Step 1.2: Hand Detection. A neural network model is used to detect the presence of a hand in the frontal hand image acquired in Step 1.1. Only when the system confirms the presence of a hand in the image will the subsequent palm vein feature extraction and recognition steps be initiated, effectively saving computational resources and significantly shortening processing time. If the detected hand is not within the view range, the system will immediately provide feedback to the user, avoiding meaningless operations and improving overall efficiency. Since the posture and orientation of the hand significantly affect the recognition results, this hand detection step also helps the system determine whether the current hand placement meets the conditions for effective recognition. If not, the system can prompt the user to adjust, thereby optimizing the recognition conditions and improving recognition accuracy. First, a large number of positive samples (images containing hands) and negative samples (images without hands) are collected to ensure image diversity, including different shooting angles, lighting conditions, hand postures, etc. To detect the presence of a hand in the image, a lightweight neural network is used. The lightweight network architecture can use an improved version of the convolutional neural network (CNN), such as the MobileNet series, EfficientNet, SqueezeNet, etc. These lightweight network structures can effectively reduce computation and parameter requirements while maintaining a certain level of detection accuracy, thus improving detection efficiency. A global average pooling layer is added after the basic network architecture. This layer transforms each feature map into a single numerical value, eliminating the spatial dimension of the feature maps and reducing the data dimensionality, preparing for the subsequent fully connected layer. Next, a fully connected layer is added, mapping the features output from the global average pooling layer to a two-dimensional vector, corresponding to the "hand present" and "hand absent" categories. Finally, the softmax activation function is used to normalize the vector output from the fully connected layer, obtaining the probability distributions for the two categories, thereby determining whether a hand is present in the image. Focal loss is used to train this network. After training and evaluation, the model can be deployed in a practical environment for hand detection. Step 1.3: Liveness Detection. A liveness detection model is used to check if a real live body exists in the frontal hand image acquired in Step 1.1. If the detection result is negative, steps 1.1 to 1.3 are repeated until a palm and a real live body are confirmed in the frontal hand image. A lightweight liveness detection model is used to filter out obvious attacks before feature extraction, reducing computational overhead. Multiple images contain more feature dimensions, providing deep learning models with more data for learning and analysis. The model can extract various feature patterns from different vein images, such as the direction of blood vessels, branching morphology, and texture details. These rich feature dimensions help the model more accurately identify liveness feature patterns, thereby improving the accuracy of liveness detection. It is much more difficult for attackers to simultaneously forge the features of palm veins and multiple finger veins than to forge vein features from a single location. The vein features of each location have their uniqueness and complexity, and the feature relationships between different locations also need to be accurately simulated. Using the palm and multiple fingers for liveness detection significantly increases the technical threshold and cost of forgery, effectively reducing the risk of being attacked by fake bodies and enhancing the system's security. In practical applications, the imaging process can be affected by various factors, such as lighting conditions, hand posture, and skin surface condition. A single vein image may be more susceptible to these interferences, leading to inaccurate feature extraction. However, frontal hand images, including palm and multiple finger images, provide more redundant information. Even if one image is disturbed, the features of other images can still be used for liveness detection. For example, when the vein image of one finger is blurred due to slight hand movement, the palm vein image and vein images of other fingers can still provide effective information for detection, ensuring the reliability of the detection results. There are certain individual differences in the characteristics of palm and finger veins, and even for the same person, vein features may undergo slight changes at different times and under different physiological states. Using the palm and multiple fingers can better adapt to these individual differences and feature variations, improving the model's generalization ability to different situations. For example, for some individuals whose hand vein features are not very obvious, combining vein images from multiple locations can provide richer information, enabling the model to perform liveness detection more accurately. First, a large number of real live hand samples and prosthetic hand samples are collected. Real liveness samples can be obtained by collecting hand images of different individuals under varying environments and lighting conditions; prosthetic samples can be obtained by creating hand prostheses of various materials and types and collecting their images. The collected images are labeled, with real liveness samples marked as "1" and prosthetic samples marked as "0". Then, a deep learning liveness detection module is constructed. The main purpose of this module is to determine whether the currently collected hand sample comes from a real live person, thus preventing prosthetic attacks. The model architecture uses an improved lightweight deep learning model as the basic model for liveness detection, followed by a global average pooling layer and several fully connected layers, using the sigmoid activation function for binary classification tasks (liveness or prosthetic).Model training and optimization utilize both live and spurious samples. During training, backpropagation is used to update model parameters, while regularization methods (such as L1 and L2 regularization) are employed to prevent overfitting. By continuously adjusting the model's structure and parameters, its classification performance is improved. The loss function used for model training is the binary cross-entropy loss function. The trained model is then deployed in a real-world application, receiving frontal hand images and performing liveness detection. Based on the model's output, it determines whether a sample is from a real live subject: if the output probability is greater than 0.5, it is considered a real live subject; otherwise, it is considered a spurious subject.

[0026] Example 3 Building upon Example 1 or 2, step 2 involves extracting the palm vein ROI and the multi-finger ROI. The original hand image typically contains a large amount of irrelevant information, such as the background surrounding the hand. Palm vein ROI extraction extracts the palm image from an image containing the palm itself. Multi-finger ROI extraction refers to the process of accurately identifying and separating the specific region containing each finger from an image containing multiple fingers. This embodiment of the invention excludes the thumb and performs ROI extraction on the remaining four fingers. By extracting the palm vein ROI and multi-finger ROI, key biometric regions can be focused on, unnecessary data can be removed, significantly reducing the amount of data required for subsequent processing and effectively saving storage space and computing resources.

[0027] Step 2 includes the following sub-steps: Step 2.1: Collect several vein images containing the palm and fingers. Mark the ROI boundaries of the palm and finger veins by labeling 27 key points on the vein images, and perform data augmentation. Specifically, collect a large number of vein images containing the palm and fingers, covering different individuals, different lighting conditions, and different hand postures, ensuring the diversity and representativeness of the dataset. For each image, manually label the ROI boundaries of the palm and finger veins to form accurate labeled data. The marker points are as follows: Figure 3 As shown, mark the vertex of each finger, the two sides of the transverse crease of each finger (4 points each), the connection point between the fingers and the palm, the outer side of the base of the little finger, the connection point between the index finger and the thumb, and two palm base points, for a total of 27 key points. The shape of the palm is determined based on the connection points between the fingers and the palm, which can form a polygon or a convex hull. For example... Figure 4As shown, for each finger, the polygonal boundary of the finger can be determined using its vertices, the key points on both sides of its creases, and the connection points between the fingers and the palm. Furthermore, due to the large data requirements of deep learning models, data augmentation techniques are employed to expand the dataset. Random rotation, translation, scaling, and brightness adjustments are performed on the collected images to generate more training samples. This increases the model's robustness, enabling it to accurately extract ROIs when faced with images of varying sizes. For example, the image can be randomly rotated by a certain angle (e.g., -15° to 15°), or translated within a certain range (e.g., translating the image size by 5% horizontally and vertically), while appropriately adjusting the brightness (e.g., a brightness variation range of ±20%).

[0028] Step 2.2: Construct a two-stage ROI extraction model. In the first stage, the image is scanned and target regions that may contain a hand are generated. In the second stage, the target regions are classified and bounding boxes for the hand, multiple fingers, and masks are generated separately. The ROI extraction model uses a weighted summation method to combine the keypoint regression loss, bounding box regression loss, and category classification loss into a total loss function. Specifically, based on the powerful image feature extraction capabilities of convolutional neural networks (CNNs), a suitable CNN model architecture for the ROI extraction task is selected, such as U-Net or Mask R-CNN. Mask R-CNN adds instance segmentation functionality to object detection, enabling simultaneous detection and segmentation of different ROI regions, and is particularly effective in distinguishing between the hand and multiple finger vein ROIs. The Mask R-CNN network is improved: in the first stage, the image is scanned and target regions that may contain a hand are generated; in the second stage, the target regions are classified and bounding boxes for the hand, multiple fingers, and masks are generated separately. Original Mask R-CNN often uses ResNet as its backbone network, which is computationally expensive. Lighter backbone networks such as MobileNetV2 and ShuffleNetV2 can be used instead. The number of layers or channels in the backbone network can be appropriately reduced according to the specific task requirements. Deformed convolutions can be used instead of ordinary convolutions. Deformed convolutions can adaptively adjust the size and shape of the receptive field, better adapting to changes in the shape of the palm and fingers, thus extracting ROIs more accurately. Furthermore, prior knowledge of the relative positions of the palm and fingers can be used to improve the Mask R-CNN network, enabling more accurate localization when predicting keypoints. When the network predicts a keypoint for a finger, if a shift in the palm's position is detected, the network can adjust the predicted position of the finger keypoint accordingly based on the relative positional relationship, thereby improving the overall accuracy of keypoint detection. Utilizing relative positional relationships makes the extraction results more stable when extracting ROIs of the palm and fingers. Even with some noise or partial occlusion in the image, as long as some keypoints are detected, the approximate positions of other keypoints can be inferred based on their relative positions, thus accurately extracting the complete palm and finger ROIs. An attention mechanism is used to allow the network to automatically focus on regions where the relative positional relationship between the palm and fingers is important. When calculating the attention weights, information such as the distance and angle between the palm center keypoint and the finger keypoints is used as input features. These features are then processed through a fully connected layer to calculate the attention weights, which are then applied to the feature map to highlight features related to the relative positional relationship.

[0029] In addition, to improve the training efficiency and accuracy of the model, the model's loss function is optimized, including the keypoint regression loss function related to keypoint loss. This is used to measure the difference between the model's predicted keypoint coordinates and the actual keypoint coordinates, guiding the model to learn accurate keypoint locations. For each keypoint, the predicted coordinates are calculated. with actual coordinates The sum of squares of the differences between them is then averaged over all key points.

[0030] Where N is the number of key points.

[0031] Bounding box regression loss related to the bounding box regression loss function This is used to adjust the position and size of anchor points to make them closer to the actual target bounding box. The Smooth L1 loss function is generally used; it's a squared loss when the error is small and a linear loss when the error is large, which reduces the impact of outliers on the loss. The formula is as follows: in, It refers to the number of anchor points that have a high degree of overlap with real hand or finger targets. It is the predicted bounding box offset. It is the actual bounding box offset.

[0032] The function is defined as follows: Category classification loss related category classification loss function Each detected ROI (Region of Interest) is classified to determine its category (e.g., palm, fingers, etc.), using the multivariate cross-entropy loss function, as shown in the formula: in, It is the number of ROIs detected. It is the true label (0 or 1) of the i-th ROI belonging to the j-th class. It is the probability that the model predicts that the ROI belongs to the j-th class.

[0033] These loss functions are combined into a total loss function using a weighted summation method. .

[0034] Step 2.3: Perform ROI extraction. Input the frontal hand image into the ROI extraction model for prediction, detect key points and bounding boxes of the palm and each finger in the image, and extract the palm vein ROI and multi-finger ROI. Post-process the ROI extraction results output by the model, including palm integrity check and ROI image quality check. The palm integrity check includes analyzing the key points and bounding box information of the palm, identifying the position and shape of the fingers, and detecting the number and shape of the fingers. Normally, the palm should contain 5 fingers. If the number of detected fingers is significantly less than 5, or the shape of the fingers is abnormal (such as excessive curvature, missing parts, etc.), it indicates that the palm may be incomplete. Considering the existence of hand disabilities and fewer than five fingers, a special user information database is established in the system to store relevant information of users with hand disabilities. When a disabled user uses the system for the first time, in addition to collecting palm vein images, it is also necessary to record their disability (such as the number and location of missing fingers, etc.) and special features of the palm. Handling anomalies: If problems are found during the hand integrity check, such as incomplete hand coverage or finger obstruction, the system can issue a prompt message, requiring the user to readjust the hand position and re-capture the image. ROI image quality includes quality checks on the cropped ROI image, such as checking if the ROI region contains sufficient vein features and whether there is noise interference. For ROIs that do not meet quality requirements, the image is re-captured.

[0035] Example 4 Building upon Examples 1, 2, or 3, step 3 extracts vein features from the palm and four fingers to generate corresponding feature hash codes. Employing a deep hash network that combines the powerful feature representation capabilities of deep learning with the efficient storage and retrieval characteristics of hashing techniques, it can capture subtle differences and complex patterns in vein textures. This eliminates the need for manually designed feature extraction methods, automatically learning discriminative features from palm and multi-finger vein images. The deep hash network converts high-dimensional palm and multi-finger vein image features into binary hash codes. Compared to traditional floating-point feature vectors, binary hash codes require significantly less storage space. For example, a 128-dimensional floating-point feature vector might require 512 bytes of storage, while a binary hash code of the same length only requires 16 bytes. This is significant for managing large-scale biometric databases, significantly reducing storage costs.

[0036] Specifically, step 3 includes the following sub-steps: Step 3.1: Construct a deep hashing network. The input layer of the deep hashing network receives image data of the palm vein ROI and multiple finger ROIs. The feature extraction layer uses a lightweight convolutional neural network, and a hashing layer is added at the end of the feature extraction layer. The hashing layer consists of fully connected layers, which can map real-valued outputs to binary values, and the output is a binary hash code. The output dimension is equal to the pre-set hash code length. The input layer receives the palm vein and multiple finger vein image data after ROI extraction. The size and number of channels of the input image will be adjusted according to the specific network design and task requirements. For example, the input size of the palm vein is 128*128*1, and the input size of the finger vein is 32*128*1. The feature extraction layer can use lightweight convolutional neural networks such as the MobileNet series and ShuffleNet series as the feature extraction infrastructure of the deep hashing network. Further simplification of the network layer and channel number is achieved on the basic network architecture. Taking MobileNetV2 as an example, it adopts an inverse residual structure and depthwise separable convolution, which can extract image features well while significantly reducing the amount of computation and parameters. These lightweight structures effectively reduce network complexity and memory consumption, meeting the requirements for real-time performance and low power consumption. Appropriately reducing the number of inverse residual modules in MobileNetV2 avoids wasting computational resources due to overly complex network structures while maintaining feature extraction capabilities. Simultaneously, the number of channels is reasonably compressed; based on actual experimental results, an appropriate channel multiplier is selected, such as reducing the standard number of channels to 0.5 or 0.75 times the original, to reduce the number of parameters and computational load. Furthermore, attention mechanisms or residual connections are introduced into the network, such as global attention, local attention, channel attention, and spatial attention, to improve feature extraction performance. Attention mechanisms can dynamically adjust the weights of feature maps, thereby better focusing on key features and enhancing the network's ability to identify important information. Combining lightweight networks with attention mechanisms enables efficient feature extraction while improving model performance in complex scenarios. The feature extraction layer outputs floating-point features of palm veins or multiple finger veins. A hash layer is added to the end of the feature extraction layer network. This layer is typically composed of fully connected layers and applies a specific activation function or quantization operation to map the real-valued output to binary values, resulting in a binary hash code. The output dimension is equal to the required hash code length. For example, the hash code length can be set to 64 bits or 128 bits, with the specific length determined by actual needs and computational resources.

[0037] Step 3.2: Design the loss function of the deep hashing network, which is a weighted sum of quantization loss, similarity loss, classification loss, and hashing loss: ,in, To quantify the loss, For similarity loss, For classifying losses, This is the hash loss. Where, , , These are the weight coefficients for each loss function, which can be adjusted according to the specific task and dataset. The classification loss uses the cross-entropy loss function to supervise the network's accuracy in classifying palm vein and multiple finger vein samples. The hash loss employs hash quantization loss, similarity preservation loss, etc., to ensure that the generated hash codes accurately reflect the sample's category information while maintaining the similarity between samples in Hamming space. To improve the discreteness and stability of the hash codes, regularization terms may be added to the loss function, such as L1 or L2 regularization of the model parameters, to prevent overfitting and improve the model's generalization ability.

[0038] The quantization loss is designed to make the real-valued vector output by the network closer to the binary value. The quantization loss function is the squared error loss, which calculates the squared error between the real-valued vector and the binary code (-1 and 1). By minimizing this error, the output is made closer to binary.

[0039] in, It is the i-th element of the real-valued eigenvector output by the network. is the corresponding binary target value (-1 or 1), and n is the length of the hash code.

[0040] Classification loss is used to ensure that the network can correctly distinguish palm vein images of different individuals, improving the discriminative power of features. Cross-entropy loss is used, and the formula is as follows: in, It is the number of categories. It is the i-th element of the real tag. It is the i-th element of the probability distribution predicted by the network.

[0041] Similarity loss ensures that similar samples (palm veins or finger veins of the same person) have similar hash codes, while dissimilar samples have different hash codes. A commonly used similarity loss function is triplet loss, which optimizes the network by comparing the distances between the anchor sample, positive samples (of the same class as the anchor sample), and negative samples (of a different class than the anchor sample).

[0042] in, It is the hash code of the anchor sample. It is the hash code of a positive sample (similar to an anchor sample). It is the hash code of the negative sample (dissimilar to the anchor sample). It is the Hamming distance. It is a positive boundary value. S Step 3.3: Train the deep hashing network and input the image data of the palm vein ROI and multiple finger ROIs into the trained deep hashing network. Calculate the palm vein features, four finger features, and feature hash codes from the network output through forward propagation. Based on the output hash codes and labels, calculate the weighted sum of classification loss, quantization loss, and similarity loss as the total loss. Update the network parameters using the backpropagation algorithm to minimize the total loss. Common optimizers include stochastic gradient descent (SGD) and Adam; choose an appropriate optimizer and learning rate based on the actual situation. During training, periodically evaluate the network performance using a validation set and adjust hyperparameters such as learning rate and batch size based on the validation results to prevent overfitting. Store the extracted hash codes in a database for subsequent matching and recognition tasks. Since hash codes are binary encodings, storage and retrieval are highly efficient. In the recognition stage, after extracting the hash codes from the image to be recognized, compare them with the hash codes in the database and calculate the Hamming distance between the hash codes. The smaller the Hamming distance, the more similar the two samples are; determine whether they belong to the same person based on a set threshold.

[0043] Example 5 Building upon embodiments 1, 2, 3, or 4, in step 4, the candidate range is first rapidly narrowed down through palm feature matching, and then features from the other four fingers are introduced for refined matching. A candidate subset (e.g., Top-N) is generated using a lower matching threshold for each finger, and subsequent fingers are verified with progressively higher thresholds. If the matching degree between the palm and fingers exceeds the threshold, subsequent matching is terminated early to reduce computational complexity and improve real-time performance. During feature matching, a series of distance values ​​are obtained by calculating the Hamming distance between binary hash codes, enabling rapid similarity comparison. The smaller the Hamming distance, the more similar the two hash codes are, and the more similar the corresponding finger vein features are. The calculation of the Hamming distance involves only bitwise operations, making it much faster than traditional Euclidean distance or cosine similarity calculations.

[0044] In step 4, the Hamming distance is used to measure the degree of difference between two strings of equal length, calculating the number of different characters at corresponding positions; for two hash codes x and y of length n, the Hamming distance... The calculation formula is: in, and Let x and y represent the i-th characters of hash codes x and y, respectively. This formula shows that the Hamming distance is the sum of the number of different characters at corresponding positions in the two hash codes.

[0045] Specifically, the matching score in step 5 is obtained by taking the reciprocal of the Hamming distance; the smaller the Hamming distance, the higher the matching score. The corresponding formula is: in, This refers to the Hamming distance. It is a very small constant, such as 0.000001, to prevent the denominator from being 0. After conversion, the matching score satisfies the characteristic that the higher the score, the higher the matching degree, making it easier to use in multimodal fusion decision-making and other processes.

[0046] Specifically, the formula for the Tanh function used in the Tanh normalization process in step 5 is: Here, x is the input value, which is the matching score to be normalized; e is the natural constant, approximately equal to 2.71828; and y has a range of (-1, 1). By inputting the matching score x into the Tanh function, the score can be mapped to the appropriate numerical range of (-1, 1), facilitating subsequent fusion operations and analysis.

[0047] Specifically, in step 5, these normalized matching scores are fused using a confidence-weighted sum rule to obtain the final recognition result. By reasonably setting the weights, the feature information of palm veins and finger veins is fully utilized to improve the accuracy and reliability of recognition. A weight is assigned to each of the five matching scores for palm vein features and four finger features. The confidence levels of different modal features are represented by a weighted prior confidence score. Prior confidence scores, based on prior knowledge or experience, pre-determine a confidence level for each modal feature. Judging from prior knowledge regarding physiological structure and recognition stability, the palm vein is considered to have higher feature stability and uniqueness, and is less susceptible to external interference factors. Therefore, the palm vein is assigned the highest prior confidence score, followed by the middle finger, index finger, ring finger, and little finger in descending order. By continuously adjusting the weights, the fused score more accurately reflects the true category of the sample, thereby improving the accuracy and reliability of recognition. A threshold for the fused score is set. The final fused score is then calculated. ,in, The matching score is the normalized value of the palm vein features. The matching score is the result of normalizing the features of the four fingers.

[0048] Specifically, a threshold for the fusion score is set. When the fused score is greater than the threshold, a match is considered successful, and the individual is identified as matching the corresponding individual in the database; if the score is less than the threshold, the match fails, and the identity cannot be determined. The threshold setting requires extensive experimentation and optimization using metrics such as false acceptance rate and false rejection rate to achieve the best recognition results. For example, after multiple experiments, a threshold of 0.6 was determined. When the fusion score is greater than 0.6, a match is considered successful, accurately identifying the individual.

[0049] Example 6 This invention also provides a full hand vein biometric identification device to prevent spoofing attacks, including a full hand vein biometric identification method to prevent spoofing attacks provided in any of the preceding embodiments.

[0050] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A method for full-hand vein biometric identification to prevent spoofing attacks, characterized in that, The method includes the following steps: Step 1: Image acquisition and detection. Acquire a frontal hand image covering the palm and finger areas, and ensure that the frontal hand image contains the palm and a real living body. Step 2: ROI extraction, performing extraction of palm vein ROI and multi-finger ROI; Step 3: Feature extraction. Extract the palm vein features and the features of the four fingers, and generate the corresponding feature hash codes. Step 4: Feature matching. First, the palm vein features are matched, and then the features of the four fingers are matched. During feature matching, the similarity is compared by calculating the Hamming distance between the feature hash codes. Step 5: Score fusion. Calculate the matching scores for the palm vein feature and the four finger features respectively, then perform Tanh normalization on the matching scores. Finally, fuse the normalized matching scores using a confidence-weighted sum rule to obtain the final recognition result.

2. The full-hand vein biometric identification method for preventing spoofing attacks as described in claim 1, characterized in that, Step 1 includes the following sub-steps: Step 1.1: Acquire the frontal hand image covering the palm and finger areas; Step 1.2: Hand detection. A neural network model is used to detect whether a hand exists in the frontal hand image acquired in Step 1.

1. Step 1.3: Liveness detection. A liveness detection model is used to detect whether a real live body exists in the frontal hand image acquired in Step 1.

1. If the detection result is negative, Step 1.1 to Step 1.3 are repeated until it is determined that a palm and a real live body exist in the frontal hand image.

3. The full-hand vein biometric identification method for preventing spoofing attacks as described in claim 1, characterized in that, The multi-finger ROI in step 2 refers to the ROI extraction performed on the four fingers excluding the thumb.

4. The full-hand vein biometric identification method for preventing spoofing attacks as described in claim 1, characterized in that, Step 2 includes the following sub-steps: Step 2.1: Collect several vein images containing the palm and fingers, mark the ROI boundaries of the palm veins and digital veins by marking 27 key points on the vein images, and perform data augmentation; Step 2.2: Construct a two-stage ROI extraction model consisting of a Mask R-CNN network. In the first stage, the image is scanned and target regions that may contain a hand are generated. In the second stage, the target regions are classified and bounding boxes for the hand, multiple fingers, and masks are generated respectively. The ROI extraction model uses a weighted summation method to combine the keypoint regression loss, bounding box regression loss, and category classification loss into a total loss function. Step 2.3: Perform the ROI extraction. Input the frontal hand image into the ROI extraction model for prediction, detect the key points and borders of the palm and each finger in the image, and extract the palm vein ROI and the multi-finger ROI.

5. The full-hand vein biometric identification method for preventing spoofing attacks as described in claim 1, characterized in that, Step 3 includes the following sub-steps: Step 3.1: Construct a deep hash network. The input layer of the deep hash network receives the image data of the palm vein ROI and the multi-finger ROI. The feature extraction layer uses a lightweight convolutional neural network, and a hash layer is added at the end of the feature extraction layer. The hash layer is composed of fully connected layers and can map real value outputs to binary values. The output is a binary hash code, and the output dimension is equal to the pre-set hash code length. Step 3.2: Design the loss function of the deep hash network, which is a weighted sum of quantization loss, similarity loss, classification loss and hash loss; Step 3.3: Train the deep hash network and input the image data of the palm vein ROI and the multi-finger ROI into the trained deep hash network to obtain the palm vein features, the features of the four fingers, and the feature hash codes.

6. The full-hand vein biometric identification method for preventing spoofing attacks as described in claim 1, characterized in that, In step 4, the Hamming distance is used to measure the degree of difference between two strings of equal length, and it calculates the number of different characters at corresponding positions; for two hash codes x and y of length n, the Hamming distance... The calculation formula is: in, and These represent the i-th character of hash codes x and y, respectively.

7. The full-hand vein biometric identification method for preventing spoofing attacks as described in claim 6, characterized in that, The matching score in step 5 is obtained by taking the reciprocal of the Hamming distance. The smaller the Hamming distance, the higher the matching score. The corresponding formula is: in, This refers to the Hamming distance. It is an extremely small constant.

8. The full-hand vein biometric identification method for preventing spoofing attacks as described in claim 7, characterized in that, The formula for the Tanh function used in the Tanh normalization process in step 5 is as follows: Where x is the input value, is the matching score to be normalized, e is the natural constant, and the range of y is (-1, 1).

9. The full-hand vein biometric identification method for preventing spoofing attacks as described in claim 8, characterized in that, In step 5, a weight is assigned to each of the five matching scores for the palm vein feature and the four finger features. The confidence levels represent different modal features. The palm vein is given the highest prior confidence, followed by the middle finger, index finger, ring finger, and little finger in descending order. The final fusion score is then calculated. ,in, The matching score is the normalized result of the palm vein features. The matching score is the result of normalizing the features of the four fingers.

10. A full-hand vein biometric identification device to prevent spoofing attacks, characterized in that, The method includes the full hand vein biometric identification method for preventing spoofing as described in any one of claims 1 to 9.

Images