A palm vein recognition method based on domain adaptation strategy

Abstract

This invention discloses a palm vein recognition method based on a domain adaptation strategy, relating to the fields of biometrics, identity authentication, and palm recognition. The method includes: Step 1: Constructing a dataset, including a source dataset and a target dataset. All data from the source dataset and a portion of the target dataset are used as the initial training set, and the remaining data from the target dataset are used as the initial test set. ROI extraction is then performed, and the initial training set is split into a formal training set, and the initial test set is split into a formal test set. Step 2: Designing an adversarial domain adaptation network. Step 3: Pre-training on the source dataset to enable the classification module to achieve a pre-defined classification performance on the source dataset. Step 4: Performing adversarial training on the formal training set to enable the palm feature extraction module to learn domain-invariant features. Training ends when the preset number of iterations or model threshold is reached. Step 5: Testing using the formal test set.

Description

Technical Field

[0001] This invention relates to the fields of biometric information recognition, identity authentication, and palm recognition technology, and in particular to a palm vein recognition method based on a domain adaptation strategy. Background Technology

[0002] Biometric identification is a technology that uses inherent physiological characteristics of the human body for identity verification. Currently, in-depth research is being conducted both domestically and internationally on biometric identification technologies such as fingerprints, faces, palm prints, irises, and signatures. Due to its high security, anti-counterfeiting features, and convenience, palm recognition technology is widely used in various fields, such as finance, security, healthcare, and access control systems, bringing greater security and convenience to people's lives and work.

[0003] Existing deep learning-based hand recognition technologies have made some progress in their respective applications. However, these methods mainly focus on recognition on a single dataset, requiring both the training and test sets to be collected under identical conditions. In practical applications, however, datasets are often collected using different devices. Due to this dataset disparity, a model trained on one dataset will experience significantly reduced performance when tested on a different dataset.

[0004] When faced with a mismatch between the source and target datasets, retraining on the target dataset, while allowing the model to adapt to its distribution, presents several drawbacks. Firstly, collecting and labeling the target dataset requires significant investment of manpower, resources, and time, especially in biometric fields like hand recognition, where acquiring high-quality hand images and accurately labeling them is a tedious and time-consuming process. Secondly, if the target dataset has limited data, retraining may lead to overfitting and poor generalization ability.

[0005] Therefore, those skilled in the art are dedicated to developing a new palm vein recognition method to overcome the aforementioned deficiencies in the existing technology. Summary of the Invention

[0006] In view of the above-mentioned deficiencies of the prior art, the technical problem to be solved by the present invention is how to avoid the decline in recognition performance when the source dataset and target dataset of palm print / palm vein data do not match.

[0007] To achieve the above objectives, the present invention provides a palm vein recognition method based on a domain adaptation strategy, characterized in that the method includes the following steps:

[0008] Step 1: Construct a dataset, including a source dataset and a target dataset. Use all the data in the source dataset and a portion of the data in the target dataset as the initial training set, and use the remaining data in the target dataset as the initial test set. Then, perform ROI extraction to obtain image data of the palm ROI region, and divide the initial training set into a formal training set and the initial test set into a formal test set.

[0009] Step 2: Design an adversarial domain adaptation network, which includes a palm feature extraction module, a classification module, and a domain discrimination module;

[0010] Step 3: Pre-train on the source dataset. The palm feature extraction module extracts palm features from the source dataset and inputs the palm features into the classification module for classification prediction. Based on the classification loss, the backpropagation algorithm is used to update the parameters of the palm feature extraction module and the classification module. Through continuous iterative training, the classification module can achieve the preset classification performance on the source dataset.

[0011] Step 4: Perform adversarial training on the formal training set. Input the formal training set into the pre-trained palm feature extraction module to obtain the corresponding feature representation. Then, input the obtained feature representation into the domain discrimination module to determine whether the feature representation comes from the source dataset or the target dataset, and calculate the adversarial loss. Simultaneously, input the obtained feature representation into the classification module, and calculate the classification loss based on the classification prediction results in the classification module. Calculate the joint loss based on the adversarial loss and the classification loss, and then use the backpropagation algorithm to update the parameters of the palm feature extraction module. Through continuous iterative training, the palm feature extraction module gradually learns domain-invariant features. When the training reaches a preset number of iterations or a model threshold, the training ends.

[0012] Step 5: In the testing phase, the trained palm feature extraction module and the classification module are tested using the formal test set to obtain the palm ID and complete the evaluation of the model performance.

[0013] Further, step 1 includes the following sub-steps:

[0014] Step 1.1: Use two or more different hand datasets as the source dataset and the target dataset respectively. The source dataset is historical data collected by existing devices, and the target dataset is data collected by new hand collection devices. Use all data in the source dataset and a portion of the data in the target dataset as the initial training set, and use the remaining data in the target dataset as the initial test set.

[0015] Step 1.2: Perform data preprocessing on the source dataset and the target dataset to unify the data format. The data preprocessing methods include, but are not limited to, normalization, enhancement, or resizing.

[0016] Step 1.3: Add hand segmentation and key point detection tasks to the target detection model. Perform hand localization and segmentation through ROI extraction, determine the position of key feature points in the hand, segment the initial training set into the formal training set, and segment the initial test set into the formal test set.

[0017] Furthermore, the target detection model in step 1.3 is the YOLO series, Faster R-CNN, or SSD.

[0018] Furthermore, the key feature points in the palm in step 1.3 include seven points: the base of the index finger, the base of the middle finger, the base of the thumb, the base of the little finger, the center of the palm, and two key points on the left and right sides at the connection between the wrist and the palm.

[0019] Furthermore, the palm feature extraction module in step 2 includes a depthwise separable convolution module, a multi-level feature extraction module, and a sub-graph Transformer module. The depthwise separable convolution module performs preliminary feature extraction, the multi-level feature extraction module stacks multiple improved inverse residual structures and linear Transformer encoders, and the sub-graph Transformer module divides the image of the palm ROI region into multiple sub-graphs of equal size, and mines and outputs a fixed-dimensional global feature vector with the help of the Transformer model.

[0020] Furthermore, the improved inverse residual structure in the multi-level feature extraction module first expands and groups the number of channels through grouped pointwise convolution, and performs convolution operations within each group. Then, a 3×3 depthwise convolution is performed on the expanded channel dimension to extract the local spatial features of the channels. At the same time, the channel attention module performs global average pooling on the features of each channel to obtain the global information of the channels. Then, a weight coefficient is generated for each channel. Finally, the number of channels is compressed back to the dimension before expansion and grouping through the grouped pointwise convolution, realizing feature fusion and linear transformation.

[0021] Furthermore, the multi-level feature extraction module replaces the ReLU activation function with, but is not limited to, the Mish function or the Swish function. The Mish function has self-gating properties and its expression is Mish(x)=x*tanh(softplus(x)); the Swish function is expressed as Swish(x)=x*sigmoid(x).

[0022] Furthermore, the improved inverse residual structure of the multi-level feature extraction module extracts feature X, and divides the feature map of feature X into multiple overlapping small patches of equal size. Each patch is regarded as an element in a sequence. The linear Transformer encoder is then used to calculate the attention weight between each patch and all other patches, dynamically determining the importance of each patch in the global features, thereby achieving the abstraction and fusion of global features. The feature map of feature X is then unfolded into P non-overlapping flattened patches. Each flattened patch is indexed and numbered. For each flattened patch, the linear Transformer encoder is applied to encode the relationship between patches, thereby obtaining the global features between patches.

[0023] Furthermore, the classification module in step 2 includes one or more fully connected layers, followed by a softmax layer for classification, and the classification loss function is the cross-entropy loss function. The number of neurons in the fully connected layer can be adjusted according to the classification task.

[0024] Further, the domain discrimination module in step 2 includes several convolutional layers, fully connected layers, and a softmax function, with batch normalization layers and activation functions added between the fully connected layers. The domain discrimination module uses a binary classification method to distinguish whether the palm features come from the source dataset and the target dataset, and then performs feature obfuscation through gradient inversion to align the edge distributions between datasets. The adversarial loss function... :

[0025]

[0026] in, The hand image represents the source dataset. The hand images represent those from the target dataset, where S is the number of hand images in the source dataset and T is the number of hand images in the target dataset. This indicates the probability that the domain discrimination module predicts the hand image originates from the target dataset.

[0027] The palm vein recognition method based on a domain adaptation strategy provided by this invention has the following technical advantages:

[0028] 1. The technical solution provided by this invention can accurately capture the potential correlations and differences between different domains through effective learning from a small amount of labeled data. This enables the model to maintain stable and good recognition performance in different application environments and scenarios, while significantly reducing the cost and time of manual annotation. Even when faced with completely new and unseen target domain data, it can quickly adapt and make accurate recognition judgments, effectively avoiding overfitting and enhancing the model's generalization ability.

[0029] 2. The lightweight design architecture of the technical solution provided by this invention significantly reduces the computational complexity of the model, reduces reliance on hardware resources, meets the needs of real-time application scenarios, and simultaneously reduces device energy consumption and extends device battery life. In some sensitive scenarios, such as medical and financial fields, the protection of data privacy is crucial. The technical solution provided by this invention reduces reliance on large amounts of data, lowers the risk of data leakage, and also reduces privacy management costs associated with data collection and storage, providing users with a more secure and reliable palm recognition solution.

[0030] The following will further explain the concept, specific structure, and technical effects of the present invention in conjunction with the accompanying drawings, so as to fully understand the purpose, features, and effects of the present invention. Attached Figure Description

[0031] Figure 1 This is a flowchart of a preferred embodiment of the palm vein recognition method of the present invention;

[0032] Figure 2 This is a schematic diagram of key points of the palm extracted according to a preferred embodiment of the present invention;

[0033] Figure 3 This is a schematic diagram of the workflow of the palm feature extraction module according to a preferred embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of depthwise separable convolution, a preferred embodiment of the present invention.

[0035] Figure 5 This is a schematic diagram of the inverse residual structure workflow of a preferred embodiment of the present invention;

[0036] Figure 6 This is a schematic diagram of the improved inverse residual structure workflow of a preferred embodiment of the present invention. Detailed Implementation

[0037] The following description, with reference to the accompanying drawings, illustrates several preferred embodiments of the present invention to make its technical content clearer and easier to understand. The present invention can be embodied in many different forms, and the scope of protection of the present invention is not limited to the embodiments mentioned herein.

[0038] The palm recognition involved in the embodiments of the present invention includes, but is not limited to, palm print or palm vein recognition, or palm print and palm vein fusion recognition, or palm back and palm back vein recognition, etc., which use palm information for biometric recognition. It uses deep learning domain adaptation technology to solve the problem of decreased recognition performance caused by mismatch between source dataset and target dataset data for palm print / palm vein data.

[0039] However, existing technologies do not fully consider the following issues:

[0040] 1) Due to differences in equipment, usage scenarios (different background information), near-infrared light sources, and lighting conditions, the collected palm images vary in contrast, clarity, and resolution. This causes a mismatch between the source dataset during training and the target dataset during actual use. The trained model struggles to cope with these uncontrollable changes in the acquisition environment in real-world scenarios, leading to a decline in model performance.

[0041] a) Data differences caused by different acquisition devices

[0042] In practical applications of palm vein recognition, the source dataset (such as palm images collected in the training set) and the target dataset (such as palm images collected in actual use scenarios) may be acquired by different devices. Different devices vary in resolution, color reproduction, and noise levels, leading to significant differences in the distribution of pixel values, texture, and other features in the palm images. For example, high-quality cameras capture clear and accurate palm images, while actual palm vein recognition modules may capture blurry, color-biased, and noisy images due to exposure and ambient lighting effects.

[0043] b) Data differences due to different environmental conditions

[0044] The source dataset of hand images may have been captured indoors under uniform lighting and simple background conditions, while the target dataset of images may have been captured outdoors under complex lighting (such as backlighting or sidelighting) and complex backgrounds (such as crowds or buildings). This environmental difference will cause changes in the appearance features of the hand (such as shadows and contrast), leading to changes in the data distribution. For example, some areas of the hand may be in shadow under backlighting, resulting in a significant difference in feature distribution compared to hand images captured under uniform indoor lighting.

[0045] c) Scarcity of labeled data

[0046] The source dataset may have relatively rich labeled data (such as some publicly available palm datasets that have been labeled with identities). Through domain adaptation techniques, the labeled data and models of the source dataset can be used to achieve better palm recognition performance with little or no labeled data in the target dataset, thus reducing the dependence on a large amount of labeled data in the target dataset.

[0047] 2) Due to privacy constraints and data security risks, data sharing between different institutions or organizations faces legal and ethical challenges. Palm print / palm vein data is sensitive personal information, and its leakage could lead to serious privacy and security problems for individuals. Therefore, when training palm recognition models, it is essential to ensure data security and prevent data theft or misuse.

[0048] To address the aforementioned issues not fully considered in existing technologies, the technical solution provided by this invention primarily involves: constructing training and testing sets for domain adaptation by combining multiple datasets; designing an adversarial domain adaptation network, mainly comprising three deep neural network modules: a palm feature extraction module, a domain discrimination module, and a classification module. The palm feature extraction module is responsible for extracting multi-dimensional high-dimensional feature information such as texture, vein distribution, and geometric shape from palm images; the domain discrimination module engages in adversarial training with the palm feature extractor in the palm feature extraction module during model training, making it difficult for the domain discrimination module to distinguish whether the input palm image data comes from the source dataset or the target dataset; the classification module performs identity recognition and classification on the palm features output by the palm feature extractor. The source and target domain datasets are input into the palm feature extraction module to extract palm features from the source and target domains. The features from the source and target domains are discriminated using the domain discrimination module, and a classifier is used to match the palm features for palm recognition. During forward propagation, classification loss and domain adversarial loss are calculated. Based on the calculated loss values, the gradient of the loss with respect to the network parameters is calculated using the backpropagation algorithm. During backpropagation, the domain adversarial loss guides the feature extraction module to update its parameters in the direction of extracting domain-invariant features, while the classification loss prompts the classification module to accurately classify the data. Through continuous iterative updates, the network gradually adapts to the data distribution of the target domain while maintaining its classification performance in the source domain.

[0049] Example 1

[0050] like Figure 1 As shown, this embodiment of the invention provides a palm vein recognition method based on a domain adaptation strategy, including the following steps:

[0051] Step 1: Construct a dataset, including a source dataset and a target dataset. Use all data in the source dataset and a portion of the data in the target dataset as the initial training set, and use the remaining data in the target dataset as the initial test set. Then, perform ROI extraction to obtain image data of the palm ROI region, and split the initial training set into a formal training set and the initial test set into a formal test set.

[0052] Step 2: Design an adversarial domain adaptation network, which includes a palm feature extraction module, a classification module, and a domain discrimination module;

[0053] Step 3: Pre-train on the source dataset. The palm feature extraction module extracts palm features from the source dataset and inputs the palm features into the classification module for classification prediction. Based on the classification loss, the backpropagation algorithm is used to update the parameters of the palm feature extraction module and the classification module. Through continuous iterative training, the classification module can achieve the pre-set classification performance on the source dataset.

[0054] Step 4: Perform adversarial training on the formal training set. Input the formal training set into the pre-trained palm feature extraction module to obtain the corresponding feature representation. Then, input the obtained feature representation into the domain discrimination module to determine whether the feature representation comes from the source dataset or the target dataset, and calculate the adversarial loss. At the same time, input the obtained feature representation into the classification module, and calculate the classification loss based on the classification prediction results in the classification module. Calculate the joint loss based on the adversarial loss and the classification loss, and then use the backpropagation algorithm to update the parameters of the palm feature extraction module. Through continuous iterative training, the palm feature extraction module gradually learns domain-invariant features. When the training reaches the preset number of iterations or the model threshold, the training ends.

[0055] Step 5: In the testing phase, use the formal test set to test the trained palm feature extraction module and classification module to obtain the palm ID and complete the evaluation of the model performance.

[0056] Example 2

[0057] Based on Example 1, step 1 includes the following sub-steps:

[0058] Step 1.1: Use two or more different hand datasets as the source dataset and the target dataset. The source dataset consists of historical data collected by existing devices (different collection devices, different environments), containing rich hand annotation information. The target dataset consists of data collected by a newly developed hand collection device, containing a small amount of labeled data. Use all data (all images) from the source dataset (source domain) and a portion of the data (a small number of images, one image per category) from the target dataset (target domain) as the initial training set. Use the remaining data from the target dataset as the initial test set, which is not available during training and is only used to evaluate the model during testing.

[0059] Step 1.2: Perform data preprocessing on the hand images in the source and target datasets to unify the data format, that is, adjust the images to a size and format suitable for network input. The data preprocessing methods include, but are not limited to, normalization, enhancement, or resizing.

[0060] Step 1.3: Add hand segmentation and key point detection tasks to the object detection model. Use ROI extraction to locate and segment the hand, determine the location of key feature points in the hand, and split the initial training set into a formal training set and the initial test set into a formal test set.

[0061] Specifically, the object detection model in step 1.3 is YOLO (You Only Look Once) series, Faster R-CNN or SSD, etc.

[0062] Specifically, the key feature points of the palm in step 1.3 include seven points: the base of the index finger, the base of the middle finger, the base of the thumb, the base of the little finger, the center of the palm, and two key points on the left and right sides where the wrist and palm connect. Figure 2 As shown.

[0063] Example 3

[0064] Based on Example 1 or 2, the adversarial domain adaptation network in step 2 includes a palm feature extraction module, a classification module, and a domain discrimination module, such as... Figure 3 As shown.

[0065] The palm feature extraction module is a lightweight network that extracts global and local features from the palm image. It includes a depthwise separable convolution module, a multi-level feature extraction module, and a sub-graph Transformer module. The depthwise separable convolution module performs initial feature extraction. The multi-level feature extraction module stacks multiple improved inverted residual blocks and a linear Transformer encoder. The sub-graph Transformer module divides the palm ROI image into multiple equal-sized sub-graphs, using the Transformer model to mine and output a fixed-dimensional global feature vector.

[0066] like Figure 4 As shown, in the depthwise convolution module, the input image first passes through a standard convolutional layer and a depthwise convolutional layer to perform preliminary feature extraction, such as extracting simple low-level features like edges and textures. Common kernel sizes and strides are typically used, such as a 3×3 kernel with a stride of 2. This reduces computational cost while preserving some of the image's basic features.

[0067] In the multi-level feature extraction module, the specific number of layers in the stacked improved inverse residual structures and the Linear Transformer encoder was determined through ablation experiments and hardware performance. The improved inverse residual structure first expands the number of channels by replacing the original pointwise convolution with grouped pointwise convolution. Grouped pointwise convolution groups the input channels and performs convolution operations within each group, significantly reducing the number of multiplication and addition operations compared to pointwise convolution operating on all channels simultaneously. Then, 3×3 depthwise convolutions are performed on the expanded channel dimension. Depthwise convolutions primarily perform convolution operations independently on each channel to extract richer local spatial features. Simultaneously, a channel attention module, such as the Squeeze-and-Excitation (SE) module, is used to obtain global information about each channel by performing global average pooling on the features of each channel, and then a weight coefficient is generated for each channel to highlight the features of important channels and suppress information from unimportant channels, thereby improving the model's sensitivity to different features. Finally, grouped pointwise convolutions compress the number of channels back to the original dimension, simultaneously achieving feature fusion and linear transformation. This structure effectively balances computational cost and feature extraction capability while maintaining the model's lightweight nature. Dimensionality increase allows the network to learn more complex features in high-dimensional space, while depthwise convolution extracts spatial features without significantly increasing computational cost. Dimensionality reduction reduces the number of parameters and computational cost, preventing the model from becoming too complex and causing overfitting, while also making the features more compact and representative.

[0068] Specifically, the multi-level feature extraction module replaces the ReLU activation function with activation functions such as Mish or Swish. The Mish function has self-gating properties, and its expression is Mish(x)=x*tanh(softplus(x)). It can adaptively adjust the output under different input conditions, improving the non-linear expressiveness of the model while maintaining relatively low computational complexity. The Swish function is expressed as Swish(x)=x*sigmoid(x), which also has a certain degree of adaptability, enabling the model to learn features more effectively during training, especially performing better when dealing with complex data.

[0069] In particular, such as Figure 5 and Figure 6 As shown, in the multi-level feature extraction module, features at a certain level are extracted through CNN and improved inverted residual blocks. The feature maps are divided into multiple overlapping, equally sized patches (overlap rate between 0% and 50%), treating each patch as an element in a sequence. A linear Transformer encoder is used to calculate the attention weights between each patch and all other patches, dynamically determining the importance of each patch in the global features, thus achieving effective abstraction and fusion of the global features. The input feature map is unfolded into P non-overlapping flattened patches. Each patch is explicitly indexed and numbered, and arranged sequentially from left to right and top to bottom. In subsequent processing, the order of the patches can be determined based on the index, ensuring that they correspond to their spatial positions in the original feature map. For each patch, a LinearTransformer encoder is applied to encode the relationships between patches, obtaining the global features between patches.

[0070] Specifically, a separable self-attention mechanism is used to obtain the importance weight of each patch relative to other patches. The separable self-attention mechanism is an improvement on the traditional self-attention mechanism, aiming to reduce computational complexity and improve efficiency in resource-constrained environments.

[0071] The separable self-attention mechanism consists of three branches: the input branch (I), the key branch (K), and the value branch (v).

[0072] Latent token generation: First, the input features pass through an embedding layer, which transforms the input data into a vector representation, resulting in a series of input tokens. These input tokens are then fed into a latent token generation module, which can be a simple linear transformation layer or a more complex neural network structure. Its purpose is to generate a set of latent tokens, which are abstract representations of the original input tokens.

[0073] Context score calculation: Input features pass through input branch I and then through a linear layer. Map each dimension of the token to a scalar to obtain a dimensional vector, then perform a softmax operation to obtain the context score. This score reflects the correlation between each input token and the potential token; the higher the score, the stronger the correlation between the token and the potential token.

[0074] Context vector calculation: Input features are processed through key branch K (weights are...) Linear projection yields ,for Each token in the input is weighted according to its corresponding context score. Then, all weighted tokens are summed to obtain a context vector that integrates information from all input tokens. (Context vector) The calculation formula is: .

[0075] Information propagation and output: Input features pass through value branches (weights are...) Linear projection onto d-dimensional space, followed by ReLU activation, yields... .

[0076] Context vector Through broadcast element multiplication operation and To merge, soon and Multiply each element accordingly to ensure that context information is propagated to each element, thus achieving [the desired result]. The result is then reweighted. After element-wise multiplication by broadcast, the result is then passed through a linear layer. A linear transformation is performed to obtain the final output, completing the reweighting and information integration of the input tokens.

[0077] When encoding each patch using the Transformer, positional encoding information is introduced. Positional encoding incorporates the patch's location information in the original feature map into the model, enabling the model to perceive the relative position of each patch.

[0078] Folding the features encoded by the Linear Transformer back to their original spatial dimensions restores the features to a spatial structure similar to the original input feature map, resulting in... This folding process is the reverse of the previous patching and flattening operations. Based on the patch index and the pixel position information within the patch, the features are accurately restored to their original spatial positions, thus ensuring that the order of the patches and the spatial order of the pixels within the patches are preserved.

[0079] Features and original features The features are merged through a concatenation operation. Convolutional layers are then used to fuse the concatenated features. These convolutional layers perform convolution operations by sliding the kernel across the feature map, further extracting higher-level features from the fused features, achieving deeper feature fusion, and making the final features more representative and discriminative.

[0080] Specifically, the Subgraph Transformer module divides the original image into multiple subgraphs. Leveraging the powerful feature extraction capabilities of the Transformer model, it performs in-depth mining of local and global features, thereby achieving accurate understanding and analysis of the image content. The Subgraph Transformer module is used only once in the hand feature extraction module. It preserves as much of the original structural information of the hand image as possible while extracting features, and automatically learns long-range dependencies between image subgraphs, which is beneficial for subsequent feature-based analysis and processing. If the entire image is directly used as input, its high resolution and high dimensionality would lead to enormous computational demands, making model training and inference extremely complex and time-consuming. Dividing the image into smaller pieces reduces the dimensionality of each piece, decreasing the amount of data the model needs to process, thus reducing computational complexity and improving model efficiency.

[0081] The working process of the subgraph Transformer module is as follows: Figure 3 As shown, specifically:

[0082] 1) Divide the image of the palm ROI region into multiple sub-images of equal size. The size and number of sub-images can be adjusted according to actual needs. Decomposing the palm image information into multiple local information helps the Transformer model better capture the local detailed features of the image.

[0083] 2) Input the segmented sub-images into the network sequentially. Before entering the Transformer model, each sub-image needs to undergo a linear projection operation. This step transforms the pixel values ​​of each sub-image into a fixed-dimensional vector representation through a linear transformation, converting the spatial information of the image into sequential information suitable for the Transformer model to process, while reducing data dimensionality and improving the computational efficiency of the model.

[0084] 3) Similarly, a separable self-attention mechanism is used as the Transformer structure for feature extraction.

[0085] 4) The Transformer model outputs a fixed-dimensional global feature vector. This global feature vector integrates information from various local parts of the image and reflects the content and features of the entire image.

[0086] Convolutional layers are used to fuse the features extracted by the subgraph Transformer module with the first-level features from the multi-level feature extraction module. These features are then fed into the multi-level feature extraction module for further feature extraction.

[0087] The features extracted by the feature extraction module are subjected to global average pooling, compressing the feature map of each channel into a scalar to obtain a fixed-length feature vector as the final hand feature representation. By stacking multi-level feature extraction modules and combining local and global features, the model can capture more comprehensive information in the image and improve its ability to understand the image. Compared with simple CNN modules or self-attention modules, it can simultaneously utilize local details and global semantic relationships, resulting in a feature representation that integrates rich local and global features.

[0088] Example 4

[0089] Based on Examples 1, 2, or 3, the classification module in step 2 includes one or more fully connected layers, followed by a Softmax layer for classification. The number of neurons in the fully connected layers can be adjusted according to the specific classification task.

[0090] The classification loss function is the cross-entropy loss function. This is used to measure the difference between the output of the classification module and the actual classification label.

[0091] Specifically, the domain discrimination module in step 2 includes several convolutional layers, fully connected layers, and a softmax function. To enhance discrimination capability, batch normalization layers and activation functions (such as ReLU, ReLU6, etc.) are added between the fully connected layers.

[0092] The domain discrimination module uses a binary classification approach to distinguish whether the palm features come from the source dataset or the target dataset. It then uses gradient inversion to obfuscate features, thereby aligning the edge distributions between datasets. The adversarial loss function... :

[0093]

[0094] in, The hand images represent those from the source dataset. The source dataset contains hand images, where S is the number of hand images in the source dataset and T is the number of hand images in the target dataset. The domain discrimination module predicts the probability that the palm image comes from the source dataset. The domain discrimination module predicts the probability that a hand image comes from the target dataset.

[0095] During model training, an adversarial training domain discriminative loss is applied to the feature extractor. Ultimately, this renders the domain discrimination module unable to distinguish whether the data originates from the source domain or the target domain. Through this adversarial process, the features generated by the hand feature extraction module become more universal, reducing inter-domain differences.

[0096] During training, the classification loss and domain adversarial loss are weighted and summed to obtain the final loss function.

[0097] Specifically, in step 3, pre-training involves learning key palm features and classification capabilities, including parameter initialization and source domain data training. Parameter initialization specifically involves initializing the parameters of the feature extractor, classifier, and discriminator. Using a random initialization method, the model is given an initial random state, allowing it to autonomously explore optimal parameters during training. Source domain data training involves inputting source domain data into the feature extractor to obtain feature representations, which are then input into the classifier for classification prediction. Based on the classification loss, the parameters of the feature extractor and classifier are updated using the backpropagation algorithm, enabling the classifier to achieve better classification performance on the source domain data. When updating parameters, a suitable optimizer needs to be selected, such as stochastic gradient descent (SGD) or Adam; the choice of optimizer affects the speed and effectiveness of parameter updates. Through continuous iterative training, the classifier achieves better classification performance on the source domain data, i.e., the classification loss gradually decreases and the classification accuracy gradually increases.

[0098] Specifically, in the adversarial training in step 4, target domain data is introduced on the basis of pre-training, and feature extraction submodule, domain discrimination module and classification submodule are trained at the same time.

[0099] The source and target domain data are input into the pre-trained feature extraction module to obtain corresponding feature representations. These features are then input into the domain discrimination module, which determines whether a feature originates from the source or target domain. Based on the output of the domain discrimination module, an adversarial loss is calculated. With the parameters of the feature extraction and classification sub-modules fixed, the gradient is calculated using backpropagation based on the adversarial loss, and the parameters of the domain discrimination module are updated.

[0100] Simultaneously, based on the classification prediction results of the classification submodule and the true labels, the classification loss is calculated. The parameters of the fixed-domain discriminant module and the classification module are then optimized according to the adversarial loss and the classification loss, following the objective function. The total loss is calculated, and then the gradient is computed using the backpropagation algorithm to update the parameters of the feature extraction submodule. It is the classification loss of the classification module (such as cross-entropy loss). This is a hyperparameter that balances the adversarial training loss with other losses. Its value is adjusted experimentally (e.g., starting from 0.1 and trying different values) to achieve the best training effect. During training, it is dynamically adjusted... The value is gradually increased as training progresses. This allows the model to focus more on feature extraction and classification accuracy in the early stages of training, and more on domain adaptation in the later stages, thereby enhancing the model's ability to generalize to data from different domains.

[0101] Through iterative training, the feature extraction module gradually learns domain-invariant features, meaning these features have similar distributions and semantics in both the source and target domains, thus reducing the impact of domain differences on model performance. During training, a learning rate decay strategy is employed, gradually reducing the learning rate as training progresses to improve model convergence and stability.

[0102] Training End and Model Saving: Training ends when the preset number of iterations is reached or the model performance no longer shows significant improvement. The parameters of the trained feature extractor and classifier are saved for use in practical applications, such as classification of target domain data or other tasks.

[0103] The preferred embodiments of the present invention have been described in detail above. It should be understood that those skilled in the art can make numerous modifications and variations based on the concept of the present invention without creative effort. Therefore, all technical solutions that can be obtained by those skilled in the art based on the concept of the present invention through logical analysis, reasoning, or limited experimentation on the basis of existing technology should be within the scope of protection defined by the claims.

Claims

1. A palm vein recognition method based on a domain adaptation strategy, characterized in that, The method comprises the following steps: Step 1, constructing a data set, including a source data set and a target data set, using all data in the source data set and part of the data in the target data set as an initial training set, using the remaining data in the target data set as an initial test set, then performing ROI extraction to obtain image data of a palm ROI region, and dividing the initial training set into a formal training set and the initial test set into a formal test set; Step 2, designing an adversarial domain adaptation network, which comprises a palm feature extraction module, a classification module and a domain discrimination module; Step 3, pre-training on the source data set, the palm feature extraction module extracts palm features in the source data set, and inputs the palm features into the classification module for classification prediction, updates the parameters of the palm feature extraction module and the classification module using the back propagation algorithm according to the classification loss, and continuously iterates the training to make the classification module reach the pre-set classification performance on the source data set; Step 4, adversarial training on the formal training set, inputting the formal training set into the pre-trained palm feature extraction module to obtain the corresponding feature representation, inputting the obtained feature representation into the domain discrimination module to judge whether the feature representation is from the source data set or the target data set, and calculating the adversarial loss; at the same time, inputting the obtained feature representation into the classification module, calculating the classification loss according to the classification prediction result in the classification module; calculating the joint loss according to the adversarial loss and the classification loss, and then updating the parameters of the palm feature extraction module using the back propagation algorithm; through continuous iterative training, the palm feature extraction module gradually learns the domain-invariant features, and when the training reaches the pre-set iteration number or model threshold, the training is completed; Step 5, in the test phase, using the formal test set to test the trained palm feature extraction module and the classification module to obtain palm ID and complete the evaluation of the model performance. 2.The palm vein recognition method based on domain adaptation strategy according to claim 1, wherein, The step 1 comprises the following sub-steps: Step 1.1, using two or more different palm data sets as the source data set and the target data set, wherein the source data set is historical data collected by an existing device, and the target data set is data collected by a new palm collection device, using all data in the source data set and part of the data in the target data set as the initial training set, and using the remaining data in the target data set as the initial test set; Step 1.2, data preprocessing is performed on the source data set and the target data set to unify the format of the data, and the method of data preprocessing comprises but is not limited to normalization, enhancement or size adjustment; Step 1.

3. Add a palm segmentation and key point detection task in the target detection model, locate and segment the palm through the ROI extraction, determine the position of the key feature points in the palm, and divide the initial training set into the formal training set and divide the initial test set into the formal test set. 3.The palm vein recognition method based on domain adaptation strategy according to claim 2, wherein, The target detection model in step 1.3 is YOLO series, Faster R-CNN or SSD. 4.The palm vein recognition method based on domain adaptation strategy according to claim 2, wherein, The key feature points in the palm in step 1.3 include 7, which are the index finger root point, middle finger root point, thumb root point, little finger root point, palm center point, and two key points at the connection between the wrist and the palm. 5.The palm vein recognition method based on domain adaptation strategy according to claim 1, wherein, The palm feature extraction module in step 2 includes a depth separable convolution module, a multi-level feature extraction module, and a sub-graph Transformer module, wherein the depth separable convolution module performs preliminary feature extraction, the multi-level feature extraction module stacks multiple improved inverse residual structures and linear Transformer encoders, and the sub-graph Transformer module divides the image of the palm ROI region into multiple sub-graphs of equal size, and outputs a fixed-dimensional global feature vector by means of the Transformer model.

6. The palm vein recognition method based on domain adaptation strategy according to claim 5, characterized in that, The improved inverse residual structure in the multi-level feature extraction module first expands and groups the channel number by point-by-point convolution, then performs convolution operation in each group, and then performs 3*3 depth convolution on the expanded channel dimension to extract the local spatial features of the channel. At the same time, the channel attention module is combined to perform global average pooling on the features of each channel to obtain the global information of the channel, and then generate a weight coefficient for each channel. Finally, the channel number is compressed back to the dimension before expansion and grouping by the point-by-point convolution, realizing the fusion and linear transformation of the features.

7. The palm vein recognition method based on domain adaptation strategy according to claim 6, characterized in that, The multi-level feature extraction module replaces the ReLU activation function with Mish function or Swish function, wherein the Mish function has a self-gating property, and its expression is Mish(x)=x*tanh(softplus(x)); the expression of the Swish function is Swish(x)=x*sigmoid(x). 8.The palm vein recognition method based on domain adaptation strategy according to claim 6, wherein, The improved inverse residual structure of the multi-level feature extraction module extracts features X, divides the feature map of the features X into multiple overlapping small patches of equal size, regards each patch as an element in a sequence, and uses the linear Transformer encoder to calculate the attention weight between each patch and all other patches to dynamically determine the importance of each patch in the global feature, realize the abstraction and fusion of the global feature, and then expand the feature map of the features X into P non-overlapping flattened patches. Index and number each flattened patch. For each flattened patch, apply the linear Transformer encoder to encode the relationship between the patches to obtain the global features between the patches. 9.The palm vein recognition method based on domain adaptation strategy of claim 1, wherein, The classification module in step 2 includes one or more fully connected layers, followed by a Softmax layer for classification, and the classification loss function is a cross-entropy loss function wherein the number of neurons in the fully connected layer can be adjusted according to the classification task.

10. The palm vein recognition method based on domain adaptation strategy according to claim 1, characterized in that, The domain discrimination module in step 2 includes several convolutional layers, fully connected layers and a Softmax function, and batch normalization layers and activation functions are added between the fully connected layers; the domain discrimination module adopts a binary classification mode to distinguish whether the palm features come from the source data set and the target data set, and then performs feature confusion through gradient reversal, so as to align the marginal distribution between the data sets, wherein the adversarial loss function : wherein, representing that the palm image comes from the source dataset, representing that the palm image comes from the target dataset, S is the number of palm images in the source dataset, and T is the number of palm images in the target dataset, denotes the probability that the domain discrimination module predicts that the palm image comes from the source dataset, denotes the probability that the domain discrimination module predicts that the palm image comes from the target dataset.

Images