Palm vein feature extraction network training method based on inter-class generated data augmentation

Abstract

The present invention provides a palm vein feature extraction network training method based on inter-class generated data augmentation: S1: A palm vein image generation model is trained using a real palm vein dataset, and a pre-trained palm vein feature extraction model is trained using the real palm vein dataset. S2: A generated palm vein dataset containing samples of several new categories is obtained, and the style latent vectors corresponding to the palm vein dataset samples of different categories are saved as input to the palm vein image generation model. S3: Intra-class data augmentation transformation and data preprocessing are performed on the real palm vein dataset and the generated palm vein dataset to obtain real data and generated data, respectively. A domain-adaptive method is used to train the palm vein feature extraction model using the real data and the generated data to obtain a trained palm vein feature extraction model. This method can solve the problem of insufficient inter-class sample data for palm vein recognition and improve the generalization performance of the palm vein feature extraction network model.

Description

Technical Field

[0001] The present invention relates to the field of biometric feature recognition technology, and more particularly to a palm vein feature extraction network training method based on inter-class generated data amplification. Background Art

[0002] In today's digital age, identity authentication technology is crucial for ensuring information security, preventing unauthorized access, and protecting personal privacy. With the advancement of technology and the expansion of its application areas, traditional authentication methods, such as passwords and PIN codes, are increasingly unable to meet people's demand for high security due to their susceptibility to cracking and forgery. Therefore, biometric recognition technology, with its unique and difficult-to-replicate characteristics, has become a research hotspot in the field of identity authentication.

[0003] Palm vein recognition technology is currently an advanced biometric identification technology that uses the vein pattern inside an individual's palm for identity verification. The principle of this technology is based on a simple yet profound observation: everyone's palm vein pattern is unique. The palm vein pattern depends on the size and shape of the veins in the palm and their distribution in the palm. Palm vein recognition technology verifies personal identity by analyzing these patterns. Compared with other biometric identification technologies, palm vein recognition technology performs well in accuracy and stability. This is due to the complexity and uniqueness of the palm vein pattern, as well as its relatively stable characteristics (not easily changed by external factors such as scars or changes in skin condition). At the same time, because the palm vein pattern is located inside the body, it is not easily damaged or forged by external factors, and is difficult to be observed by outsiders. Palm vein recognition technology also has high security.

[0004] The latest palm vein recognition technology primarily utilizes deep learning networks to extract the user's palm vein features, thereby achieving accurate identity authentication. The performance of deep learning networks is highly dependent on large amounts of high-quality training data. However, in current palm vein recognition research, publicly available palm vein datasets are relatively limited in size and diversity, limiting the space for algorithm optimization and performance improvement. Furthermore, due to privacy concerns, public datasets cannot be directly used for actual product development and optimization. Furthermore, the palm vein data collection process is a time-consuming and labor-intensive task. Therefore, insufficient training data is a key issue in current palm vein recognition research, limiting its practical application and development. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings and deficiencies in the prior art and provide a palm vein feature extraction network training method based on inter-class generated data augmentation. This method can solve the problem of insufficient inter-class sample data for palm vein recognition and improve the generalization performance of the palm vein feature extraction network model.

[0006] In order to achieve the above object, the present invention is implemented by the following technical solution: a palm vein feature extraction network training method based on inter-class generated data augmentation, characterized in that it includes the following steps:

[0007] S1. A palm vein image generation model is obtained by training a real palm vein dataset, and a pre-trained palm vein sign extraction model is obtained by training a real palm vein dataset.

[0008] S2. Use the pre-trained palm vein feature extraction model to extract features from all samples in the real palm vein dataset to obtain a real feature set;

[0009] The palm vein image generation model trained in S1 is used to generate palm vein images to obtain a palm vein dataset; the palm vein dataset is subjected to feature extraction using the pre-trained palm vein feature extraction model to obtain a feature set;

[0010] Obtain several palm vein dataset samples with different categories from the real palm vein dataset, and add the features of the palm vein dataset samples with different categories to the feature set to obtain a generated palm vein dataset with several new category samples. And save the style latent vectors corresponding to the palm vein dataset samples with different categories that are input to the palm vein image generation model;

[0011] S3. Establish a palm vein feature extraction model, and perform intra-class data augmentation transformation and data preprocessing on the real palm vein dataset and the generated palm vein dataset obtained in S2 to obtain real data and generated data respectively;

[0012] A domain adaptation method is used to train the palm vein feature extraction model using both real data and generated data. The loss function of the palm vein feature extraction model includes the classification loss on the real data and generated data, as well as the domain adaptation loss. Finally, the trained palm vein feature extraction model is obtained.

[0013] In step S1, the palm vein image generation model adopts the StyleGAN2 model.

[0014] Step S2 includes the following steps:

[0015] S21, using the pre-trained palm vein feature extraction model to extract features from all samples in the real palm vein training dataset to obtain a real feature set;

[0016] S22. Repeat the following process until the target number of generated samples is reached: randomly generate an input latent vector z, pass the input latent vector z through the StyleGAN2 feature mapping network to obtain an initial style latent vector w, input the initial style latent vector w into the palm vein image generation model to generate a palm vein image, and obtain palm vein data; extract features from the palm vein data through the pre-trained palm vein feature extraction model to obtain features;

[0017] The feature is retrieved from the real feature set. If the distance between the feature and the nearest real feature is greater than a threshold, the palm vein image corresponding to the feature is considered to be a new category. The feature is added to the real feature set and the style latent vector w generated for the palm vein image is saved.

[0018] S23. According to S22, a palm vein dataset with several new category samples is obtained, and the style latent vectors w corresponding to the palm vein dataset samples of different categories and input to the palm vein image generation model are saved.

[0019] In step S3, a domain adaptation method is used to train the palm vein feature extraction model using both real data and generated data. The number of iterations, batch size, and learning rate are set. Each iteration process is:

[0020] S31, performing a series of intra-class data augmentation transformations and data preprocessing on the real palm vein dataset to obtain real data;

[0021] S32, adding random weak noise to the style latent vector w in step S23 to achieve intra-class data augmentation, and replacing the initial style latent vector w in step S22 with the noisy style latent vector as the input of the palm vein image generation model;

[0022] The palm vein dataset is subjected to a series of intra-class data augmentation transformations and data preprocessing to obtain generated data;

[0023] S33, applying the real data and the generated data to the palm vein feature extraction model to obtain real data features and generated data features, respectively, and calculating the domain adaptation loss between the two;

[0024] S34, calculating the real data classification loss and the generated data classification loss respectively through the classifier using the real data features and the generated data features;

[0025] S35. Backpropagate through domain adaptation loss, real data classification loss, and generated data classification loss, update the gradient, and obtain the trained palm vein feature extraction model.

[0026] The way to add random weak noise to the style latent vector w is to obtain noise with a distribution range of (-1, 1) and the same dimension as the style latent vector w, then multiply the noise by a scaling factor and add it to the style latent vector w.

[0027] The intra-class data augmentation transformations used in the real palm vein dataset and the generated palm vein dataset include: random perspective transformation, random rotation, random translation, scale jitter and brightness jitter.

[0028] In each training iteration, the same number of samples are sampled from both the real and generated palm vein datasets. Specifically, two batch samplers are used to obtain the current batch of data. The batch sampler can start a new round of sampling after all samples in the dataset have been sampled once, ensuring that each sample is sampled a nearly identical number of times.

[0029] Domain Adaptation Loss It is calculated using the multi-core maximum mean difference, and the calculation formula is:

[0030]

[0031] Where N is the size of the sample batch, and are the i-th samples from real data and generated data in the batch, respectively, and k is a convex combination of multiple feature kernel functions;

[0032] The classifier consists of a fully connected layer and a Dropout layer. The classifier for real data and the classifier for generated data are independent of each other. The real data classification loss and generated data classification loss The ArcFace loss function is used for calculation, and the formula is:

[0033]

[0034] Among them, s and m are preset parameters, θ j is the real data sample (x i ,y i ) feature vector or generate data samples (x i ,y i ) feature vector and the weight vector of the corresponding category j in the classifier.

[0035] The final total loss function is calculated as follows:

[0036]

[0037] The present invention is based on a palm vein feature extraction network training method using inter-class generated data augmentation, which primarily addresses the problem of insufficient inter-class sample data for palm vein recognition. The present invention utilizes a generative model to generate palm vein images of new identities. At the same time, the new categories generated using data augmentation based on style latent variables have a certain degree of intra-class diversity, and can be combined with commonly used intra-class data augmentation techniques to further enhance intra-class diversity. The feature extraction model is then trained by combining the generated dataset with the real dataset to improve the model's generalization performance. Furthermore, due to the distribution differences between the generated dataset and the real dataset, if the generated dataset and the real dataset are directly merged into one dataset for training, the generated dataset will contaminate the high-quality real dataset, resulting in a decrease in the performance of the feature extraction model. The present invention addresses this problem by using a domain-adaptive training framework.

[0038] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0039] 1. Based on the use of a generative model to generate new inter-class palm vein identity data, the present invention adopts a style latent variable noise method to achieve intra-class data augmentation of the generated samples. At the same time, combined with the commonly used intra-class data augmentation method, it further increases the intra-class diversity of the generated samples and real samples. Then, a domain adaptation method is used to train the feature extraction network on both the generated dataset and the real dataset. This provides a new solution to the problem of insufficient data in palm vein recognition technology research.

[0040] 2. The present invention adopts a method of generating training data in real time, which only requires pre-saving the style latent vectors of inter-class samples. At the same time, it adopts intra-class data augmentation based on style latent variables, which greatly reduces the space occupied by the generated dataset and increases the intra-class diversity of the generated data.

[0041] 3. The present invention can further improve the generalization performance of the palm vein feature extraction network based on the use of only intra-class data augmentation. At the same time, the present invention can be applied to any palm vein feature extraction network. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 This is the overall framework diagram of the palm vein feature extraction network training method for inter-class generated data amplification proposed in the present invention;

[0043] Figure 2 Network structure diagram of the model for palm vein image generation. DETAILED DESCRIPTION

[0044] The present invention will be described in further detail below with reference to the accompanying drawings and specific embodiments.

[0045] Example

[0046] In this embodiment, the existing palm vein dataset is divided into a training set and a test set in a certain ratio of 5:5. The training set is used as the real palm vein dataset to train the palm vein image generation model based on StyleGAN2. At the same time, the real palm vein dataset is used to train a pre-trained palm vein sign extraction model.

[0047] The palm vein feature extraction network training method based on inter-class generated data augmentation of the present invention comprises the following steps:

[0048] S1. A palm vein image generation model is obtained by training a real palm vein dataset, and a pre-trained palm vein sign extraction model is obtained by training a real palm vein dataset.

[0049] S2. Use the pre-trained palm vein feature extraction model to extract features from all samples in the real palm vein dataset to obtain a real feature set;

[0050] The palm vein image generation model trained in S1 is used to generate palm vein images to obtain a palm vein dataset; the palm vein dataset is subjected to feature extraction using a pre-trained palm vein feature extraction model to obtain a feature set;

[0051] Obtain several palm vein dataset samples with different categories from the real palm vein dataset, and add the features of the palm vein dataset samples with different categories to the feature set to obtain a generated palm vein dataset with several new category samples. And save the style latent vectors corresponding to the palm vein dataset samples with different categories that are input to the palm vein image generation model;

[0052] S3. Establish a palm vein feature extraction model. The palm vein feature extraction model of this embodiment uses MobileNetV2. At the same time, perform intra-class data augmentation transformation and data preprocessing on the real palm vein dataset and the generated palm vein dataset obtained in S2 to obtain real data and generated data respectively.

[0053] A domain adaptation method is used to train the palm vein feature extraction model using both real data and generated data. The loss function of the palm vein feature extraction model includes the classification loss on the real data and generated data, as well as the domain adaptation loss. Finally, the trained palm vein feature extraction model is obtained.

[0054] The palm vein image generation model uses the StyleGAN2 model, and its training configuration includes:

[0055] (1) The dimensions of both input latent variables and style latent variables are 512;

[0056] (2) The loss function consists of three terms: the original GAN ​​loss, the R1 regularization loss, and the path length regularization loss. Both regularization losses use the lazy regularization technique. The calculation interval of the R1 regularization term is 16 iterations, and the weight is 16×5=80; the calculation interval of the path length regularization term is 4 iterations, and the weight is 4×2=8;

[0057] (3) The Adam optimization algorithm is used, where the learning rate of the generator is 1.6×10 -3 , parameter betas=(0,0.99 0.8 ), the learning rate of the discriminator is 1.88×10 -3 , parameter betas=(0,0.99 0.94 );

[0058] (4) The image size of the real palm vein dataset is scaled to (256, 256) and the image is normalized (the mean and variance are both 127, 5);

[0059] The training was performed for 100,000 iterations, with a batch size of 32 for each iteration.

[0060] Step S2 includes the following steps:

[0061] S21, using the pre-trained palm vein feature extraction model to extract features from all samples in the real palm vein training dataset to obtain a real feature set;

[0062] S22. Repeat the following process until the target number of generated samples is reached: randomly generate an input latent vector z, pass the input latent vector z through the StyleGAN2 feature mapping network to obtain an initial style latent vector w, input the initial style latent vector w into the palm vein image generation model to generate a palm vein image, and obtain palm vein data; extract features from the palm vein data through the pre-trained palm vein feature extraction model to obtain features;

[0063] The feature is retrieved from the real feature set. If the distance between the feature and the nearest real feature is greater than a threshold, the palm vein image corresponding to the feature is considered to be a new category. The feature is added to the real feature set and the style latent vector w generated for the palm vein image is saved.

[0064] S23. According to S22, a palm vein dataset with several new category samples is obtained, and the style latent vectors w corresponding to the palm vein dataset samples of different categories and input to the palm vein image generation model are saved.

[0065] In step S3, a domain adaptation method is used to train the palm vein feature extraction model using both real data and generated data. The number of iterations, batch size, and learning rate are set. Each iteration process is:

[0066] S31, performing a series of intra-class data augmentation transformations and data preprocessing on the real palm vein dataset to obtain real data;

[0067] S32, adding random weak noise to the style latent vector w in step S23 to achieve intra-class data augmentation, and replacing the initial style latent vector w in step S22 with the noisy style latent vector as the input of the palm vein image generation model;

[0068] The palm vein dataset is subjected to a series of intra-class data augmentation transformations and data preprocessing to obtain generated data;

[0069] S33, applying the real data and the generated data to the palm vein feature extraction model to obtain real data features and generated data features, respectively, and calculating the domain adaptation loss between the two;

[0070] S34, calculating the real data classification loss and the generated data classification loss respectively through the classifier using the real data features and the generated data features;

[0071] S35. Backpropagate through domain adaptation loss, real data classification loss, and generated data classification loss, update the gradient, and obtain the trained palm vein feature extraction model.

[0072] The way to add random weak noise to the style latent vector w is to obtain noise with a distribution range of (-1, 1) and the same dimension as the style latent vector w, then multiply the noise by a scaling factor and add it to the style latent vector w.

[0073] In the above steps, the intra-class data augmentation transformations used in the real palm vein dataset and the generated palm vein dataset include: random perspective transformation, random rotation, random translation, scale jittering and brightness jittering.

[0074] In each training iteration, the same number of samples are sampled from both the real and generated palm vein datasets. Specifically, two batch samplers are used to obtain the current batch of data. The batch sampler can start a new round of sampling after all samples in the dataset have been sampled once, ensuring that each sample is sampled a nearly identical number of times.

[0075] Domain Adaptation Loss It is calculated using the multi-core maximum mean difference, and the calculation formula is:

[0076]

[0077] Where N is the size of the sample batch, and are the i-th samples from real data and generated data in the batch, respectively, and k is a convex combination of multiple feature kernel functions;

[0078] The classifier consists of a fully connected layer and a Dropout layer. The classifier for real data and the classifier for generated data are independent of each other. The real data classification loss and generated data classification loss The ArcFace loss function is used for calculation, and the formula is:

[0079]

[0080] Among them, s and m are preset parameters, θ j is the real data sample (x i ,y i ) feature vector or generate data samples (x i ,y i ) feature vector and the weight vector of the corresponding category j in the classifier.

[0081] The final total loss function is calculated as follows:

[0082]

[0083] In addition, this embodiment uses the Adam optimizer with a learning rate of 0.01 and 100,000 training iterations. After the above training process is completed, a palm vein feature extraction model with better generalization performance than only intra-class data augmentation can be obtained. This model can be used in actual palm vein recognition systems to extract feature vectors for matching and authentication.

[0084] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered as equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A palm vein feature extraction network training method based on inter-class generated data augmentation, characterized by: The following steps are involved: S1. A palm vein image generation model is obtained by training a real palm vein dataset, and a pre-trained palm vein feature extraction model is obtained by training a real palm vein dataset. S2. Use the pre-trained palm vein feature extraction model to extract features from all samples in the real palm vein dataset to obtain a real feature set; Generate palm vein images using the palm vein image generation model trained in S1 to obtain a palm vein dataset; extract features from the palm vein dataset using the pre-trained palm vein feature extraction model to obtain a feature set; Obtain several palm vein dataset samples with different categories from the real palm vein dataset, and add the features of the palm vein dataset samples with different categories to the feature set to obtain a generated palm vein dataset with several new category samples. And save the style latent vectors corresponding to the palm vein dataset samples with different categories that are input to the palm vein image generation model; S3. Establish a palm vein feature extraction model, and perform intra-class data augmentation transformation and data preprocessing on the real palm vein dataset and the generated palm vein dataset obtained in S2 to obtain real data and generated data respectively; A domain adaptation method is used to train the palm vein feature extraction model using both real data and generated data. The loss function of the palm vein feature extraction model includes the classification loss on the real data and the generated data, as well as the domain adaptation loss. Finally, the trained palm vein feature extraction model is obtained. In step S3, a domain adaptation method is used to train the palm vein feature extraction model using both real data and generated data. The number of iterations, batch size, and learning rate are set. Each iteration process is: S31, performing a series of intra-class data augmentation transformations and data preprocessing on the real palm vein dataset to obtain real data; S32, adding random weak noise to the initial style latent vector w to achieve intra-class data augmentation, and replacing the initial style latent vector w with the noisy style latent vector as the input of the palm vein image generation model; The palm vein dataset is subjected to a series of intra-class data augmentation transformations and data preprocessing to obtain generated data; S33, applying the real data and the generated data to the palm vein feature extraction model to obtain real data features and generated data features, respectively, and calculating the domain adaptation loss between the two; S34, calculating the real data classification loss and the generated data classification loss respectively through the classifier using the real data features and the generated data features; S35. Backpropagate through the domain adaptation loss, the real data classification loss, and the generated data classification loss, update the gradient, and obtain the trained palm vein feature extraction model; The method of adding random weak noise to the initial style latent vector w is to obtain noise with a distribution range of (-1, 1) and the same dimension as the initial style latent vector w, then multiply the noise by a scaling factor and add it to the initial style latent vector w.

2. The palm vein feature extraction network training method based on inter-class generated data augmentation according to claim 1, characterized in that: In step S1, the palm vein image generation model adopts the StyleGAN2 model.

3. The palm vein feature extraction network training method based on inter-class generated data augmentation according to claim 1, characterized in that: Step S2 includes the following steps: S21, using the pre-trained palm vein feature extraction model to extract features from all samples in the real palm vein training dataset to obtain a real feature set; S22. Repeat the following process until the target number of generated samples is reached: randomly generate an input latent vector z, pass the input latent vector z through the StyleGAN2 feature mapping network to obtain an initial style latent vector w, input the initial style latent vector w into the palm vein image generation model to generate a palm vein image, and obtain palm vein data; extract features from the palm vein data through the pre-trained palm vein feature extraction model to obtain features; The feature is retrieved from the real feature set. If the distance between the feature and the nearest real feature is greater than a threshold, the palm vein image corresponding to the feature is considered to be a new category. The feature is added to the real feature set and the initial style latent vector w generated for the palm vein image is saved. S23. According to S22, a palm vein dataset with several new category samples is obtained, and the initial style latent vectors w corresponding to the palm vein dataset samples of different categories and input to the palm vein image generation model are saved.

4. The palm vein feature extraction network training method based on inter-class generated data augmentation according to claim 1, characterized in that: The intra-class data augmentation transformations used in the real palm vein dataset and the generated palm vein dataset include: random perspective transformation, random rotation, random translation, scale jitter and brightness jitter.

5. The palm vein feature extraction network training method based on inter-class generated data augmentation according to claim 1, characterized in that: In each training iteration, the same number of samples are sampled from the real palm vein dataset and the generated palm vein dataset.

6. The palm vein feature extraction network training method based on inter-class generated data augmentation according to claim 1, characterized in that: Domain Adaptation Loss It is calculated using the multi-core maximum mean difference, and the calculation formula is: Where N is the size of the sample batch, and are the i-th samples from real data and generated data in the batch, respectively, and k is a convex combination of multiple feature kernel functions.

7. The palm vein feature extraction network training method based on inter-class generated data augmentation according to claim 6, characterized in that: The classifier consists of a fully connected layer and a Dropout layer. The classifier for real data and the classifier for generated data are independent of each other. The real data classification loss and generated data classification loss The ArcFace loss function is used for calculation, and the formula is: Among them, s and m are preset parameters, θ j is the real data sample (x i ,y i ) feature vector or generate data samples (x i ,y i ) The angle between the feature vector and the weight vector of the corresponding category j in the classifier; The final total loss function is calculated as follows:

Images