A palm print and palm vein recognition method based on ShuffleNetV2 neural network

Through the palm print and palm vein recognition method based on ShuffleNetV2 neural network, combined with the sliding window method and attention SE module, the problems of poor anti-counterfeiting and resource limitation of single mode recognition technology are solved, and efficient feature extraction and recognition accuracy is achieved. Suitable for resource-constrained devices.

CN118658213BActive Publication Date: 2024-11-08CHENGDU BEDIT INFORMATION TECH CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411131655.2
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-19
Publication Date
2024-11-08
Estimated Expiration
2044-08-19

AI Technical Summary

Technical Problem

Single-modal palm print or palm vein biometric recognition technology has poor anti-counterfeiting, low recognition accuracy, and is difficult to deploy in environments with limited resources, and is greatly affected by environmental noise and deception attacks.

Method used

The palm print and palm vein recognition method based on ShuffleNetV2 neural network was adopted to extract feature maps through the sliding window method, introduce attention SE module and channel shuffling operations, and combine FocalLoss loss function and h-swish activation function to build lightweight features. Extract the model, suitable for resource-constrained devices.

Benefits of technology

It improves recognition accuracy and generalization capabilities, reduces computing resource consumption, enhances the ability to capture palm image features, and is suitable for smartphones or embedded systems, providing flexible identity recognition solutions.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118658213B_ABST
    Figure CN118658213B_ABST
Patent Text Reader

Abstract

The present invention relates to the field of biometric identification technology, and in particular to a palm print and palm vein identification method based on a ShuffleNetV2 neural network. The method comprises the following steps: S1. Based on the extracted image of the region of interest, a sliding window method is used to obtain a feature map of the image as a training sample set; S2. According to the feature map of the training sample set, channel segmentation is performed, and then an attention SE module is introduced to train the ShuffleNetV2 neural network model through the Focal Loss loss function; S3. The feature map of the user registered image is input into the trained neural network model to generate a feature vector, and stored as a template feature vector; S4. An image to be identified is input, and a feature vector to be identified is generated through feature extraction of the neural network model, and compared with the template feature vector for identification. The method can reduce the consumption of computing resources while ensuring accuracy, and provide a more flexible identity identification solution.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of biometric identification, and in particular to a palm print and palm vein identification method based on a ShuffleNetV2 neural network. Background Art

[0002] Biometrics, i.e., automatic identity authentication based on biological and behavioral characteristics, has greatly facilitated people's daily lives. So far, various biometrics, such as face, fingerprint, iris, palm print, and palm vein, have been studied for identifying individuals. Among them, palm print and palm vein provide rich information about the palm epidermis and vascular structure, respectively. Both biometrics provide a non-invasive and effective method for automatic identification or verification of individuals, and have achieved satisfactory performance. However, single modality (recognition based only on palm print or palm vein features) has limitations. Palm prints are not particularly anti-counterfeiting and are easily illegally obtained and forged. At the same time, it is also difficult to perform live verification. Affected by unfavorable factors such as environment, noise, and spoofing attacks, the security of single biometric recognition technology needs to be strengthened, and the recognition accuracy needs to be improved.

[0003] Invention patent CN115188084A discloses a palm print and palm vein multimodal identity recognition system and method, which extracts hand features, uses ResNet as the main structure, introduces SE modules, constructs SE-ResNet network structure, inputs preprocessed images into SE-ResNet network structure, generates feature distribution by adding a global pooling layer, and completes the extraction of information coding; in order to obtain the correlation between channels, ReLU activation function and sigmoid gate control mechanism are combined to complete feature recalibration. Since palm print and palm vein recognition algorithms are often carried on mobile terminals and embedded devices, they have lightweight characteristics, while ResNet has a large model size and computing requirements, which may cause deployment problems and limit their use in palm print and palm vein recognition scenarios. Summary of the invention

[0004] In view of the above technical problems, the present invention provides a palm print and palm vein recognition method based on ShuffleNetV2 neural network. As a lightweight model, ShuffleNetV2 is suitable for environments with limited resources such as mobile phones or Internet of Things devices for palm print and palm vein recognition.

[0005] A palm print and palm vein recognition method based on ShuffleNetV2 neural network includes the following steps:

[0006] S1. Based on the extracted image of the region of interest, a sliding window method is used to obtain a feature map of the image as a training sample set;

[0007] S2. According to the feature map of the training sample set, channel segmentation is performed, and then the attention SE module is introduced to train the ShuffleNetV2 neural network model through the FocalLoss loss function;

[0008] S3. The feature map of the user registered image is input into the trained neural network model to generate a feature vector, and is stored as a template feature vector;

[0009] S4. Input the image to be identified, extract the features of the neural network model to generate the feature vector to be identified, and compare and identify it with the template feature vector.

[0010] The basic scale of the sliding window of the sliding window method of the present invention is 70 pixels×70 pixels, and the step length of the sliding window is 35 pixels.

[0011] In step S1 of the present invention, image quality assessment and data cleaning are performed on the feature graph of the training sample set.

[0012] Preferably, the image quality assessment is divided into three stages. When one stage of assessment is qualified, the next stage of assessment is carried out; the three stages are: manual judgment of clarity, image information entropy judgment, and standard deviation judgment of image grayscale values.

[0013] Preferably, the data cleaning includes:

[0014] S11. Use the OpenCV database to resize the images to a uniform size;

[0015] S12. Calculate the hash value of the image to determine whether the image is repeated; determine the repeated images by the hash value, manually determine whether they are repeated, and remove the repeated images;

[0016] S13. Repair the image with poor quality obtained by image quality assessment, and use the inpaint function in the OpenCV database to repair the damaged part of the image;

[0017] S14. Save the cleaned image data.

[0018] The S2 step described in the present invention forms a branch after the SE module, and the branch has a convolution structure, and the convolution structure includes 3 convolution layers; the feature map processed by the SE module is spliced ​​with the feature map extracted by the branch, and then a channel shuffling operation is performed, and finally a feature vector is output after a global pooling operation.

[0019] The present invention merges features after branching, and the model can flexibly adjust the combination of features according to task requirements. The branch structure design can perform multi-task learning, and parallel branches allow the model to process feature extraction and attention mechanisms at the same time, improving the generalization ability and accuracy of the model; because the branches are independent, the number and complexity of parameters of each branch can be adjusted as needed to adapt to data sets and task requirements of different sizes, which is flexible.

[0020] The present invention splices the feature map processed by the SE module with the feature map after convolution to integrate the feature information of different stages. In ShuffleNetV2, the main purpose of feature map splicing is to fuse feature information of different levels or sources to enhance the network's ability to represent input data. By splicing feature maps of different scales and abstract levels, the network can capture richer contextual information, thereby improving its performance in tasks such as image classification and target detection. By splicing feature maps of different levels, ShuffleNetV2 can fuse information of different scales and abstract levels, achieve information complementarity and enhancement, help the network better capture details and contextual information in the image, and improve recognition accuracy. The feature map splicing operation in ShuffleNetV2 is highly flexible, and the level and method of splicing can be adjusted according to the characteristics of specific tasks and data sets, so that ShuffleNetV2 can be applied to different application scenarios and has a wide range of applicability.

[0021] The present invention adopts channel shuffle operation to solve the side effects of grouped convolution: In ShuffleNetV2, branched convolution is introduced to reduce computational complexity. However, branched convolution may lead to poor information flow between channels, affecting model performance. Channel shuffle operation aims to solve this problem by improving the information flow across feature channels through shuffle operation. Channel shuffle operation can improve the expressiveness of features without adding additional computational cost, thereby improving the performance of the model to a certain extent.

[0022] The present invention adopts global pooling, which enables the network to extract the most significant features from the feature map and compress these features into a more compact representation. Global pooling can replace the traditional pooling layer and fully connected layer, simplify the network structure, and make the model more concise.

[0023] Preferably, the three convolutional layers are a 1×1 convolutional layer, a 3×3 depth convolutional layer and a 1×1 convolutional layer, respectively, and batch normalization and activation function h-swish operations are performed after the two 1×1 convolutional layers.

[0024] Batch normalization reduces the internal covariate shift problem during training, thereby speeding up the training of neural networks; it also reduces the gradient problem and allows the use of higher learning rates. The activation function h-swish operation introduces nonlinearity, allowing the network to learn and simulate complex function mappings. In addition, it can alleviate the gradient vanishing problem; the activation function h-swish is adaptive, allowing the network to automatically adjust the activation level according to the input value during training, which helps the network better adapt to different data distributions and characteristics.

[0025] Preferably, the construction and training of the neural network model are based on the PyTorch deep learning framework.

[0026] Further preferably, the feature maps that have not been convolved and the feature maps that have been convolved are made to have the same size in multiple dimensions such as the number of channels, height and width, and then the feature maps are concatenated.

[0027] In step S4 of the present invention, cosine similarity is used to compare and identify the feature vector to be identified with the template feature vector.

[0028] The beneficial effects of the present invention are:

[0029] 1. The present invention applies the ShuffleNetV2 neural network model to palm print and palm vein recognition. As a lightweight network architecture, ShuffleNetV2 can reduce computing resource consumption while ensuring accuracy. It can be more easily deployed on resource-constrained devices, such as smartphones or embedded systems, to provide a more flexible identity recognition solution.

[0030] 2. The recognition method of the present invention has efficient feature extraction capabilities. The sliding window method can accurately capture the image features of the region of interest (ROI), effectively reducing the amount of calculation while ensuring the integrity of key information. Through the introduction of channel segmentation and attention mechanism (i.e., SE module), the neural network can adaptively adjust the importance of different channels, thereby enhancing the focus on key features, better capturing key features in palm images, and further improving recognition accuracy.

[0031] 3. The model structure of the present invention contains branched convolutions. The branched structure allows the model to process feature extraction and attention mechanisms at the same time, improving the generalization ability and accuracy of the model, but this may lead to poor information flow between channels and affect model performance. Adding channel shuffling operations is intended to solve this problem, improve the flow of information across feature channels, and improve the expressiveness of features without increasing additional computing costs, thereby improving the performance of the model.

[0032] 4. The ShuffleNetV2 neural network itself is a lightweight network structure. It uses the h-swish activation function instead of the ReLU activation function. On the basis of the performance of the activation function, it can further improve the efficiency of calculation, so that this neural network can run in a resource-constrained environment (such as mobile devices or embedded systems). BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a network structure diagram of the ShuffleNetV2 model of the present invention. DETAILED DESCRIPTION

[0034] In order to more clearly and in detail illustrate the technical solution of the present invention, the present invention is further described below through relevant embodiments. The following embodiments are only for specific explanation of the implementation method of the present invention, and do not limit the protection scope of the present invention.

[0035] Example 1

[0036] A palm print and palm vein recognition method based on ShuffleNetV2 neural network includes the following steps:

[0037] S1. Based on the extracted image of the region of interest, a sliding window method is used to obtain a feature map of the image as a training sample set;

[0038] S2. According to the feature map of the training sample set, channel segmentation is performed, and then the attention SE module is introduced to train the ShuffleNetV2 neural network model through the FocalLoss loss function;

[0039] S3. The feature map of the user registered image is input into the trained neural network model to generate a feature vector, and is stored as a template feature vector;

[0040] S4. Input the image to be identified, extract the features of the neural network model to generate the feature vector to be identified, and compare and identify it with the template feature vector.

[0041] Example 2

[0042] This embodiment is based on embodiment 1:

[0043] The basic scale of the sliding window of the sliding window method is 70 pixels×70 pixels, and the step length of the sliding window is 35 pixels.

[0044] In the step S1, image quality assessment and data cleaning are performed on the feature maps of the training sample set.

[0045] The image quality assessment is divided into three stages. When one stage of assessment is qualified, the next stage of assessment will be carried out. The three stages are: manual judgment of clarity, image information entropy judgment, and standard deviation judgment of image grayscale values.

[0046] The first stage: manual judgment of clarity. When the human eye can clearly see that the palm print image and palm vein image are blurred, it means that the clarity is unqualified; if the clarity is qualified, the next step of quality assessment is carried out.

[0047] The second stage: Calculate the information entropy of the image. For palm print and palm vein images, high information entropy means that the image contains rich structural details, which is beneficial for feature extraction. Here, if the information entropy value is greater than 6.5 bits, the image clarity is considered qualified.

[0048] Calculate the grayscale histogram: First, you need to get the grayscale histogram of the image, which shows the frequency or probability of each grayscale level in the image. Let p i represents the probability of gray level i appearing, and all p i The sum of is 1.

[0049] Calculate information entropy: Use H to represent information entropy, then the calculation formula for information entropy is:

[0050]

[0051] N is the number of gray levels (e.g., in an 8-bit grayscale image, N=256). b is typically 2.

[0052] When the probability of a gray level i appearing is 0, it is stipulated that The value of is 0 to avoid undefined conditions.

[0053] The third stage: calculate the standard deviation of the grayscale value of the image. The larger the standard deviation of the grayscale value, the more dispersed the grayscale value distribution of the image, that is, the higher the contrast of the image.

[0054] Here, if the standard deviation of the palm print image is greater than 50 and the standard deviation of the palm vein image is greater than 60, the image contrast is considered to be qualified.

[0055] The data cleaning includes:

[0056] S11. Use the cv2.resize() function of the OpenCV database to resize the image to a uniform size;

[0057] S12. Calculate the hash value of the image to determine whether the image is repeated; determine the repeated images by the hash value, manually determine whether they are repeated, and remove the repeated images; calculate the Hamming distance between the hash values.

[0058] Here, when the Hamming distance between the hash values ​​of two images is less than 5, the two images are considered to be duplicates.

[0059] S13. Repair the image with poor quality obtained by image quality assessment, and use the inpaint function in the OpenCV database to repair the damaged part of the image;

[0060] S14. Save the cleaned image data.

[0061] Example 3

[0062] This embodiment is based on embodiment 1:

[0063] The S2 step, the construction and training of the neural network model are based on the PyTorch deep learning framework.

[0064] According to the feature map of the training sample set, channel segmentation is performed, and then the attention SE module is introduced. A branch is formed after the SE module, and the branch has a convolution structure, which includes 3 convolution layers; the feature map processed by the SE module is spliced ​​with the feature map extracted by the branch, and then the channel shuffle operation is performed, and finally the feature vector is output after the global pooling operation.

[0065] The three convolutional layers are a 1×1 convolutional layer, a 3×3 depth convolutional layer and a 1×1 convolutional layer, respectively. Batch normalization and activation function h-swish operations are performed after the two 1×1 convolutional layers.

[0066] In the depthwise convolution, the step size is set to 1 and appropriate padding is performed so that the values ​​in other dimensions remain unchanged and the features without convolution are the same. Figure 1 To.

[0067] Padding refers to adding one or more layers of zero-valued pixels around the borders of the input feature map.

[0068] Example 4

[0069] This embodiment is based on Embodiment 3:

[0070] The unconvolved feature map and the convolved feature map are made to have the same size in multiple dimensions such as the number of channels, height, and width, and then the feature maps are concatenated.

[0071] Specific steps:

[0072] (1) Determine the input for splicing

[0073] The concatenated input includes the feature map that has not been convolved (denoted as Feature_A) and the feature map that has been convolved (denoted as Feature_B).

[0074] (2) Channel dimension alignment

[0075] Number of channels: 1024

[0076] Dimension matching: Before concatenation, ensure that all feature maps Feature_A and Feature_B to be concatenated are consistent in multiple dimensions, including channel, height, and width. Call functions in PyTorch, including resizing, cropping, padding, etc., to make the dimensions consistent in non-channel dimensions.

[0077] (3) Splicing

[0078] Concatenation axis: Concatenate along dimension 1. In PyTorch, specify dim=1 when calling torch.cat().

[0079] Example 5

[0080] This embodiment is based on embodiment 1:

[0081] In the step S4, cosine similarity is used to compare and identify the feature vector to be identified with the template feature vector.

[0082] The above-mentioned embodiments only express the specific implementation of the present invention, and the description thereof is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. It should be pointed out that, for ordinary technicians in this field, several variations and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A palm print and palm vein recognition method based on ShuffleNetV2 neural network, characterized in that: The following steps are involved: S1. Based on the extracted image of the region of interest, a sliding window method is used to obtain a feature map of the image as a training sample set; S2. According to the feature map of the training sample set, channel segmentation is performed, and then the attention SE module is introduced to train the ShuffleNetV2 neural network model through the Focal Loss function; S3. The feature map of the user registered image is input into the trained neural network model to generate a feature vector, and is stored as a template feature vector; S4. Input the image to be identified, extract the features of the neural network model to generate a feature vector to be identified, and compare and identify it with the template feature vector; In the S2 step, a branch is formed after the SE module, and the branch has a convolution structure, and the convolution structure includes 3 convolution layers; the feature map processed by the SE module is spliced ​​with the feature map extracted by the branch, and then a channel shuffling operation is performed, and finally a feature vector is output after a global pooling operation.

2. The palm print and palm vein recognition method based on ShuffleNetV2 neural network according to claim 1 is characterized in that: The basic scale of the sliding window of the sliding window method is 70 pixels×70 pixels, and the step length of the sliding window is 35 pixels.

3. The palm print and palm vein recognition method based on ShuffleNetV2 neural network according to claim 1 is characterized in that: In the step S1, image quality assessment and data cleaning are performed on the feature maps of the training sample set.

4. The palm print and palm vein recognition method based on ShuffleNetV2 neural network according to claim 3 is characterized in that: The image quality assessment is divided into three stages. When one stage of assessment is qualified, the next stage of assessment will be carried out; The three stages are: manual judgment of clarity, image information entropy judgment and image gray value standard deviation judgment.

5. The palm print and palm vein recognition method based on ShuffleNetV2 neural network according to claim 3 is characterized in that: The data cleaning includes: S11. Use the OpenCV database to resize the image to a uniform size; S12. Calculate the hash value of the image to determine whether the image is repeated; determine the repeated images by the hash value, manually determine whether they are repeated, and remove the repeated images; S13. Repair the image with poor quality obtained by image quality assessment, and use the inpaint function in the OpenCV database to repair the damaged part of the image; S14. Save the cleaned image data.

6. According to claim 1, the palm print and palm vein recognition method based on ShuffleNetV2 neural network is characterized in that: The three convolutional layers are a 1×1 convolutional layer, a 3×3 depth convolutional layer and a 1×1 convolutional layer, respectively. Batch normalization and activation function h-swish operations are performed after the two 1×1 convolutional layers.

7. The palm print and palm vein recognition method based on ShuffleNetV2 neural network according to claim 1 is characterized in that: The construction and training of the neural network model are based on the PyTorch deep learning framework.

8. The palm print and palm vein recognition method based on ShuffleNetV2 neural network according to claim 7 is characterized in that: The unconvolved feature map and the convolved feature map are made to have the same size in multiple dimensions such as the number of channels, height, and width, and then the feature maps are concatenated.

9. The palm print and palm vein recognition method based on ShuffleNetV2 neural network according to claim 1, characterized in that: In the step S4, cosine similarity is used to compare and identify the feature vector to be identified with the template feature vector.