Feature fusion palm print and palm vein matching method based on deep hash network

Through the feature fusion method based on deep hash network, the palm print and palm vein images are mapped into binary hash values, and the approximate hash matching algorithm is used to solve the problems of low recognition accuracy and high computational complexity in the existing technology. , achieving efficient and robust palm print and palm vein recognition.

CN118918616AInactive Publication Date: 2024-11-08BEIJING INST OF TECH
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202410992962.3
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-23
Publication Date
2024-11-08
Estimated Expiration
Not applicable · inactive patent

AI Technical Summary

Technical Problem

The existing palm print and palm vein recognition methods have low recognition accuracy when processing image quality and low contrast, are susceptible to noise and light changes, have high computational complexity, and may have category imbalances in large-scale data sets. This results in a long recognition time and is difficult to adapt to real-time application scenarios.

Method used

Using a feature fusion method based on deep hash network, the palm print and palm vein images are mapped into binary hash values, and matching tests are performed through an approximate hash matching algorithm. The MPSNet model is used to extract features and construct them through a hash tree, reducing the computational complexity. and storage space.

Benefits of technology

It improves recognition accuracy, enhances robustness, reduces computing complexity and storage space, and is suitable for real-time recognition applications in complex environments.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118918616A_ABST
    Figure CN118918616A_ABST
Patent Text Reader

Abstract

The invention discloses a feature fusion palm print and palm vein detection system based on a deep hash network. The system has a training mode and a matching test mode. The detection system comprises a data preprocessing module, a pre-training module, a deep hash network and an approximate hash matching module. The data preprocessing module is connected with the pre-training module; the data preprocessing module and the pre-training module are connected with the deep hash network, and the deep hash network is connected with the approximate hash matching module; and constructing a feature extraction backbone network of the deep Hash network based on an MPSNet model. The invention further provides a feature fusion palm print and palm vein matching method based on the deep Hash network. Training and matching testing are carried out based on the palm print and palm vein images of a plurality of given subjects, and a search structure is constructed. According to the feature fusion palm print and palm vein detection system based on the deep Hash network, the matching method has the advantages of being high in recognition precision, small in storage space, high in calculation speed and good in robustness in a complex environment.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention belongs to the technical field of image processing and pattern recognition, and relates to a feature fusion palmprint and palm vein matching method based on a deep hash network. Background Art

[0002] Currently, methods for identifying palm prints and palm veins include encoding-based methods, principal component analysis methods, and deep learning methods.

[0003] However, existing palm print and palm vein recognition methods have some defects, which limit their effectiveness and performance in practical applications. These include but are not limited to the following points: First, some existing methods have low recognition accuracy when processing images with poor quality or low contrast, and are easily affected by noise and lighting changes. Second, some methods may have class imbalance when processing large-scale data sets, resulting in poor recognition of minority categories by the model. In addition, some methods have high computational complexity, resulting in a long recognition time in practical applications, which is not conducive to the application of real-time recognition scenarios.

[0004] Therefore, the present application is committed to improving or solving the above-mentioned problems, aiming to improve recognition accuracy, reduce computational complexity, and solve the problem of category imbalance, so as to be more suitable for various practical application scenarios. Summary of the invention

[0005] The purpose of the present invention is to address the situation of large-scale palm print and palm vein classification. The existing one-hot vector as output classification has the defects of low accuracy, high complexity, unsuitability for the field and failure to be recognized in test. A feature fusion palm print and palm vein matching method based on a deep hash network is proposed. The method performs feature fusion based on a deep hash network, maps palm print and palm vein images into binary hash values, makes the Hamming distance between hash values ​​of different types larger, and the Hamming distance between hash values ​​of the same type smaller, thereby judging whether two groups of data are of the same type, and finally performs matching test through an approximate hash matching algorithm.

[0006] In order to achieve the above object, the present invention adopts the following technical scheme:

[0007] In the first aspect, the present invention proposes a feature fusion palm print and palm vein detection system based on a deep hash network, which has a training mode and a matching test mode; the detection system includes a data preprocessing module, a pretraining module, a deep hash network and an approximate hash matching module; the data preprocessing module is connected to the pretraining module; the preprocessing module and the pretraining module are connected to the deep hash network, and the deep hash network is connected to the approximate hash matching module; the deep hash network is constructed by a feature extraction backbone network based on the MPSNet model; the data preprocessing module is used to preprocess the data input into the deep hash network to obtain preprocessed data, and the preprocessed data is respectively input into the pretraining module and the deep hash network; the data pretraining module receives the preprocessed data , feature extraction is linked to the classifier with the same number of categories, with one-hot encoding as output, cross entropy as loss function, and Adam optimizer for pre-training, and the feature extraction submodule parameters are saved; the deep hash network in training mode includes feature extraction submodule, classifier submodule and loss function submodule; the deep hash network in matching training mode does not calculate the loss function, but only extracts features from the image through the feature extraction submodule after preprocessing, and then maps it to a hash value through the classifier submodule; approximate hash matching module, in training mode, after the deep hash network training is completed, the hash value of each input class is constructed into a hash tree through the hash tree construction submodule; in matching test mode, the hash value matching submodule matches the hash value converted from the preprocessed image.

[0008] The deep hash network includes a data preprocessing module for a training mode and a matching test mode; the data preprocessing module for the training mode includes a ROI extraction submodule, a data enhancement submodule, and an image enhancement submodule; the data preprocessing module for the matching test mode includes a ROI extraction submodule and an image enhancement submodule.

[0009] In the training mode, the preprocessing module receives the original palm print and palm vein images, and successively performs ROI extraction, data enhancement and image enhancement, and then inputs the enhanced image into the deep hash network; in the matching test mode, the preprocessing module receives the original palm print and palm vein images, and successively performs ROI extraction and image enhancement, and then inputs the enhanced image into the deep hash network.

[0010] The feature extraction submodule is the feature extraction backbone network of MPSNet; the classifier submodule includes a fully connected layer, an activation submodule and a Tanh function submodule; the fully connected layer and the activation submodule are connected to the Tanh function submodule.

[0011] In the training mode, the deep hash network receives palm print and palm vein images, which are processed separately by two feature extraction submodules to generate fused features, and then the fused features are input into the classifier submodule; the classifier submodule maps the fused features into hash values ​​and inputs the hash values ​​into the loss function submodule to back-propagate the gradients and update the parameters of the feature extraction submodule and the classifier submodule; in the matching test mode, the loss function is not calculated, and the image is only pre-processed and then extracted features by the feature extraction submodule, and then mapped into hash values ​​by the classifier submodule.

[0012] The approximate hash matching module includes a hash tree construction submodule and a hash value matching submodule, which respectively work in training mode and matching test mode; in the training mode, after the deep hash network training is completed, the hash value of each input class is constructed into a hash tree through the hash tree construction submodule; in the matching test mode, the hash value converted from the above-mentioned image is input into the hash value matching submodule for matching.

[0013] In a second aspect, the present invention provides a feature fusion palm print and palm vein matching method based on a deep hash network, which performs training, matching test and constructs a search structure based on a given number of subject palm print and palm vein images; the training comprises the following steps:

[0014] S10, preprocessing the palm print and palm vein training set, obtaining training data input to the deep hash network module, and obtaining preprocessed data;

[0015] S11. For the preprocessed data, perform feature extraction, link the classifier with the same number of categories, use one-hot encoding as output, cross entropy as loss function, and Adam optimizer for pre-training, and save the feature extraction submodule parameters;

[0016] S12. For a given palm print and palm vein training set, train a deep hash network to calculate the hash value of each sample, average the hash values ​​of samples of the same type to find a representative hash value, and then record the representative hash values ​​of all classes;

[0017] S13. Based on the representative hash values ​​of all classes, a hash tree is constructed by a hash tree construction method in approximate hash value matching.

[0018] The representative hash value mentioned in S13 refers to: finding the average of the hash values ​​of the same type of samples and finding the hash value with the smallest sum of Hamming distances to all the same type of hash values.

[0019] The matching and constructing of the search structure comprises the following steps:

[0020] S20, the data preprocessing module receives the original palm print and palm vein images, extracts the ROI through the ROI extraction module, and enhances the image through the image enhancement module, to obtain a preprocessed image; and then inputs the preprocessed image into the deep hash network;

[0021] S21, based on the deep hash network, extract features from the preprocessed image through the feature extraction submodule in turn, then connect the classifier of the deep hash network, and map it into a hash value through the classifier;

[0022] S22. Input the hash value into an approximate hash value matching module for matching to obtain a matching result.

[0023] The preprocessed image is subjected to feature extraction submodule to extract features, specifically: the received palm print and palm vein images are processed into features respectively through two feature extraction submodules and then fused; the classifier is mapped into a hash value, specifically: the classifier in the classifier submodule maps the features fused by the feature extraction submodule into a hash value.

[0024] Compared with the prior art, the beneficial effects of the present invention are:

[0025] 1. The method has the advantages of high recognition accuracy and good robustness in complex environments;

[0026] 2. The method is based on a deep hash network module, which has the advantages of small storage space and fast calculation speed;

[0027] 3. The method is based on the proposed approximate hash value matching method and has the advantage of low complexity in matching. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of the present invention. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0029] Figure 1 is a workflow diagram of a method for training a deep hashing network and building a hash tree;

[0030] Figure 2 It is a workflow diagram for matching a target palm print with a palm vein image;

[0031] Figure 3 Schematic diagram of ROI extraction, a) is the image binarized by Otsu algorithm, b) is the ROI boundary after contour approximation, c) is finger gap recognition and rotation angle determination, d) is the ROI after regularization after rotation frame selection;

[0032] Figure 4Schematic diagram of data enhancement, a) is cylindrical projection transformation, b) is perspective transformation;

[0033] Figure 5 Schematic diagram of image enhancement, a) is the histogram equalization enhancement method, b) is the enhancement method based on the Jerman filter;

[0034] Figure 6 It is a structural diagram of the feature fusion deep hashing network;

[0035] Figure 7 It is the structural diagram of the feature extraction backbone network;

[0036] Figure 8 This is an introduction diagram of the approximate hash value matching method;

[0037] Fig. 9 This is a graph showing the time consumption of building a hash tree with tens of millions of classes.

[0038] Fig.10 This is a graph showing the time consumption for hash value matching of tens of millions of classes. DETAILED DESCRIPTION

[0039] In order to clearly describe the technical solutions of the embodiments of the present invention, in the embodiments of the present invention, words such as "first" and "second" are used to distinguish the same items or similar items with basically the same functions and effects. For example, the first threshold and the second threshold are only used to distinguish different thresholds, and their order is not limited. Those skilled in the art can understand that words such as "first" and "second" do not limit the quantity and execution order, and words such as "first" and "second" do not necessarily limit them to be different.

[0040] It should be noted that, in the present invention, words such as "exemplary" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "exemplary" or "for example" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplary" or "for example" is intended to present related concepts in a specific way.

[0041] In the present invention, "at least one" means one or more, and "plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship. The following at least one item (items) or similar expressions thereof refer to any combination of these items, including any combination of single items (items) or plural items (items). For example, at least one item (items) of a, b or c can mean: a, b, c, the combination of a and b, the combination of a and c, the combination of b and c, or the combination of a, b and c, where a, b, c can be single or multiple.

[0042] Example 1

[0043] The feature fusion palm print and palm vein detection method includes data preprocessing, deep hash network training and testing, and approximate hash matching;

[0044] The data preprocessing, deep hash network training and testing, and approximate hash matching all have corresponding operations according to the different training modes and matching test modes;

[0045] The feature fusion palm print and palm vein detection method is based on the feature fusion palm print and palm vein detection system, including a data preprocessing module, a pre-training module, a deep hash network and an approximate hash matching module;

[0046] The deep hashing network is constructed by a feature extraction backbone network based on the MPSNet model;

[0047] The preprocessing module is used to preprocess the data input into the deep hash network to obtain preprocessed data, and the preprocessed data is respectively input into the pretraining module and the deep hash network; the preprocessing is different according to the training mode and the matching test mode of the deep hash network;

[0048] Supplement the input and output, connection relationship and function of the pre-training module;

[0049] Preprocessing of deep hashing network model training, including ROI extraction submodule, data enhancement submodule, and image enhancement submodule;

[0050] Data preprocessing module, including ROI extraction module, data enhancement module and image enhancement module;

[0051] In the training mode, the data preprocessing receives the original palm print and palm vein images, and then performs ROI extraction, data enhancement and image enhancement in sequence, and then inputs the enhanced images into the deep hashing network;

[0052] In the matching test mode, the data preprocessing module receives the original palm print and palm vein images, performs ROI extraction and image enhancement in sequence, and then inputs the enhanced images into the deep hashing network;

[0053] Deep hashing network, including feature extraction submodule, classifier submodule and loss function submodule;

[0054] In the training mode, the received palm print and palm vein images are processed separately by the two feature extraction submodules to generate fused features, which are then input into the classifier submodule;

[0055] The classifier submodule maps the fused feature into a hash value and inputs the hash value to the loss function submodule to back-propagate the gradient and update the parameters of the feature extraction submodule and the classifier submodule;

[0056] In the matching test mode, the deep hashing network does not calculate the loss function, but only passes the image through the feature extraction network and then maps it to a hash value through the classifier;

[0057] Preprocessing of deep hash network model matching test, including ROI extraction submodule and image enhancement submodule;

[0058] The deep hash network module includes a feature extraction network submodule, a classifier submodule and a loss function submodule;

[0059] The feature extraction network submodule is the feature extraction backbone network of MPSNet;

[0060] The classifier submodule includes a 3-layer fully connected and activated submodule and a Tanh function submodule;

[0061] The 3-layer fully connected and activated submodules are connected to the Tanh function submodule;

[0062] The approximate hash matching module includes a hash tree construction submodule and a hash value matching submodule;

[0063] After the deep hash network training is completed, the hash value of each input class is constructed into a hash tree through the hash tree construction submodule in the approximate hash value matching module;

[0064] In the matching test mode, the hash value converted from the aforementioned image is input into the hash value matching submodule in the approximate hash value matching module;

[0065] A feature fusion palm print and palm vein matching method based on deep hashing network is used to train, match and test the palm print and palm vein images of several given subjects and build a search structure.

[0066] The training comprises the following steps:

[0067] S10, preprocessing the palm print and palm vein training set, obtaining training data input to the deep hash network module, and obtaining preprocessed data;

[0068] The preprocessed data is the data of the deep hash network in the training mode;

[0069] The preprocessing includes ROI extraction, data enhancement and image enhancement;

[0070] S11. For the preprocessed data, perform feature extraction, link the classifier with the same number of categories, use one-hot encoding as output, cross entropy as loss function, and Adam optimizer for pre-training, and save the feature extraction module parameters;

[0071] Data preprocessing module, deep hash network and approximate hash matching module;

[0072] Data preprocessing module, including ROI extraction module, data enhancement module and image enhancement module;

[0073] In the training mode, the data preprocessing module receives the original palm print and palm vein images, extracts the ROI through the ROI extraction module, enhances the data through the data enhancement module, and enhances the image through the image enhancement module to obtain the enhanced image; then the enhanced image is input into the deep hash network;

[0074] Deep hashing network, including feature extraction submodule, classifier submodule and loss function submodule;

[0075] In the training mode, the received palm print and palm vein images are processed into features by the two feature extraction submodules, and then fused and input to the classifier submodule; the classifier in the classifier submodule maps the fused features into hash values ​​and inputs the hash values ​​into the loss function to back-propagate gradients and update the parameters of the feature extraction submodule and the classifier submodule;

[0076] In training mode, the hash value of each input class is constructed into a hash tree through the hash tree construction submodule of the approximate hash value matching module;

[0077] S12. For the preprocessed data, use the features extracted by the feature extraction submodule to connect the classifier of the deep hash network, calculate the gradient through the loss function and update the classifier parameters to obtain a trained deep hash network;

[0078] S13. For a given palm print and palm vein training set, use the trained deep hash network to calculate the hash value of each sample, average the hash values ​​of samples of the same type to find the representative hash value, and then record the representative hash values ​​of all classes;

[0079] The representative hash value refers to: averaging the hash values ​​of samples of the same type and finding the hash value with the smallest sum of Hamming distances to all hash values ​​of the same type;

[0080] S14, constructing a hash tree based on the representative hash values ​​of all classes by using a hash tree construction method in approximate hash value matching;

[0081] The matching, a feature fusion palm print and palm vein matching method based on deep hash network, comprises the following steps:

[0082] S20, the data preprocessing module receives the original palm print and palm vein images, extracts the ROI through the ROI extraction module, and enhances the image through the image enhancement module, to obtain a preprocessed image; and then inputs the preprocessed image into the deep hash network;

[0083] S21, relying on the trained deep hash network, extract features from the preprocessed data through the feature extraction submodule in turn, then connect the classifier of the deep hash network, and map it into a hash value through the classifier;

[0084] The preprocessed data is extracted features through the feature extraction submodules in turn, specifically:

[0085] The received palm print and palm vein images are processed into features and then fused through two feature extraction submodules;

[0086] The classifier is mapped to a hash value, specifically: the classifier in the classifier submodule maps the features fused by the feature extraction submodule to a hash value;

[0087] S22, inputting the hash value into an approximate hash value matching module for matching, and obtaining a matching result;

[0088] The approximate hash value matching module performs matching by using an approximate hash matching method, specifically including:

[0089] S221, Initialization: Using different Hamming distances d l As the stratification criterion, the minimum Hamming distance d L =Threshold is considered to be the threshold for determining whether two hashes are of the same type.

[0090] S222, construct a hash tree: for a set of hash values ​​H with different categories, ,h′∈H,label(h)≠label(h′), select one of the hash values ​​as the root node, and then iteratively add nodes to the tree one by one. For the parent node v at the lth level, its hash value is h. If you want to add a child node to it (the corresponding hash value is h new ), then the iteration unit is run iteratively.

[0091] S223, for the built hash table, search for a hash value h match Corresponding labels: directly on the root node and h match Run the iteration unit.

[0092] The iteration rules are:

[0093] For each child node v of v son (The corresponding hash value is h son ):

[0094] S2230, determine the Hamming distance D h (h son ,h new )<d l Is it true? If so, you can continue to try to add the node and add it to the list of Nodes that may be iterated in the next step possible , otherwise it returns adding failure.

[0095] S2231, determine the Hamming distance D h (h son ,h new )>Whether Threshold is met. If so, return failure to add the node (for matching, return success and the label of the matching node). Otherwise, continue to try to add the node.

[0096] S2231, try to use Nodes possible Calls an iteration unit to try to add or match a new hash value in its children.

[0097] Example 2

[0098] This example uses the Tongji palm print and vein dataset to train a deep hashing network model.

[0099] Mode 1: Training a deep hashing network. The process can be referred to Figure 1 :

[0100] S1: Dataset division: First, the Tongji palm print and palm vein dataset is divided into a training set and a test set to ensure that the model is validated on an independent dataset. Since each of the 600 classes in the Tongji dataset has 20 palm print and palm vein images, 14 of them are selected as the training set.

[0101] S2: Data preprocessing: In the data preprocessing stage, ROI extraction, data enhancement, and image enhancement are performed on each image. Figure 3. First, the image is binarized using the Otsu algorithm, and then the edge is approximated by the Douglas-Peucker algorithm to determine the palm frame area. Next, the image is rotated according to the position of the finger joints, and the palm frame area is intercepted. Finally, the selected frame area is regularized to generate a 128*128 image. At the same time, the training set images are subjected to cylindrical projection transformation and perspective transformation data enhancement. The effect of data enhancement can be referred to Figure 4 . Then the palm print and palm vein images are enhanced by histogram equalization and Jerman filtering. Figure 5 .

[0102] S3: Model training: Use the preprocessed data to train the deep hashing network model of feature-level fusion. The feature-level fusion method is referenced Figure 6 , the feature extraction module of the deep hashing network is as follows Figure 7 , here we take the generation of a hash value of 512 bits as an example. First, connect the feature extraction module and the classifier, use one-hot encoding as the output, use cross entropy as the loss function, use the Adam optimizer for pre-training, and save the feature extraction module parameters. Next, use the saved feature extraction module to connect the classifier of the deep hash network, use the loss function proposed in the present invention to calculate the gradient, and update the classifier parameters through the Adam optimizer to obtain a trained deep hash network. The trained deep hash network is tested with a validation set, and an equal error rate of 0.56% can be achieved. In addition, the optimal threshold for judging whether two hash values ​​are similar can be calculated, and the threshold is calculated to be 173.

[0103] S4: Hash value calculation: For a given palm print and palm vein training set, use the trained deep hash network to calculate the hash value of each sample, and average the hash values ​​of samples of the same type. The specific method is to find a representative hash value so that the sum of its Hamming distance with all hash values ​​of the same type is the smallest, and record the representative hash values ​​of all classes.

[0104] S5: Hash tree construction: method flow reference Figure 8 For the representative hash values ​​of all classes, a hash tree is constructed by the hash tree construction algorithm in the approximate hash value matching algorithm proposed in the present invention for subsequent matching. Although limited by the size of the data set, that is, the Tongji data set in this example has only 600 classes, in the case of tens of millions of data classes, the time complexity of constructing a hash tree is close to linear complexity. The results are shown in Table 1. Fig. 9 .

[0105] Mode 2: Match target image, process reference Figure 2 :

[0106] S1: Data preprocessing: For the palm vein images of a given subject, the same data preprocessing operations as the training set are performed, including ROI extraction and image enhancement.

[0107] S2: Hash value calculation: Input the preprocessed image into the trained deep hash network model to obtain the hash value of the target image.

[0108] S3: Approximate hash value matching: Use the approximate hash value matching method proposed in this invention to match the target image through the hash tree to obtain the label. For the matching time of tens of millions, the basic logarithmic complexity can be referred to Fig.10 , the matching time for tens of millions of classes is 1.33ms on average when the CPU is i7-13900HX and the programming language is python, and the real-time performance is relatively good.

[0109] Although the present invention is described herein in conjunction with various embodiments, in the process of implementing the claimed invention, those skilled in the art may understand and implement other variations of the disclosed embodiments by viewing the drawings, the disclosure, and the drawings, etc. In the specification, the word "comprising" does not exclude other components or steps, and "one" or "an" does not exclude multiple situations. A single processor or other unit can implement several functions listed in the specification. Certain measures are recorded in different embodiments, but this does not mean that these measures cannot be combined to produce good results.

[0110] Although the present invention has been described in conjunction with specific features and embodiments thereof, it is apparent that various modifications and combinations may be made thereto without departing from the spirit and scope of the present invention. Accordingly, this specification and the accompanying drawings are merely exemplary illustrations of the present invention and are deemed to cover any and all modifications, variations, combinations or equivalents within the scope of the present invention. Obviously, those skilled in the art may make various modifications and variations to the present invention without departing from the spirit and scope of the present invention. Thus, the present invention is intended to include such modifications and variations if they fall within the scope of the present invention and its equivalents.

Claims

1. A feature fusion palm print and palm vein detection system based on deep hash network, with training mode and matching test mode; characterized in that: The detection system includes a data preprocessing module, a pretraining module, a deep hash network and an approximate hash matching module; the data preprocessing module is connected to the pretraining module; the data preprocessing module and the pretraining module are both connected to the deep hash network, and the deep hash network is connected to the approximate hash matching module; the deep hash network is constructed based on the feature extraction backbone network of the MPSNet model; The data preprocessing module is used to preprocess the data input into the deep hash network to obtain preprocessed data, and the preprocessed data is respectively input into the pretraining module and the deep hash network; the pretraining module receives the preprocessed data, performs feature extraction, links to the classifier with the same number of categories, uses one-hot encoding as output, cross entropy as loss function, and performs pretraining with Adam optimizer, and saves the parameters of the feature extraction submodule; The deep hashing network in training mode includes a feature extraction submodule, a classifier submodule, and a loss function submodule; The deep hash network matching the training mode does not calculate the loss function. It only extracts features from the data through the feature extraction submodule after preprocessing, and then maps it to a hash value through the classifier submodule; The approximate hash matching module includes a hash tree construction submodule and a hash value matching submodule, which work in a training mode and a matching test mode respectively.

2. According to claim 1, a feature fusion palmprint and palm vein detection system based on deep hash network is characterized in that: The deep hashing network includes a data preprocessing module for training mode and matching test mode; The data preprocessing module of the training mode includes a ROI extraction submodule, a data enhancement submodule, and an image enhancement submodule; The data preprocessing module matching the test pattern includes a ROI extraction submodule and an image enhancement submodule.

3. According to claim 2, a feature fusion palmprint and palm vein detection system based on deep hash network is characterized in that: In the training mode, the data preprocessing module receives the original palm print and palm vein images, extracts them through the ROI extraction submodule, enhances them through the data enhancement submodule and the image enhancement submodule, and then inputs the enhanced images into the deep hashing network; In the matching test mode, the data preprocessing module receives the original palm print and palm vein images, extracts them through the ROI extraction submodule and enhances them through the image enhancement submodule in turn, and then inputs the enhanced images into the deep hash network.

4. According to claim 1, a palm print and palm vein detection system based on deep hash network with feature fusion is characterized in that: The feature extraction submodule is the feature extraction backbone network of MPSNet; the classifier submodule includes a fully connected layer, an activation submodule and a Tanh function submodule; the fully connected layer and the activation submodule are connected to the Tanh function submodule.

5. According to claim 4, a palm print and palm vein detection system based on deep hash network is characterized in that: In the training mode, the deep hash network receives palm print and palm vein images, which are processed by two feature extraction submodules to generate fused features, and then the fused features are input into the classifier submodule. The classifier submodule maps the fused feature into a hash value and inputs the hash value to the loss function submodule to back-propagate the gradient and update the parameters of the feature extraction submodule and the classifier submodule; In the matching test mode, the loss function is not calculated. The image is only preprocessed and then passed through the feature extraction submodule to extract features, and then mapped to a hash value through the classifier submodule.

6. According to the feature fusion palmprint and palm vein detection system based on deep hash network in claim 1, it is characterized in that: In the training mode, after the deep hash network training is completed, the hash value of each input class is constructed into a hash tree through the hash tree construction submodule; in the matching test mode, the hash value converted from the above image is input into the hash value matching submodule for matching.

7. A feature fusion palm print and palm vein matching method based on a deep hash network is used to train, match and test the palm print and palm vein images of a given number of subjects and construct a search structure. The method is characterized by: The training comprises the following steps: S10, preprocessing the palm print and palm vein training set, obtaining training data input to the deep hash network module, and obtaining preprocessed data; S11. For the preprocessed data, perform feature extraction and link the classifiers with the same number of categories, then use one-hot encoding as output, cross entropy as loss function, and Adam optimizer for pre-training, and save the feature extraction submodule parameters; S12. For a given palm print and palm vein training set, train a deep hash network to calculate the hash value of each sample included in the palm print and palm vein training set, average the hash values ​​of samples of the same type to obtain a representative hash value, and then record the representative hash values ​​of all classes; S13. Based on the representative hash values ​​of all classes, a hash tree is constructed by a hash tree construction method in approximate hash value matching.

8. The feature fusion palmprint and palm vein matching method based on deep hash network according to claim 7 is characterized in that: The representative hash value mentioned in S12 refers to: finding the average of the hash values ​​of the same type of samples and finding the hash value with the smallest sum of Hamming distances to all the same type of hash values.

9. The feature fusion palmprint and palm vein matching method based on deep hash network according to claim 7 is characterized in that: The matching comprises the following steps: S20, the data preprocessing module receives the original palm print and palm vein images, extracts the ROI through the ROI extraction submodule, and enhances the image through the image enhancement submodule in turn, to obtain a preprocessed image; and then inputs the preprocessed image into the deep hash network; S21, based on the deep hash network, extracting features from the preprocessed image through the feature extraction submodule in sequence, then connecting the classifier of the deep hash network, and mapping them into hash values ​​through the classifier; S22: Input the hash value into an approximate hash value matching module for matching to obtain a matching result.

10. The feature fusion palmprint and palm vein matching method based on deep hash network according to claim 9 is characterized in that: The preprocessed image is extracted features through the feature extraction submodule, specifically: The received palm print and palm vein images are processed into features and then fused through two feature extraction submodules; The classifier is mapped to a hash value, specifically: the classifier in the classifier submodule maps the features fused by the feature extraction submodule to a hash value.

Citation Information

Patent Citations

  • Palmprint image recognition method based on convolutional neural network and Hashing code

    CN107403153A

  • NEPVR-based palm vein recognition method and system

    CN118279940A