Finger vein recognition method, device and storage medium based on unsupervised domain adaptation

Abstract

The embodiments of the present invention provide a finger vein recognition method, device, electronic device and storage medium based on unsupervised domain adaptation. The method includes: collecting finger vein images; preprocessing the finger vein images to obtain region of interest (ROI) images; performing feature extraction processing on the ROI images according to a preset finger vein data set and a preset unsupervised domain adaptation feature extraction network to obtain finger vein features; matching processing on the finger vein features according to a preset finger vein database and a preset classification network to obtain finger vein recognition results. According to the solution of the embodiments of the present invention, the problem of poor generalization ability in the finger vein recognition system trained with a single data set is solved, and the present invention realizes unsupervised learning of finger veins, and feature extraction can be completed without making finger vein labels of the subject.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a finger vein recognition method, device, electronic device and storage medium based on unsupervised domain adaptation. Background Art

[0002] As people's demand for security systems continues to grow, biometric recognition has attracted more and more attention and has become one of the most critical and challenging tasks in information security. Among them, finger vein recognition technology has been widely used in information security, online payment and other fields due to its characteristics of liveness recognition and strong anti-counterfeiting. In recent years, finger vein recognition has attracted the attention of more researchers. The finger vein recognition system mainly includes two processes: feature extraction and matching recognition. Finger veins contain a lot of irregular texture information, shadow parts and noise. The finger vein images of the same finger have similar information, but there are significant differences between different fingers. Therefore, people usually select functional patterns from finger veins and matching strategies for recognition.

[0003] Most of the existing finger vein recognition technologies are focused on reducing the recognition error rate from a single data set, without considering that the finger vein recognition system in actual applications needs to collect vein information from different people. A single data set will make the recognition system's generalization ability poor, resulting in large differences in recognition rates for different users, which will undoubtedly greatly reduce the security of the finger vein recognition system. Existing vein extraction work based on deep learning requires labels to be made in advance for network learning, that is, the finger vein feature extraction stage is a supervised learning method. However, this is impractical in actual applications. We cannot extract corresponding labels for users for the network to extract features, and making accurate labels also requires a lot of manpower and material resources. Summary of the invention

[0004] The present invention aims to solve at least one of the technical problems existing in the prior art.

[0005] To this end, the present invention proposes a finger vein recognition method based on unsupervised domain adaptation, which solves the problem of poor generalization ability in the finger vein recognition system trained with a single data set. In addition, the present invention realizes unsupervised learning of finger veins, and feature extraction can be completed without making finger vein labels of the subject, which has a crucial impact on the application of finger vein recognition systems based on deep learning.

[0006] The present invention also proposes a device that applies the above-mentioned finger vein recognition method based on unsupervised domain adaptation.

[0007] The present invention also proposes an electronic device using the above-mentioned finger vein recognition method based on unsupervised domain adaptation.

[0008] The present invention also proposes a computer-readable storage medium that applies the above-mentioned finger vein recognition method based on unsupervised domain adaptation.

[0009] According to a finger vein recognition method based on unsupervised domain adaptation according to an embodiment of the first aspect of the present invention, the method includes:

[0010] Collect finger vein images;

[0011] Preprocessing the finger vein image to obtain a region of interest ROI image;

[0012] Performing feature extraction processing on the ROI image according to a preset finger vein data set and a preset unsupervised domain adaptation feature extraction network to obtain finger vein features;

[0013] The finger vein features are matched according to a preset finger vein database and a preset classification network to obtain a finger vein recognition result.

[0014] The finger vein recognition method based on unsupervised domain adaptation according to the embodiment of the present invention has at least the following beneficial effects: first, a finger vein image is collected; then, the collected finger vein image is preprocessed to obtain an ROI image; then, feature extraction processing is performed on the ROI image according to a preset finger vein data set and a preset unsupervised domain adaptation feature extraction network to obtain finger vein features; finally, matching processing is performed on the obtained finger vein features according to a preset finger vein database and a preset classification network to obtain finger vein recognition results. Through the above technical solution, the problem of poor generalization ability in the finger vein recognition system trained with a single data set is solved, and finger vein unsupervised learning is realized. Feature extraction can be completed without making finger vein labels of the subject, which has a crucial impact on the application of finger vein recognition systems based on deep learning.

[0015] According to some embodiments of the present invention, the finger vein dataset includes a source domain image and a source domain label, and the feature extraction processing of the ROI image according to the preset finger vein dataset and the preset unsupervised domain adaptation feature extraction network to obtain the finger vein feature includes:

[0016] Inputting the source domain image and the source domain label into the unsupervised domain adaptation feature extraction network to pre-train the unsupervised domain adaptation feature extraction network;

[0017] Based on the pre-trained unsupervised domain adaptation feature extraction network, the ROI image is segmented to obtain a target domain extraction result and an initial target domain pseudo label;

[0018] Determining underlying features of the source domain image as a metadata set;

[0019] Performing correction processing on a preset noise transfer matrix based on the metadata set;

[0020] Based on the target domain extraction result and the corrected noise transfer matrix, the initial target domain pseudo-label is corrected to obtain the finger vein feature.

[0021] According to some embodiments of the present invention, matching the finger vein features according to a preset finger vein database and a preset classification network to obtain a finger vein recognition result includes:

[0022] Based on the classification network, the finger vein feature and the finger vein database are paired to obtain a matching score;

[0023] The finger vein recognition result is obtained according to the matching score.

[0024] According to some embodiments of the present invention, the collecting of finger vein images includes:

[0025] The finger vein image is acquired based on the finger vein acquisition device;

[0026] Wherein, the finger vein collection device includes:

[0027] A finger vein collection device, the finger vein collection device comprising a finger vein collection barrel, the finger vein collection barrel is used for the finger of the subject to be collected to be placed in, and a plurality of infrared light sources are evenly arranged around the finger vein collection barrel;

[0028] A capture button, used to trigger different infrared light sources to light up to realize different image capture modes, wherein the image capture modes include transmission capture mode, side-shooting capture mode and reflection capture mode;

[0029] A processor, the finger vein collection device, the infrared light source and the collection button are all connected to the processor.

[0030] According to some embodiments of the present invention, the finger vein acquisition device further includes a plurality of indicator lights, which are connected to the processor and are used to indicate completion status of different image acquisition modes.

[0031] According to some embodiments of the present invention, acquiring the finger vein image based on a finger vein acquisition device includes:

[0032] In response to a first instruction of the acquisition button, the finger vein acquisition device acquires a transmission acquisition image; and in response to a second instruction of the acquisition button, the finger vein acquisition device acquires a side emission acquisition image; and in response to a third instruction of the acquisition button, the finger vein acquisition device acquires a reflection acquisition image;

[0033] The transmission collection image, the side-shooting collection image and the reflection collection image are corrected and synthesized based on the processor to obtain the finger vein image.

[0034] According to a second aspect of an embodiment of the present invention, a finger vein recognition device based on unsupervised domain adaptation comprises:

[0035] The first unit is used for collecting finger vein images;

[0036] The second unit is used to preprocess the finger vein image to obtain a region of interest ROI image;

[0037] The third unit is used to perform feature extraction processing on the ROI image according to a preset finger vein data set and a preset unsupervised domain adaptation feature extraction network to obtain finger vein features;

[0038] The fourth unit is used to match the finger vein features according to a preset finger vein database and a preset classification network to obtain a finger vein recognition result.

[0039] According to some embodiments of the present invention, the third unit includes:

[0040] A fifth unit, configured to input the source domain image and the source domain label into the unsupervised domain adaptation feature extraction network to pre-train the unsupervised domain adaptation feature extraction network;

[0041] A sixth unit is used to perform segmentation processing on the ROI image based on the pre-trained unsupervised domain adaptation feature extraction network to obtain a target domain extraction result and an initial target domain pseudo label;

[0042] A seventh unit is used to determine the underlying features of the source domain image as a metadata set;

[0043] An eighth unit is used to perform correction processing on a preset noise transfer matrix based on the metadata set;

[0044] The ninth unit is used to correct the initial target domain pseudo-label based on the target domain extraction result and the corrected noise transfer matrix to obtain the finger vein feature.

[0045] According to an embodiment of the third aspect of the present invention, the electronic device includes: a memory, a control processor, and a computer program stored in the memory and executable on the control processor, wherein the control processor implements the finger vein recognition method based on unsupervised domain adaptation as described above when executing the computer program.

[0046] According to a computer-readable storage medium of an embodiment of the fourth aspect of the present invention, the computer-readable storage medium stores computer-executable instructions, and when the computer-executable instructions are executed by a control processor, the finger vein recognition method based on unsupervised domain adaptation as described above is implemented.

[0047] Other features and advantages of the present invention will be described in the following description, and partly become apparent from the description, or understood by practicing the present invention. The purpose and other advantages of the present invention can be realized and obtained by the structures particularly pointed out in the description, claims and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The accompanying drawings are used to provide further understanding of the technical solution of the present disclosure and constitute a part of the specification. Together with the embodiments of the present disclosure, they are used to explain the technical solution of the present disclosure and do not constitute a limitation on the technical solution of the present disclosure.

[0049] Figure 1 is a flow chart of a finger vein recognition method based on unsupervised domain adaptation provided by an embodiment of the present invention;

[0050] Figure 2 It is a specific flow chart of feature extraction of a finger vein recognition method based on unsupervised domain adaptation provided by an embodiment of the present invention;

[0051] Figure 3 It is a specific flow chart of matching processing of a finger vein recognition method based on unsupervised domain adaptation provided by an embodiment of the present invention;

[0052] Figure 4 It is a specific flow chart of collecting finger vein images of a finger vein recognition method based on unsupervised domain adaptation provided by an embodiment of the present invention;

[0053] Figure 5 is a specific flow chart of collecting finger vein images of a finger vein recognition method based on unsupervised domain adaptation provided by another embodiment of the present invention;

[0054] Figure 6 It is a structural schematic diagram of an unsupervised domain adaptation feature extraction network of a finger vein recognition method based on unsupervised domain adaptation provided by an embodiment of the present invention;

[0055] Figure 7 1 is a schematic diagram of a network structure of a classification network of a finger vein recognition method based on unsupervised domain adaptation provided by an embodiment of the present invention;

[0056] Figure 8 It is a schematic diagram of the structure of a finger vein collection device provided by an embodiment of the present invention;

[0057] Fig. 9is a schematic diagram of the cross-sectional structure of a finger vein collection device provided by an embodiment of the present invention;

[0058] Fig.10 It is a schematic diagram of the side structure of a finger vein collection device provided by an embodiment of the present invention;

[0059] Fig.11 is a working principle diagram of a finger vein collection device provided by an embodiment of the present invention;

[0060] Fig.12 It is a schematic diagram of the structure of a finger vein recognition device based on unsupervised domain adaptation provided by an embodiment of the present invention;

[0061] Fig.13 It is a schematic diagram of the device structure provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0062] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0063] In the description of the present invention, "several" means one or more, "more" means more than two, "greater than", "less than", "exceed" etc. are understood as not including the number itself, and "above", "below", "within" etc. are understood as including the number itself. If there is a description of "first" or "second", it is only used for the purpose of distinguishing the technical features, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features or implicitly indicating the order of the indicated technical features.

[0064] In the description of the present invention, unless otherwise clearly defined, terms such as setting, installing, connecting, etc. should be understood in a broad sense, and technicians in the relevant technical field can reasonably determine the specific meanings of the above terms in the present invention based on the specific content of the technical solution.

[0065] The present invention provides a finger vein recognition method, device, electronic device and storage medium based on unsupervised domain adaptation, the method comprising: collecting finger vein images; preprocessing the finger vein images to obtain region of interest ROI images; performing feature extraction processing on the ROI images according to a preset finger vein data set and a preset unsupervised domain adaptation feature extraction network to obtain finger vein features; matching processing on the finger vein features according to a preset finger vein database and a preset classification network to obtain finger vein recognition results. Through the above technical solution, the problem of poor generalization ability in the finger vein recognition system trained with a single data set is solved, and finger vein unsupervised learning is realized, and feature extraction can be completed without making finger vein labels of the subject, which has a crucial impact on the application of finger vein recognition systems based on deep learning.

[0066] The embodiments of the present invention are further described below in conjunction with the accompanying drawings.

[0067] like Figure 1 As shown, Figure 1 1 is a flow chart of a finger vein recognition method based on unsupervised domain adaptation provided by an embodiment of the present invention. The method includes but is not limited to step S100, step S200, step S300 and step S400:

[0068] Step S100, collecting a finger vein image;

[0069] Step S200, preprocessing the finger vein image to obtain a region of interest ROI image;

[0070] Step S300, performing feature extraction processing on the ROI image according to a preset finger vein data set and a preset unsupervised domain adaptation feature extraction network to obtain finger vein features;

[0071] Step S400, matching processing is performed on the finger vein features according to a preset finger vein database and a preset classification network to obtain a finger vein recognition result.

[0072] It should be noted that, first, the finger vein image is collected; then, the collected finger vein image is preprocessed to obtain the ROI image; then, the ROI image is subjected to feature extraction processing according to the preset finger vein data set and the preset unsupervised domain adaptation feature extraction network to obtain the finger vein features; finally, the obtained finger vein features are matched according to the preset finger vein database and the preset classification network to obtain the finger vein recognition result. Through the above technical solution, the problem of poor generalization ability in the finger vein recognition system trained with a single data set is solved, and finger vein unsupervised learning is realized. Feature extraction can be completed without making finger vein labels of the subject, which has a crucial impact on the application of finger vein recognition systems based on deep learning.

[0073] It is worth noting that preprocessing the finger vein image to obtain the region of interest ROI image is a prior art and will not be described in detail here.

[0074] In addition, in one embodiment, the finger vein dataset includes source domain images and source domain labels, such as Figure 2 As shown, the above step S300 may include but is not limited to step S310, step S320, step S330, step S340 and step S350.

[0075] Step S310, inputting the source domain image and the source domain label into the unsupervised domain adaptation feature extraction network to pre-train the unsupervised domain adaptation feature extraction network;

[0076] Step S320, segmenting the ROI image based on the pre-trained unsupervised domain adaptation feature extraction network to obtain a target domain extraction result and an initial target domain pseudo label;

[0077] Step S330, determining the underlying features of the source domain image as a metadata set;

[0078] Step S340, performing correction processing on the preset noise transfer matrix based on the metadata data set;

[0079] Step S350: Based on the target domain extraction result and the corrected noise transfer matrix, the initial target domain pseudo-label is corrected to obtain the finger vein feature.

[0080] It should be noted that, in the process of feature extraction, the source domain image and source domain label are first input into the unsupervised domain adaptation feature extraction network to pre-train the unsupervised domain adaptation feature extraction network; then, the ROI image is segmented based on the pre-trained unsupervised domain adaptation feature extraction network to obtain the target domain extraction result and the initial target domain pseudo-label; then, the underlying features of the source domain image are determined as a metadata set; then, the preset noise transfer matrix is ​​corrected based on the metadata set; finally, based on the target domain extraction result and the corrected noise transfer matrix, the initial target domain pseudo-label is corrected to obtain the finger vein feature.

[0081] It is worth noting that Figure 6As shown, the embodiment of the present invention provides an unsupervised domain adaptation network MDAnet as a feature extraction network in the finger vein recognition system. The feature extraction network is an unsupervised domain adaptation method based on self-training, which aims to solve the problem that in practical applications, the user's finger vein label is not required. The existing other source domain finger vein images are used to guide the user's finger vein image to generate pseudo labels, and finally the user's finger vein is extracted, and the feature extraction image is sent to the recognition module. First, the source domain image with labels is input into the segmentation network for training, and the labels are obtained through the source domain segmentation loss function. Then, the target domain image is subjected to feature extraction through a segmentation network with pre-trained weights of the source domain image, and the noise transfer matrix (NTM) and the generated target domain pseudo-label are introduced to adjust the segmentation loss function. Improve the feature extraction effect of the target domain. Finally, a more accurate NTM is estimated in a learning manner through a domain-aware meta-learning strategy, so that the noise in the pseudo-labels has the least impact on the actual segmentation. The source domain data with real labels is adaptively selected as the meta-data set through the domain predictor, with the aim of obtaining meta-knowledge of the underlying label distribution of clean data in the target domain to guide the derivation of NTM.

[0082] The unsupervised domain adaptation method based on self-training is one of the commonly used methods in the field of unsupervised domain adaptation. Its core idea is to use the idea of ​​self-training to adapt the segmentation model and learn information in a specific field. However, there is a problem that needs to be solved in the self-training method. The target domain pseudo-labels generated by guiding the source domain data generally have a lot of noise, which will lead to low accuracy of the generated target domain pseudo-labels and directly lead to unstable network training and biased predictions. A feasible method is to enhance the noise tolerance characteristics of target domain risk minimization through loss correction.

[0083] Assume that the generated pseudo labels It can be obtained by the basic noise transfer matrix (NTM) T∈[0,1] C×C Convert to true label Y t , where T represents the probability of converting the true label j to the pseudo label k containing noise, that is:

[0084]

[0085] If we optimize the feature extraction network f(·) directly on the noisy pseudo-labeled target data w , we will get the noise label The noise transfer matrix (NTM) connects the posterior probabilities of the noisy pseudo labels The formula for the category probability of the true label can be expressed as:

[0086]

[0087] p(,w) represents the feature extraction network f(·) w The probability of

[0088] The loss function of the self-training method is:

[0089]

[0090] Where X S is the source domain data, Y S is the source domain label, X T is the target domain data, The pseudo label generated for the target domain, f(,w) represents the pseudo label generated for the target domain in the feature extraction network f(·) w is being carried out.

[0091] Given NTM(T) modify and correct the self-training loss in Eq. It is expressed as:

[0092]

[0093] The corrected loss function promotes the similarity between the noise-adapted class posterior probabilities and the noisy pseudo-labels. By obtaining the NTM, it is possible to correct the pseudo-labels mixed with noise and even train the segmentation model with noisy data.

[0094] The effectiveness of loss correction methods is highly dependent on the estimation of NTM(T), and the estimation of T directly affects the generation of pseudo labels in the target domain. This paper adopts a domain-aware meta-learning strategy to solve the estimation problem of NTM. The domain-aware meta-learning strategy estimates T by using the empirical risk on metadata with clean labels minimized, and optimizes the segmentation network, and uses the supervisory signal corrected by the previous approximation T on unlabeled target data to optimize the segmentation network.

[0095] The key idea of ​​the domain-aware meta-learning strategy is to estimate T. First, we construct a set of meta-data sets {X M ,M M}={x m ,y m} m∈M , here we choose the source domain data to construct such a metadata dataset because it has clean knowledge of the underlying label distribution. Using the constructed metadata dataset to estimate the NTM, the NTM can be updated to T * , the formula for the update method is as follows:

[0096]

[0097] in,

[0098]

[0099] where x m The image in the metadata set, y m The label of the metadata dataset, x t represents the target domain data, represents the generated target domain pseudo label, w(T) * represents the optimal segmentation network with minimum correction loss on the noisy pseudo-labeled target data, and the updated T * The empirical loss of metadata is minimized using the optimal segmentation network. By guiding the estimation of T through risk minimization on the metadata dataset, this method can avoid false supervision signals and improve the generalization ability of the adaptive model.

[0100] Using the estimated T * , the noisy pseudo-label target data can be effectively used to optimize the segmentation network. The segmentation network of the source data and the target data is jointly optimized. At this time, the unsupervised domain adaptation objective function of this method can be expressed as:

[0101]

[0102] By jointly optimizing the NTM associated with the segmentation network, the domain-aware meta-learning strategy can simultaneously estimate the noise distribution in pseudo-labels and perform loss correction for low segmentation error of target data.

[0103] Jointly optimize NTM(T) and segmentation network f(·) w There are three main steps: Pseudo-optimization: First, the segmentation network pre-trained on the source domain data is copied. The copied network is called the meta-network. The meta-network has the same weight parameter W as the segmentation network. i , at this time, the target domain image is input into the meta-network for training, and the generated pseudo-label and noise transfer matrix (NTM) are used to The loss function corrects the network and updates the parameters W in the meta-network in the direction of the loss function gradient descent. i , the updated network parameters are This optimization can be expressed as:

[0104]

[0105] Among them, γ v represents the learning rate, represents the gradient of the network, represents the pseudo label of the target domain, x t represents the target domain image, T i Represents the corresponding noise matrix at this time.

[0106] Meta-learning optimization, input the images of the meta-dataset into the meta-network with updated parameters Its corresponding metadata label is obtained through segmentation loss Guide the meta-network to learn, and through the meta-network Updated noise transfer matrix T i , after updating to T i+1 , the process can be expressed as:

[0107]

[0108] where γ m Meta-learning optimized learning rate, represents the gradient value of T, y m Indicates the label corresponding to the metadata image, x m Represents an image in a metadata set.

[0109] True optimization, finally the target domain data is input into the original segmentation network for training, that is, the parameter is W i The segmentation network is then updated by updating the corrected noise transfer matrix T i+1 and the corresponding pseudo labels To optimize the segmentation network and extract features from the target domain image. The optimized segmentation network parameters are W i+1 , which can be expressed as:

[0110]

[0111] where γ a represents the learning rate, represents the gradient of the network, represents the pseudo label of the target domain, x t represents the target domain image, y s represents the source domain label, x s Represents the source domain image.

[0112] In addition, in one embodiment, if Figure 3 As shown, the above step S400 may include but is not limited to step S410 and step S420.

[0113] Step S410, pairing the finger vein feature with the finger vein database based on the classification network to obtain a matching score;

[0114] Step S420, obtaining the finger vein recognition result according to the matching score.

[0115] It should be noted that first, the finger vein features and the finger vein database are paired based on the classification network to obtain a matching score; finally, the finger vein recognition result is obtained according to the matching score.

[0116] It is worth noting that in the feature matching stage, the present invention uses the classification network EfficientNet to perform training and testing for matching after feature extraction, and finally outputs the corresponding finger vein matching results according to the matching score. During matching, the same finger images of the same collector are paired in pairs, recorded as positive samples, and different fingers are paired in pairs, recorded as negative samples. The matching network is trained using positive and negative sample pairs, and the data set is divided into a ratio of 6:3:1 (training set; validation set; test set). The network structure of EfficientNet is as follows: Figure 7 shown.

[0117] In addition, in one embodiment, if Figure 4 As shown, the above step S100 may include but is not limited to step S110.

[0118] Step S110: acquiring a finger vein image based on the finger vein acquisition device.

[0119] Among them, Figures 8 to 10 As shown, the finger vein collection device includes:

[0120] The finger vein collection device comprises a finger vein collection barrel 1000, the finger vein collection barrel 1000 is used for the finger of the subject to be collected to be placed, and a plurality of infrared light sources 1400 are evenly arranged around the circumference of the finger vein collection barrel 1000;

[0121] The acquisition button 1100 is used to trigger different infrared light sources 1400 to light up to realize different image acquisition modes, including transmission acquisition mode, side-shooting acquisition mode and reflection acquisition mode;

[0122] The processor 1200 , the finger vein collection device, the infrared light source 1400 and the collection button 1100 are all connected to the processor 1200 .

[0123] It should be noted that finger vein images can be collected by a finger vein collection device; wherein the finger vein collection device includes: a finger vein collection device, the finger vein collection device includes a finger vein collection barrel 1000, the finger vein collection barrel 1000 is used for the collection subject's finger to be placed, and a plurality of infrared light sources 1400 are evenly arranged around the finger vein collection barrel 1000; a collection button 1100, used to trigger different infrared light sources 1400 to light up to achieve different image collection modes, the image collection modes including transmission collection mode, side-shooting collection mode and reflection collection mode; a processor 1200, the finger vein collection device, the infrared light source 1400 and the collection button 1100 are all connected to the processor 1200.

[0124] In some embodiments, the finger vein acquisition device further includes a plurality of indicator lights 1300 , which are connected to the processor 1200 , and are used to indicate the completion status of different image acquisition modes.

[0125] In addition, in one embodiment, if Figure 5 As shown, the above step S110 may include but is not limited to step S111 and step S112.

[0126] Step S111, in response to a first instruction of the acquisition button, the finger vein acquisition device acquires a transmission acquisition image; and in response to a second instruction of the acquisition button, the finger vein acquisition device acquires a side emission acquisition image; and in response to a third instruction of the acquisition button, the finger vein acquisition device acquires a reflection acquisition image;

[0127] Step S112: Based on the processor, the transmission collected image, the side-shooting collected image and the reflection collected image are corrected and synthesized to obtain a finger vein image.

[0128] It should be noted that, in response to the first instruction of the acquisition button, the finger vein acquisition device acquires a transmission acquisition image; and in response to the second instruction of the acquisition button, the finger vein acquisition device acquires a side-shooting acquisition image; and in response to the third instruction of the acquisition button, the finger vein acquisition device acquires a reflection acquisition image; based on the processor, the transmission acquisition image, the side-shooting acquisition image and the reflection acquisition image are corrected and synthesized to obtain a finger vein image.

[0129] It should be noted that, at present, there are three main ways of collecting finger veins: transmission collection, side-shooting collection and reflection collection. These three methods have their own characteristics. The imaging effect of finger vein images alone is not good. The clarity of finger vein images has a very important impact on subsequent finger vein feature extraction and finger vein recognition. Therefore, the present invention proposes a new type of finger vein collection device, which can combine the advantages of the three collection methods to collect clear finger vein images and improve the accuracy of the finger vein recognition system.

[0130] The specific design of the finger vein collection device is as follows Figures 8 to 10 As shown, it can be seen from the cross-section of the device that there are a plurality of infrared light sources 1400 distributed around the acquisition device. When different acquisition buttons are pressed, the infrared light sources 1400 at different positions light up in combination. A total of three images are acquired, and the three imaging results are sent to the processor 1200, which integrates and compares them and finally synthesizes a clear finger vein acquisition image.

[0131] For example, the specific working process is as follows: First, the subject puts the finger into the finger vein collection device. After the finger is put in, press the collection button at the top of the collection device. At this time, the infrared lights in the upper, upper left and upper right directions will light up at the same time. The positions of the infrared lights are as follows: Fig. 9 As shown in the cross-section of the acquisition device, the first finger vein image is acquired. The present invention refers to the first acquisition as enhanced transmission acquisition. After the acquisition is completed, the yellow indicator light will light up, indicating that the first acquisition is completed. After completing the first step, press the middle acquisition button. At this time, the infrared lights in the three directions of the left, right, and top will light up at the same time, and the second finger vein acquisition will be performed. The second acquisition is called enhanced side-shooting acquisition. After the acquisition is completed, the green indicator light will light up. Finally, press the acquisition button at the bottom to acquire the finger vein image for the third time. At this time, the acquisition lights in the three directions of the bottom, lower left, and lower right will light up at the same time. After the final finger vein acquisition is completed, the red indicator light will light up. The third acquisition is called enhanced reflection acquisition. The finger vein images obtained by the three acquisitions are sent to the processor and matched according to the three acquired images. If the similarity of the three images reaches more than 90%, the three images will be synthesized and corrected, and the acquired finger vein images will be output. If the similarity of the three images is less than 90% due to the finger placement problem, the three indicator lights will light up at the same time, prompting the person being acquired to re-acquire. The principle process of the acquisition equipment is as follows. Fig.11 shown.

[0132] The technical solution provided by the above-mentioned embodiments not only solves the problem of poor generalization ability in the finger vein recognition system trained with a single data set, but also realizes unsupervised learning of finger veins, and feature extraction can be completed without making finger vein labels of the subject, which has a crucial impact on the application of finger vein recognition systems based on deep learning. There is currently no similar technology. In addition, in order to solve the differences in finger vein imaging, the present invention proposes a finger vein acquisition device, which is more advanced than the existing technology, and the extracted finger vein image is clearer and more accurate, and can be used in other finger vein recognition systems, or fingerprint recognition systems. The present invention is of great significance to the finger vein recognition market.

[0133] In addition, if Fig.12 As shown, an embodiment of the present invention further provides a finger vein recognition device 10 based on unsupervised domain adaptation, the device comprising:

[0134] The first unit 100 is used to collect finger vein images;

[0135] The second unit 200 is used to pre-process the finger vein image to obtain a region of interest ROI image;

[0136] The third unit 300 is used to perform feature extraction processing on the ROI image according to a preset finger vein data set and a preset unsupervised domain adaptation feature extraction network to obtain finger vein features;

[0137] The fourth unit 400 is used to perform matching processing on the finger vein features according to a preset finger vein database and a preset classification network to obtain a finger vein recognition result.

[0138] In some embodiments, the third unit 300 includes:

[0139] A fifth unit 310 is used to input the source domain image and the source domain label into the unsupervised domain adaptation feature extraction network to pre-train the unsupervised domain adaptation feature extraction network;

[0140] The sixth unit 320 is used to perform segmentation processing on the ROI image based on the pre-trained unsupervised domain adaptation feature extraction network to obtain a target domain extraction result and an initial target domain pseudo label;

[0141] A seventh unit 330 is used to determine the underlying features of the source domain image as a metadata set;

[0142] An eighth unit 340 is configured to perform correction processing on a preset noise transfer matrix based on the metadata data set;

[0143] The ninth unit 350 is used to correct the initial target domain pseudo-label to obtain the finger vein feature based on the target domain extraction result and the corrected noise transfer matrix.

[0144] It should be noted that since the finger vein recognition device 10 based on unsupervised domain adaptation in this embodiment and the finger vein recognition method based on unsupervised domain adaptation in the above embodiment are based on the same inventive concept, the corresponding contents in the method embodiment are also applicable to the present system embodiment and will not be described in detail here.

[0145] In addition, if Figure 7 As shown, an embodiment of the present invention further provides an electronic device 600, including: a memory 610, a control processor 620, and a computer program stored in the memory 610 and executable on the control processor 620, wherein the control processor 620 executes the computer program to implement the finger vein recognition method based on unsupervised domain adaptation in the above embodiment, for example, to execute the above described Figure 1 Steps S100 to S400 of the method, Figure 2 Steps S310 to S350 of the method, Figure 3 Steps S410 to S420 of the method, Figure 4 Step S110 of the method, Figure 5 Method steps S111 to S112 in .

[0146] In addition, an embodiment of the present invention further provides a computer-readable storage medium, which stores computer-executable instructions, which are executed by a control processor or controller, for example, by a control processor in the above-mentioned device embodiment, so that the above-mentioned control processor can execute the finger vein recognition method based on unsupervised domain adaptation in the above-mentioned embodiment, for example, execute the above-mentioned Figure 1 Steps S100 to S400 of the method, Figure 2 Steps S310 to S350 of the method, Figure 3 Steps S410 to S420 of the method, Figure 4 Step S110 of the method, Figure 5 Method steps S111 to S112 in .

[0147] It will be appreciated by those skilled in the art that all or some of the steps and systems in the disclosed method above may be implemented as software, firmware, hardware and appropriate combinations thereof. Some physical components or all physical components may be implemented as software executed by a processor, such as a central processing unit, a digital signal processor or a microprocessor, or may be implemented as hardware, or may be implemented as an integrated circuit, such as an application specific integrated circuit. Such software may be distributed on a computer-readable medium, which may include a computer storage medium (or a non-transitory medium) and a communication medium (or a temporary medium). As known to those skilled in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information (such as computer-readable instructions, data structures, program modules or other data). Computer storage media include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technologies, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tapes, disk storage or other magnetic storage devices, or any other medium that may be used to store desired information and may be accessed by a computer. Furthermore, it is well known to those skilled in the art that communication media typically embodies computer readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and may include any information delivery media.

[0148] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the above-mentioned implementation mode. Technical personnel familiar with the field can also make various equivalent deformations or substitutions without violating the spirit of the present invention. These equivalent deformations or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A finger vein recognition method based on unsupervised domain adaptation, characterized in that: The method comprises: Collect finger vein images; Preprocessing the finger vein image to obtain a region of interest ROI image; Performing feature extraction processing on the ROI image according to a preset finger vein data set and a preset unsupervised domain adaptation feature extraction network to obtain finger vein features; Matching the finger vein features according to a preset finger vein database and a preset classification network to obtain a finger vein recognition result; The finger vein dataset includes a source domain image and a source domain label, and the feature extraction process is performed on the ROI image according to the preset finger vein dataset and the preset unsupervised domain adaptation feature extraction network to obtain the finger vein feature, including: Inputting the source domain image and the source domain label into the unsupervised domain adaptation feature extraction network to pre-train the unsupervised domain adaptation feature extraction network; Based on the pre-trained unsupervised domain adaptation feature extraction network, the ROI image is segmented to obtain a target domain extraction result and an initial target domain pseudo label; Determining underlying features of the source domain image as a metadata set; Performing correction processing on a preset noise transfer matrix based on the metadata set; Based on the target domain extraction result and the corrected noise transfer matrix, the initial target domain pseudo-label is corrected to obtain the finger vein feature; The step of inputting the source domain image and the source domain label into the unsupervised domain adaptation feature extraction network to pre-train the unsupervised domain adaptation feature extraction network includes: Inputting the source domain image into the unsupervised domain adaptation feature extraction network for training; A preset source domain segmentation loss function is used to perform feature extraction processing on the source domain label.

2. The finger vein recognition method based on unsupervised domain adaptation according to claim 1, characterized in that: The matching process of the finger vein features according to the preset finger vein database and the preset classification network to obtain the finger vein recognition result includes: Based on the classification network, the finger vein feature and the finger vein database are paired to obtain a matching score; The finger vein recognition result is obtained according to the matching score.

3. The finger vein recognition method based on unsupervised domain adaptation according to claim 1, characterized in that: The collecting of the finger vein image comprises: The finger vein image is acquired based on the finger vein acquisition device; Wherein, the finger vein collection device includes: A finger vein collection device, the finger vein collection device comprising a finger vein collection barrel, the finger vein collection barrel is used for the finger of the subject to be collected to be placed in, and a plurality of infrared light sources are evenly arranged around the finger vein collection barrel; A capture button, used to trigger different infrared light sources to light up to realize different image capture modes, wherein the image capture modes include transmission capture mode, side-shooting capture mode and reflection capture mode; A processor, the finger vein collection device, the infrared light source and the collection button are all connected to the processor.

4. The finger vein recognition method based on unsupervised domain adaptation according to claim 3 is characterized in that: The finger vein acquisition device further includes a plurality of indicator lights, which are connected to the processor and are used to indicate the completion status of different image acquisition modes.

5. The finger vein recognition method based on unsupervised domain adaptation according to claim 3, characterized in that: The acquiring of the finger vein image based on the finger vein acquisition device includes: In response to a first instruction of the acquisition button, the finger vein acquisition device acquires a transmission acquisition image; and in response to a second instruction of the acquisition button, the finger vein acquisition device acquires a side emission acquisition image; and in response to a third instruction of the acquisition button, the finger vein acquisition device acquires a reflection acquisition image; The transmission collection image, the side-shooting collection image and the reflection collection image are corrected and synthesized based on the processor to obtain the finger vein image.

6. A finger vein recognition device based on unsupervised domain adaptation, characterized in that: The device comprises: The first unit is used for collecting finger vein images; The second unit is used to preprocess the finger vein image to obtain a region of interest ROI image; The third unit is used to perform feature extraction processing on the ROI image according to a preset finger vein data set and a preset unsupervised domain adaptation feature extraction network to obtain finger vein features; The fourth unit is used to match the finger vein features according to a preset finger vein database and a preset classification network to obtain a finger vein recognition result; The finger vein dataset includes source domain images and source domain labels, and the third unit includes: A fifth unit, configured to input the source domain image and the source domain label into the unsupervised domain adaptation feature extraction network to pre-train the unsupervised domain adaptation feature extraction network; A sixth unit is used to perform segmentation processing on the ROI image based on the pre-trained unsupervised domain adaptation feature extraction network to obtain a target domain extraction result and an initial target domain pseudo label; A seventh unit is used to determine the underlying features of the source domain image as a metadata set; An eighth unit is used to perform correction processing on a preset noise transfer matrix based on the metadata set; A ninth unit is used to correct the initial target domain pseudo-label to obtain the finger vein feature based on the target domain extraction result and the corrected noise transfer matrix; The step of inputting the source domain image and the source domain label into the unsupervised domain adaptation feature extraction network to pre-train the unsupervised domain adaptation feature extraction network includes: Inputting the source domain image into the unsupervised domain adaptation feature extraction network for training; A preset source domain segmentation loss function is used to perform feature extraction processing on the source domain label.

7. An electronic device, characterized in that: It includes at least one control processor and a memory for communicating with the at least one control processor; the memory stores instructions that can be executed by the at least one control processor, and the instructions are executed by the at least one control processor so that the at least one control processor can execute the finger vein recognition method based on unsupervised domain adaptation as described in any one of claims 1 to 5.

8. A computer-readable storage medium storing computer-executable instructions, characterized in that: When the computer executable instructions are executed by the control processor, the finger vein recognition method based on unsupervised domain adaptation as described in any one of claims 1 to 5 is implemented.

Images