Finger vein recognition training method, testing method and related device

Abstract

The present invention discloses a finger vein recognition training method, a testing method and related devices, which are applied to the field of image processing. The finger vein recognition training method includes: obtaining a plurality of finger vein image training sets; obtaining a lightweight multi-source domain adaptation network; inputting the source domain finger vein image training set into the source domain branch for pre-training; inputting the target domain finger vein image training set into the target domain branch for pre-training; inputting the target domain finger vein image training set into the backbone branch for training, and performing feature migration in the backbone branch through the branch intermediate layer feature migration module during the training of the backbone branch to obtain a migration training model; calculating the domain migration loss corresponding to the completion of the training of the migration training model through the domain migration loss converter; optimizing the backbone branch according to the domain migration loss, and using the optimized backbone branch as the finger vein recognition model. This finger vein recognition training method further improves the recognition ability and recognition efficiency of the finger vein recognition model.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular, to a finger vein recognition training method, a testing method and related devices. Background Art

[0002] With the continuous growth of people's demand for security systems, biometric recognition has attracted more and more attention and become one of the most critical and challenging tasks in information security. Among them, finger vein recognition technology has been widely used in the fields of information security, online payment, etc. due to its characteristics such as live body recognition and strong anti-counterfeiting performance. In related technologies, the generalization ability of finger vein recognition technology is poor, resulting in a large difference in the recognition rate of finger vein recognition systems for different users, which greatly reduces the security of finger vein recognition systems. Summary of the Invention

[0003] The present invention aims to solve at least one of the technical problems existing in the prior art. To this end, the embodiments of the present application provide a finger vein recognition training method, a testing method and related devices, which can improve the generalization ability of finger vein recognition technology and the recognition ability of finger vein systems.

[0004] In a first aspect, the present invention provides a finger vein recognition training method, including:

[0005] Obtaining a plurality of finger vein image training sets, wherein the plurality of finger vein image training sets include a plurality of source domain finger vein image training sets and a target domain finger vein image training set;

[0006] Obtaining a lightweight multi-source domain adaptation network, the lightweight multi-source domain adaptation network including a plurality of source domain branches, a backbone branch, a target domain branch, a branch intermediate layer feature migration module and a domain migration loss converter;

[0007] Inputting the source domain finger vein image training set into the source domain branch for pre-training to obtain first feature data;

[0008] Inputting the target domain finger vein image training set into the target domain branch for pre-training to obtain second feature data;

[0009] Inputting the target domain finger vein image training set into the backbone branch for training, and during the training process of the backbone branch, performing feature migration on the first feature data and the second feature data in the backbone branch through the branch intermediate layer feature migration module to obtain a migration training model;

[0010] Calculating the domain migration loss corresponding to the completion of the backbone branch training based on the migration training model through the domain migration loss converter;

[0011] Optimize the backbone branch according to the domain migration loss, and obtain a finger vein recognition model based on the optimized backbone branch.

[0012] The finger vein recognition training method provided by the first aspect of the present invention has at least the following beneficial effects: The finger vein recognition training method obtains a finger vein image training set, which includes multiple finger vein training images and can be used to train a finger vein recognition model; Train the finger vein recognition model according to the obtained lightweight multi-source domain adaptation network. The lightweight source domain adaptation network includes multiple source domain branches, a backbone branch, a target domain branch, a branch intermediate layer feature migration module, and a domain migration loss converter. Input the source domain finger vein image training set into the source domain branch for pre-training, and input the target domain finger vein image training set into the target domain branch for pre-training to make the performance of the source domain branch and the target domain branch reach the best. Then, optimize the backbone branch through the branch intermediate layer feature migration module and the domain migration loss converter to improve the performance of the backbone branch and obtain a finger vein recognition model. This finger vein recognition training method improves the generalization ability of finger vein recognition technology, enhances the extraction effect of the finger vein recognition model for target domain images, further improves the recognition ability and recognition efficiency of the finger vein recognition model, and enhances the user experience.

[0013] According to some embodiments of the present invention, both the first feature data and the second feature data include multiple feature maps; The method of inputting the target domain finger vein image training set into the backbone branch for training and performing feature migration on the first feature data and the second feature data in the backbone branch through the branch intermediate layer feature migration module during the training process of the backbone branch to obtain a migration training model includes:

[0014] Obtain intermediate layer feature maps from the first feature data and the second feature data respectively, where the intermediate layer feature map is at least one of the corresponding multiple feature maps;

[0015] Align the intermediate layer feature map with the feature map to be migrated in the backbone branch through a converter or a regression function;

[0016] Calculate the feature migration loss between the source domain branch and the target domain branch according to the aligned intermediate layer feature map;

[0017] Adjust the loss coefficient of the feature migration loss between the source domain branch and the target domain branch to obtain the feature migration loss of the branch intermediate layer feature migration module;

[0018] According to the feature transfer loss of the branch intermediate layer feature transfer module, narrow the distance metric of the feature similarity between the feature map to be transferred in the backbone branch and the intermediate layer feature map, so that the feature distribution of the feature map to be transferred in the backbone branch approaches that of the source domain branch and the target domain branch, and obtain a transfer training model.

[0019] According to some embodiments of the present invention, calculating the domain transfer loss corresponding to the completion of training of the transfer training model by the domain transfer loss converter includes:

[0020] Distill the first summary score output by the source domain branch and the second summary score output by the backbone branch through the source domain branch loss converter to obtain the vanilla KD loss, where the source domain branch loss converter is one of the domain transfer loss converters;

[0021] Calculate the similarity between the output image of the source domain branch and the finger vein label, where the finger vein label is a pre-set finger vein image;

[0022] Perform a weighted operation on the similarity and the vanilla KD loss to obtain a source domain loss function;

[0023] Based on the target domain branch loss converter, according to the third summary score, output image and the second summary score output by the target domain branch, obtain a target domain loss function, where the target domain branch loss converter is one of the domain transfer loss converters;

[0024] Adjust the loss coefficients of the source domain loss function and the target domain loss function to determine an intermediate loss;

[0025] Perform a weighted operation on the intermediate loss and the supervision loss of the backbone branch to obtain a domain transfer loss.

[0026] According to some embodiments of the present invention, before performing the weighted operation on the intermediate loss and the supervision loss of the backbone branch to obtain a domain transfer loss, the method further includes:

[0027] Obtain a first category vector and a second category vector, where the first category vector is the category vector of the source domain branch and the target domain branch, and the second category vector is the average value of all category vectors in the backbone branch;

[0028] Calculate the cross-entropy loss between the first category vector and the second category vector, and use the calculation result as the supervision loss of the backbone branch.

[0029] According to some embodiments of the present invention, before obtaining the trained finger vein recognition model by passing the finger vein image training set through the lightweight multi-source domain adaptation network, at least one of the following is included:

[0030] Performing edge detection on the finger vein image training set and removing false edges;

[0031] Performing midline fitting and rotation correction on the finger vein image training set after removing false edges to unify the angles of the fingers in the finger vein image training set;

[0032] Performing black filling on the finger vein image training set after midline fitting and rotation correction and intercepting the inner tangent region of the finger;

[0033] Finding the positions of finger joints according to the finger vein image training set after intercepting the inner tangent region of the finger;

[0034] Intercepting the region of interest of the finger vein image training set according to the positions of the finger joints and updating the region of interest as the finger vein image training set.

[0035] According to some embodiments of the present invention, both the source domain branch and the target domain branch are SGUnetV1 network architectures, wherein the SGUnetV1 network architecture includes five layers of encoders and four layers of decoders.

[0036] According to some embodiments of the present invention, the backbone branch is the network architecture obtained by removing the two layers of encoders and two layers of decoders with the most channels from the SGUnetV1 network architecture.

[0037] In a second aspect, the present invention provides a finger vein recognition test method, including:

[0038] Obtaining a finger vein test image and obtaining a finger vein recognition model, wherein the finger vein recognition model is trained by the finger vein recognition training method according to any item of the first aspect;

[0039] Inputting the finger vein test image into the finger vein recognition model to obtain a finger vein feature image;

[0040] Searching for the finger vein feature image in a multi-source database and determining whether there is user information corresponding to the finger vein test image.

[0041] Since the finger vein recognition test method in the second aspect applies the finger vein recognition training method according to any item of the first aspect, it thus has all the beneficial effects of the first aspect of the present invention.

[0042] In a third aspect, the present invention provides a finger vein recognition device, including:

[0043] Finger vein recognition training device, the finger vein pre-training device is used to process a finger vein image training set and obtain a trained finger vein recognition model;

[0044] Finger vein recognition testing device, the finger vein recognition testing device is used to pass a finger vein test image through the finger vein recognition model and obtain a corresponding recognition result according to the image output by the finger vein recognition model.

[0045] Since the finger vein recognition device of the third aspect applies the finger vein recognition training method of any item in the first aspect and the finger vein recognition testing method of any item in the second aspect, it thus has all the beneficial effects of the first aspect of the present invention.

[0046] Fourth aspect, the present invention provides a computer storage medium, including computer-executable instructions stored therein, the computer-executable instructions being used to execute the finger vein recognition training method of any item in the first aspect and / or the finger vein recognition testing method of any item in the second aspect.

[0047] Since the computer storage medium of the fourth aspect can execute the finger vein recognition training method of any item in the first aspect and / or the finger vein recognition testing method of any item in the second aspect, it thus has all the beneficial effects of the first aspect and / or the second aspect of the present invention.

[0048] The additional aspects and advantages of the present invention will be partly given in the following description, partly will become obvious from the following description, or will be understood through the practice of the present invention. Description of the Drawings

[0049] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required to be used in the embodiments or the related art descriptions. Obviously, the following drawings are only some embodiments of the embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without creative efforts.

[0050] Figure 1 It is a structural block diagram of the finger vein recognition device provided by the embodiment of the present application;

[0051] Figure 2 It is a main step diagram of the finger vein recognition training method provided by an embodiment of the present application;

[0052] Figure 3 It is a step diagram of the branch intermediate layer feature migration module working in the finger vein recognition training method provided by the embodiment of the present application;

[0053] Figure 4 It is a step diagram of the domain migration loss converter working in the finger vein recognition training method provided by the embodiment of the present application;

[0054] Figure 5 It is a step diagram for calculating the supervision loss of the backbone branch of the finger vein recognition training method provided by an embodiment of the present application;

[0055] Figure 6 It is a step diagram for image preprocessing of the finger vein recognition training method provided by an embodiment of the present application;

[0056] Figure 7 It is a step diagram for the finger vein recognition test method provided by an embodiment of the present application;

[0057] Figure 8 It is a schematic diagram of the lightweight multi-source domain adaptation network of the finger vein recognition test method provided by an embodiment of the present application;

[0058] Figure 9 It is a schematic diagram of the SGUnetV1 network architecture of the finger vein recognition test method provided by an embodiment of the present application;

[0059] Figure 10 It is a schematic diagram of the backbone branch of the finger vein recognition test method provided by an embodiment of the present application;

[0060] Figure 11 It is a schematic diagram of the encoder and decoder of the finger vein recognition test method provided by an embodiment of the present application;

[0061] Figure 12 It is a schematic diagram of the domain transfer loss converter of the finger vein recognition test method provided by an embodiment of the present application;

[0062] Figure 13 It is the effect diagram of image preprocessing of the finger vein recognition test method provided by an embodiment of the present application. Detailed implementation manners

[0063] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the embodiments of the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the embodiments of the present application.

[0064] It should be noted that although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that in the flowchart. Terms such as "first" and "second" in the specification, claims, and the above-mentioned drawings are used to distinguish similar objects and do not necessarily have to be used to describe a specific order or sequence.

[0065] It should also be understood that references to "one embodiment" or "some embodiments" etc. described in the specification of the embodiments of the present application mean that specific features, structures or characteristics described in connection with that embodiment are included in one or more embodiments of the embodiments of the present application. Thus, statements such as "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments" etc. that appear in different places in this specification do not necessarily all refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprise", "include", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0066] With the growing demand for security systems, biometric recognition has attracted increasing attention and become one of the most critical and challenging tasks in information security. Among them, finger vein recognition technology has been widely used in the fields of information security, network payment, etc. due to its characteristics such as live body recognition and strong anti-counterfeiting. In recent years, finger vein recognition has attracted more researchers' attention. A finger vein recognition system mainly includes two processes: feature extraction and matching recognition. Finger veins contain a lot of irregular texture information, shadow parts and noises. The finger vein images of the same finger have similar information, but there are significant differences between different fingers. Therefore, people usually select functional modes from finger veins and matching strategies for recognition.

[0067] The existing finger vein recognition technology has achieved fruitful results. However, many efforts are dedicated to reducing the recognition error rate from a single data set, without considering that in practical applications, a finger vein recognition system needs to collect the vein information of different people. And a single data set will make the generalization ability of the recognition system poor. When the same model is tested on other data sets, the extracted finger veins have poor effects, and the performance evaluation indicators drop significantly, resulting in large differences in the recognition rates for different users, which will undoubtedly greatly reduce the security of the finger vein recognition system. In addition, most of the existing work is carried out in the PC environment. However, this is unrealistic in practical applications. If every finger vein recognition system has the same high computing power as the PC, it will make the manufacturing cost of the finger vein recognition system too high to achieve wide application. At present, it is still very difficult to extract and match the finger veins of users on platforms with low computing power; some models that can barely be deployed will also face too long running time and it is difficult to achieve real-time finger vein recognition, which is still far from being applied to the market.

[0068] Based on this, the embodiments of the present application provide a finger vein recognition training method, a testing method, and related devices. The embodiments of the present application apply the multi-source domain adaptation technology to the field of finger vein recognition and propose a lightweight multi-source domain adaptation network (Lite-HDNet), which can transfer the general representations of multiple source domains to a single target domain, improve the generalization ability of the finger vein recognition model, and at the same time improve the extraction effect of target domain images. Moreover, the finger vein recognition model has the lightweight characteristic and can quickly recognize finger veins on an embedded platform, bringing a better experience to users.

[0069] The following further elaborates on the embodiments of the present application with reference to the accompanying drawings.

[0070] As Figure 1 shown, Figure 1 is the structural block diagram of the finger vein recognition device provided by the embodiments of the present application. In the Figure 1 example, the finger vein recognition device is mainly divided into two parts: a finger vein recognition training device and a finger vein recognition testing device.

[0071] Among them, the finger vein recognition training device and the finger vein recognition testing device are communicatively connected. The finger vein recognition training device includes an image preprocessing unit, a pre-training segmentation unit, a domain migration unit, and a pre-training recognition unit. The finger vein pre-training device is used to process the finger vein image training set and obtain a trained finger vein recognition model. The finger vein recognition testing device includes an input unit, a usage unit, an image preprocessing unit, a vein image extraction unit, a storage unit, and a recognition unit. When the user inputs finger vein information, the input unit, the image preprocessing unit, and the storage unit are used. When the user uses the finger vein recognition device, the usage unit, the image preprocessing unit, the storage unit, and the recognition unit are used. The finger vein recognition testing device is used to pass the finger vein test image through the finger vein recognition model and obtain the corresponding recognition result according to the image output by the finger vein recognition model.

[0072] The image preprocessing unit is communicatively connected to the pre-training segmentation unit. The image preprocessing unit is used to preprocess the finger vein training images. Among them, the preprocessing mainly includes image enhancement, denoising, and region of interest extraction.

[0073] The pre-training segmentation unit is communicatively connected to the image preprocessing unit and the domain migration unit respectively. The pre-training segmentation unit is used to input the source domain finger vein image training set into the source domain branch for pre-training and input the target domain finger vein image training set into the target domain branch for pre-training.

[0074] The domain migration unit is respectively communicatively connected to the pre-trained segmentation unit, the pre-trained recognition unit, and the finger vein image extraction unit. The domain migration unit is used to obtain a trained finger vein recognition model by passing the finger vein image training set through a lightweight multi-source domain adaptation network, and save the relevant information of the finger vein recognition model to the finger vein image extraction unit.

[0075] The pre-trained recognition unit is respectively connected to the domain migration unit and the recognition unit. The pre-trained recognition unit is used to take the segmentation result output by the domain migration unit as input, train a classification model, and save the relevant information of the classification model to the recognition unit.

[0076] When a user enters finger vein information, the entry unit is communicatively connected to the image preprocessing unit. The entry unit is used to collect the user's finger vein image; the image preprocessing unit is communicatively connected to the entry unit and the finger vein image extraction unit; the finger vein image extraction unit is communicatively connected to the image preprocessing unit and the storage unit. The finger vein image extraction unit is used to extract veins from the finger vein image through the finger vein recognition model, and save the user's vein information to the storage unit; the storage unit is communicatively connected to the finger vein image extraction unit, and the storage unit is used to store the user's vein image after extraction.

[0077] When the user uses it, the usage unit is communicatively connected to the image preprocessing unit. The usage unit is used to collect the user's finger vein image; the image preprocessing unit is communicatively connected to the entry unit and the finger vein image extraction unit; the finger vein image extraction unit is communicatively connected to the recognition unit. The finger vein image extraction unit is used to extract veins from the finger vein image through the finger vein recognition model, and input the user's vein information into the recognition unit; the recognition unit is communicatively connected to the finger vein image extraction unit and the storage unit. The recognition unit is used to match and recognize the image extracted by the finger vein image extraction unit with the vein image in the storage unit to recognize the user's identity.

[0078] The device and application scenarios described in the embodiments of the present application are for more clearly illustrating the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided by the embodiments of the present application. Those skilled in the art know that with the evolution of the system architecture and the emergence of new application scenarios, the technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

[0079] Those skilled in the art can understand that Figure 1 the device structure shown in

[0080] In Figure 1In the shown device structure, each module can separately call the stored finger vein recognition training program and / or finger vein recognition test program to execute the finger vein recognition test method and / or finger vein recognition training method.

[0081] Based on the above device, various embodiments of the finger vein recognition training method of the embodiments of the present application are proposed.

[0082] As Figure 2 shown, Figure 2 is the main step diagram of the finger vein recognition training method provided by an embodiment of the present application. The finger vein recognition training method includes but is not limited to the following steps:

[0083] Step S100: Obtain multiple finger vein image training sets, where the multiple finger vein image training sets include multiple source domain finger vein image training sets and a target domain finger vein image training set.

[0084] It can be understood that the source domain finger vein image training set is an existing finger vein image set.

[0085] It can be understood that then the target domain finger vein image training set is an expected finger vein image set to be learned.

[0086] Step S200: Obtain a lightweight multi-source domain adaptation network, which includes multiple source domain branches, a main branch, a target domain branch, a branch intermediate layer feature transfer module, and a domain transfer loss converter.

[0087] It can be understood that by training the finger vein recognition model through the lightweight multi-source domain adaptation network (Lite-HDNet), the recognition model has stronger generalization ability, the extracted finger veins have more general expressions, and fast finger vein extraction can also be achieved on an embedded platform.

[0088] Step S300: Input the source domain finger vein image training set into the source domain branch for pre-training to obtain first feature data.

[0089] It should be noted that the first feature data includes the source domain finger vein image training set passing through the source domain branch intermediate layer feature map, output image, etc.

[0090] Step S400: Input the target domain finger vein image training set into the target domain branch for pre-training to obtain second feature data.

[0091] It should be noted that the second feature data includes the source domain finger vein image training set passing through the target domain branch intermediate layer feature map, output image, etc.

[0092] In some embodiments, when the lightweight multi-source domain adaptation network optimizes the backbone branch through the domain transfer loss converter, the source domain branch is pre-trained in a stochastic gradient descent manner, and the lightweight multi-source domain adaptation network includes multiple source domain branches. Different source domain branches have different preferred gradient descent directions. The process of obtaining the gradient is as follows:

[0093]

[0094] where θ (τ) is the parameter of the backbone branch at iteration τ, is the KD loss with respect to the backbone branch and the m-th source domain branch, represents the gradient of the KD loss between the backbone branch and the m-th source domain branch, d is the learning direction of the desired optimization represents the inner product of and the desired optimization direction d, v is the magnitude reflecting whether d is the loss m descent direction, C > 0 is the regularization parameter, ξ is the slack variable allowing violation of The s.t. in

[0095] represents the constraint condition. Through the Lagrange multiplier, the dual equation of the optimal gradient formula can be obtained:

[0096]

[0097] Through the KKT condition, the gradient d( τ ) of the descent direction can be obtained as:

[0098]

[0099] where, is the solution of the dual equation. Therefore, the gradient for adjustment can be obtained based on the above It should be noted that optimizing the network structure in a stochastic gradient descent manner is an optimization of the lightweight multi-source domain adaptation network. Optimizing the lightweight multi-source domain adaptation network before finger vein recognition training can improve the performance of the network, making the performance of the finger vein recognition model obtained in the finger vein recognition training method better.

[0100] Step S500: Input the target domain finger vein image training set into the backbone branch for training, and perform feature transfer on the first feature data and the second feature data on the backbone branch through the branch intermediate layer feature transfer module during the training of the backbone branch to obtain a transfer training model.

[0101]

[0102] ​It should be noted that the transfer training model is the backbone branch after passing through the branch intermediate layer feature transfer module.

[0103] Step S600: Calculate the domain transfer loss corresponding to the completion of the backbone branch training based on the transfer training model through the domain transfer loss converter.

[0104] Step S700: Optimize the backbone branch according to the domain transfer loss, and obtain a finger vein recognition model based on the optimized backbone branch.

[0105] It should be noted that with reference to Figure 8 , Figure 8 is a schematic diagram of the lightweight multi-source domain adaptation network of the finger vein recognition test method provided by the embodiments of the present application. The multi-source domain adaptation network provided by the embodiments of the present application includes multiple source domain branches, a single target domain branch, a single backbone branch, a branch intermediate layer feature transfer module, and a domain transfer loss converter. The branch intermediate layer transfer module realizes knowledge transfer between the source domain branch, the target domain branch, and the backbone branch through feature loss. The outputs of the source domain branch, the target domain branch, and the backbone branch are input to the domain transfer loss converter, and the backbone branch is optimized and trained through the domain transfer loss output by the domain transfer loss converter.

[0106] It should be noted that the finger vein recognition training method provided by the embodiments of the present application uses the multi-source domain adaptation technology, and the multi-source domain adaptation technology is the key to solving image differences. The multi-source domain adaptation technology can combine multiple data sets for joint training, enabling the same model to have the characteristics of multiple data sets. The knowledge of different data sets is concentrated in one model for learning and integration, thereby improving the generalization ability and model performance of the deep learning model. Applying the multi-source domain adaptation technology to the finger vein recognition training method can not only improve the generalization ability of the finger vein recognition model, reduce the recognition rate difference between different users, but also improve the recognition performance of the finger vein recognition model, and further increase the security of the finger vein recognition device.

[0107] Therefore, the finger vein recognition training method provided by the embodiments of the present application obtains a finger vein image training set, which includes multiple finger vein training images and can be used to train a finger vein recognition model; the finger vein recognition model is trained according to the obtained lightweight multi-source domain adaptation network. The lightweight source domain adaptation network includes multiple source domain branches, a backbone branch, a target domain branch, an intermediate layer feature transfer module for branches, and a domain transfer loss converter. The source domain finger vein image training set is input into the source domain branch for pre-training, and the target domain finger vein image training set is input into the target domain branch for pre-training to make the performance of the source domain branch and the target domain branch reach the best. Then, the backbone branch is optimized through the intermediate layer feature transfer module for branches and the domain transfer loss converter to improve the performance of the backbone branch, and a finger vein recognition model is obtained. This finger vein recognition training method improves the generalization ability of the finger vein recognition technology, enhances the extraction effect of the finger vein recognition model for target domain images, further improves the recognition ability and recognition efficiency of the finger vein recognition model, and enhances the user experience.

[0108] It should be noted that both the source domain branch and the target domain branch are SGUnetV1 network architectures. Among them, the SGUnetV1 network architecture includes five layers of encoders and four layers of decoders.

[0109] It should be noted that referring to Figure 9 , Figure 9 is a schematic diagram of the SGUnetV1 network architecture of the finger vein recognition test method provided by the embodiments of the present application. The SGUnetV1 network architecture includes five layers of encoders and four layers of decoders; the multiple layers of encoders are connected through max pooling layers, while the multiple layers of decoders are connected through upsampling layers. The feature map output by the Nth layer encoder can be copied, cut, and then stitched with the feature map output by the (M - N)th layer decoder after upsampling, and the stitched result can be used as the input of the (M - N + 1)th layer decoder, where M is a positive integer greater than or equal to 1 and less than 5, and N is a positive integer greater than or equal to 1 and less than or equal to M.

[0110] In some embodiments, referring to Figure 9 , the size of the finger vein recognition training image is 64x64x1. After passing through the first-level first feature extraction module and max pooling, a feature map of 32x32x64 is obtained. The feature map is input into the second-level feature extraction module. After the finger vein recognition training image passes through the first-level first feature extraction module, it is copied, cut, and stitched with the second feature map and then input into the fourth-level second feature extraction module. At this time, the second feature map is the result obtained by upsampling the feature map output by the third-level second feature extraction module.

[0111] It should be noted that the backbone branch is the network architecture after removing the two encoder layers and two decoder layers with the most channels from the SGUnetV1 network architecture.

[0112] It can be understood that the number of layers of the encoder and decoder in the backbone branch is less than that of the source domain branch and the target domain branch, which enhances the lightweight characteristics of the backbone branch.

[0113] In some embodiments, referring to Figure 10 , Figure 10 is a schematic diagram of the backbone branch of the finger vein recognition test method provided by the embodiments of the present application. The backbone branch deletes the two encoder layers and two decoder layers with the most channels on the basis of the SGUnetV1 network architecture of the source domain branch and the target domain branch, that is, removes the encoder and decoder of the bottom two layers. The network structure of the target domain branch of the embodiments of the present application sparsifies the encoder and decoder modules and strengthens the correlation between sparse channels, so as to improve the performance of the finger vein recognition model. Among them, the number of parameters of the backbone branch is only 60.59K, and it can be deployed on the embedded platform without quantization, and can easily realize the rapid extraction of finger veins.

[0114] It should be noted that the structures of the encoder and decoder are similar, and the structure is a depthwise convolutional layer, a point convolutional layer, a point convolutional layer, a lightweight attention module, and a depthwise convolutional layer arranged in sequence.

[0115] It should be noted that referring to Figure 11 , Figure 11 is a schematic diagram of the encoder and decoder of the finger vein recognition test method provided by the embodiments of the present application. The encoder and decoder are sequentially provided with a depthwise convolutional layer, a point convolutional layer, a point convolutional layer, a lightweight attention module, and a depthwise convolutional layer. A Relu activation function and a batch normalization layer are arranged between the depthwise convolutional layer and the point convolutional layer at the entrance of the feature extraction module. A batch normalization layer is arranged between the two point convolutional layers. A Relu activation function and a batch normalization layer are arranged between the point convolutional layer and the lightweight attention module. A batch normalization layer is arranged between the point convolutional layer and the depthwise convolutional layer at the exit of the feature extraction module.

[0116] It should be noted that assuming that the input feature maps of the first feature extraction module and the second feature extraction module are First, use a 3x3 depthwise convolutional layer (depthwise conv) to perform feature extraction in the depth direction on the input feature map. At this time, no dimensional compression is performed on the input feature Figure X i and the extracted spatial features have stronger expressiveness. Then, the feature map after the first depthwise convolutional layer As the input of the hourglass-shaped pointwise convolution layer (1x1 pointwise conv), the hourglass-shaped pointwise convolution layer consists of two pointwise convolutions, and its design follows the structure of positive residual. The first pointwise convolution first compresses the feature dimension by a ratio r, and the output at this time is The second pointwise convolution restores the dimension to the input feature dimension. On the premise of extracting sufficient channel feature information, it reduces the huge overhead brought by pointwise convolution. After successive layer-by-layer convolutions and pointwise convolutions in the front, the input feature map extracts rich features in both the depth and channel directions. However, in the pointwise convolution layer, in order to reduce the overhead of pointwise convolution, the idea of first compressing and then expanding of positive residual is followed, which will cause some spatial information to be lost at the reduced pointwise convolution. To make up for the loss of these features, a 1x1 layer-by-layer convolution layer is added at the end to supplement the spatial feature information and make up for the lost part of the spatial information.

[0117] It can be understood that the source domain branch, the target domain branch, and the backbone branch retain the U-shaped architecture formed by the feature module, and replace the traditional 3x3 convolution with the hourglass-shaped depthwise separable convolution. On the basis of adopting the hourglass-shaped depthwise separable convolution, a lightweight attention module ECA is added to improve the model performance without increasing the additional number of parameters of the model.

[0118] It should be noted that referring to Figure 3 and Figure 12 , Figure 3 is the step diagram of the working of the branch intermediate layer feature migration module of the finger vein recognition training method provided by the embodiment of the present application. Step S500 includes but is not limited to the following steps:

[0119] Step S510: Obtain intermediate layer feature maps from the first feature data and the second feature data respectively, where the intermediate layer feature map is at least one of the corresponding multiple feature maps.

[0120] It can be understood that both the first feature data and the second feature data include multiple feature maps. Exemplarily, referring to Figure 12 , in the embodiments of,[], each layer of the encoder or decoder will output the corresponding feature map.

[0121] It can be understood that the number of intermediate layers of the backbone branch is less than that of the source domain branch and the target domain branch, and the intermediate layer feature map is the intermediate layer feature map output by the encoder or decoder with the same number of channels as the backbone branch. For example, referring to Figure 9 and Figure 10, when it is determined that the feature map to be migrated is the feature map output by the second-layer encoder of the dry branch, the feature map output by the second-layer encoder of the source domain branch or the target domain branch is obtained as the intermediate-layer feature map; when it is determined that the feature map to be migrated is the feature map output by the second-layer decoder of the backbone branch, the feature map output by the fourth-layer decoder of the source domain branch or the target domain branch is obtained as the intermediate-layer feature map.

[0122] Step S520: Align the intermediate-layer feature map with the feature map to be migrated in the backbone branch through a converter or a regression function.

[0123] Step S530: Calculate the feature migration loss between the source domain branch and the target domain branch according to the aligned intermediate-layer feature map.

[0124] In some embodiments, the feature migration loss L MFKD can be expressed as:

[0125]

[0126] If the Euclidean norm is used as the distance metric, the feature migration loss can be expressed as:

[0127]

[0128] where M is the number of source domain branches or target domain branches, m represents the current source domain branch or target domain branch, D(·) represents the distance metric of the similarity of two features, F s is the intermediate-layer feature map, F t is the feature map to be migrated in the backbone branch, r t is the regression function or converter, r s is the regression function or converter, r t and r s are used to align the sizes of the feature maps in the source domain branch or target domain branch and the backbone branch.

[0129] Step S540: Adjust the loss coefficient of the feature migration loss between the source domain branch and the target domain branch to obtain the feature migration loss of the branch intermediate-layer feature migration module.

[0130] In some embodiments, in order to control the knowledge migration ratio between the target domain branch and the source domain branch and make the effect of the finger vein recognition model reach the best, two hyperparameters α and β (i.e., loss coefficients) are introduced on the basis of the original feature migration loss. Finally, the loss function of the branch intermediate-layer feature migration module can be expressed as:

[0131]

[0132] where, denoted as the feature migration loss of the source domain branch, Denoted as the feature transfer loss of the target domain branch.

[0133] It can be understood that the two hyperparameters α and β are the loss coefficients of the feature transfer losses of the source domain branch and the target domain branch.

[0134] Step S550: According to the feature transfer loss of the branch intermediate layer feature transfer module, narrow the distance metric of the feature similarity between the feature map to be transferred and the intermediate layer feature map in the backbone branch, so that the feature distribution of the feature map to be transferred in the backbone branch approaches that of the source domain branch and the target domain branch, and obtain the transfer training model.

[0135] It should be noted that the branch intermediate layer transfer mechanism is designed to transfer the feature information extracted from the source domain intermediate layer during training. Its core idea is similar to feature map distillation in transfer learning, aiming to match the feature map extracted from the intermediate layer of the source domain branch or the target domain branch model with the feature map extracted from the backbone branch. The feature maps of the source domain branch or the target domain and the backbone branch are aligned through a converter, and then the feature transfer loss is used to narrow the distribution of the two types of features, enabling the backbone branch to imitate the "planned route" of the source domain branch or the target domain for training, and making the feature distribution of the intermediate layer of the backbone branch simulate the source domain branch or the target domain as much as possible, ultimately ensuring that the backbone branch obtains performance similar to that of the source domain branch or the target domain. The difference between the branch intermediate layer transfer mechanism and feature map distillation is that the branch intermediate layer transfer mechanism not only needs to narrow the feature distribution between the feature map extracted from the backbone branch and the feature map extracted from the source domain branch, but also needs to retain the feature distribution of the target domain finger vein recognition training images themselves. And the backbone branch needs to consider fast extraction of finger veins on the embedded platform, so a relatively shallow deep learning model is designed for the backbone branch. Since the network structure of the backbone branch is relatively shallow, many deep vein feature information cannot be extracted. Therefore, the branch intermediate layer feature transfer module also needs to transfer the feature information of the intermediate layer of the target domain branch, enabling the shallow backbone branch to obtain more feature information, that is, to obtain deep feature information.

[0136] It should be noted that the feature distribution of the feature map to be transferred in the backbone branch approaching the source domain branch enables the generated finger vein recognition model to recognize the source domain finger vein test images, while the feature distribution of the feature map to be transferred in the backbone branch approaching the target domain branch enables the generated finger vein recognition model to recognize the target domain finger vein test images.

[0137] It should be noted that with reference to Figure 4 , Figure 4 is the step diagram of the operation of the domain transfer loss converter of the finger vein recognition training method provided in the embodiments of the present application. Step S600 includes but is not limited to the following steps:

[0138] Step S610: Distill the first aggregated score output by the source domain branch and the second aggregated score output by the main branch through the source domain branch loss converter to obtain the vanilla KD loss, where the source domain branch loss converter is one of the domain migration loss converters.

[0139] It can be understood that the aggregated score, also known as logits, refers to the aggregated scores belonging to each category obtained after the output image summarizes various information inside the branch network.

[0140] It can be understood that by passing the first aggregated score output by the source domain branch and the second aggregated score output by the main branch through the source domain branch loss converter, the vanilla KD loss is obtained. Then the vanilla KD loss L v can be expressed as:

[0141]

[0142] where H(·) is the cross-entropy, which is used to soften the output differences between the source domain branch and the main branch. p s is the category vector of the source domain branch, p m is the category vector of the main branch, a s is the first aggregated score output by the source domain branch, a m is the second aggregated score output by the main branch, T is the temperature for softening the aggregated scores to obtain finer-grained information, σ(·) is the softmax operation using the temperature T, p[k] represents the k-th component of the vector p, and K is the total number of components in the vector.

[0143] Step S620: Calculate the similarity between the output image of the source domain branch and the finger vein label, where the finger vein label is a pre-set finger vein image.

[0144] It can be understood that by verifying the output image of the source domain branch with the finger vein label, the similarity between the output image and the finger vein label can be calculated through the Dice loss. During the calculation of the Dice loss, when predicting the binary segmentation pixels p i and the label binary pixels g i for the sum, then the similarity between the output image of the source domain branch and the finger vein label, that is, the Dice loss L Dice can be expressed as:

[0145]

[0146] Step S630: Perform a weighted operation on the similarity and the vanilla KD loss to obtain the source domain loss function.

[0147] It should be noted that the similarity, i.e., the Dice loss L Dice and the vanilla KD loss L v are weighted to obtain the source domain loss function. Then the source domain loss function L s can be expressed as:

[0148]

[0149] Step S640: Based on the target domain branch loss converter, according to the third aggregated score, the output image and the second aggregated score output by the target domain branch, obtain the target domain loss function, where the target domain branch loss converter is one of the domain transfer loss converters.

[0150] It should be noted that the principle of the target domain loss converter is the same as that of the source domain loss converter. Specifically, first, the third aggregated score output by the target domain branch and the second aggregated score output by the backbone branch are passed through the target domain branch loss converter to obtain the vanilla KD loss, and the Dice loss between the output image of the target domain branch and the finger vein label is calculated. The Dice loss and the vanilla KD loss are weighted to obtain the target domain loss function.

[0151] Step S650: Adjust the loss coefficients of the source domain loss function and the target domain loss function to determine the intermediate loss.

[0152] It can be understood that after obtaining the source domain loss function and the target domain loss function, the obtained loss functions are weighted. In order to retain the features of the target domain while learning the source domain knowledge and make the model effect reach the best, two hyperparameters γ and θ are introduced to control the transfer ratio of the two branches during the loss function weighting. At this time, the intermediate loss function can be expressed as:

[0153] L 1 = γL s + θL T

[0154] where L s is the source domain loss function, L T is the target domain loss function, and L 1 is the intermediate loss.

[0155] Step S660: Weight the intermediate loss and the supervision loss of the backbone branch to obtain the domain transfer loss.

[0156] It can be understood that the supervision loss of the backbone branch is used to initially soften the output difference between the source domain and the target domain.

[0157] It can be understood that the intermediate loss is weighted with the supervision loss of the backbone branch to obtain the domain transfer loss, then L HD can be expressed as:

[0158] L HD = L 1 + L BCE = γL s + θL T + L BCE

[0159] wherein, L s is the source domain loss function, L T is the target domain loss function, L 1 is the intermediate loss, γ and θ are two hyperparameters, and L BCE is the supervision loss of the backbone branch.

[0160] In some embodiments, the supervision loss L BCE of the backbone branch can be expressed as:

[0161] l(x,y) = L BCE = {l 1 ,..., l N} T , l n = -ω n [y n ·logx n + (1 - y n )·log(1 - x n )]

[0162] wherein, {l 1 ,..., l N} T represents the loss vector represented by a batch in the image, l n is the loss of the nth sample in this batch in the image, ω n is the weight corresponding to the nth sample, x n is the predicted value corresponding to the nth sample, x n is a probability value, y n is the category to which the nth sample belongs, and in the supervision loss L BCE of the backbone branch, y n takes a value of 0 or 1.

[0163] It can be understood that the domain transfer loss converter essentially adjusts the loss function for training the backbone branch model, so that the output image of the backbone branch can more directly learn the general representation of the source domain.

[0164] It can be understood that the role of the domain transfer loss converter is to enable the output image of the backbone branch to more directly learn the general representation of the source domain, while the intermediate layer feature transfer module of the branch is to narrow the feature information between the backbone branch and the intermediate layers of other branches. The ultimate goal of both is to make the finger vein recognition model more generalizable, and extracting finger vein features at different levels can ensure the model is more stable.

[0165] It should be noted that referring to Figure 5 , Figure 5 is a step diagram for calculating the supervision loss of the backbone branch of the finger vein recognition training method provided by the embodiments of the present application. This step is before S660, and the supervision loss calculation includes but is not limited to the following steps:

[0166] Step S671: Obtain the first category vector and the second category vector. Among them, the first category vector is the category vector of the source domain branch and the target domain branch, and the second category vector is the average value of all category vectors in the backbone branch.

[0167] Step S672: Calculate the cross-entropy loss between the first category vector and the second category vector, and use the calculation result as the supervision loss of the backbone branch.

[0168] In some embodiments, the aggregated score output of the source domain branch and the target domain branch and the second aggregated branch output by the backbone branch are input into the domain transfer loss converter together. The difference in the output between the probability source domain and the target domain is initially softened through the supervision loss BCE loss of the backbone branch, and then weighted and combined with the Dice loss. The supervision loss BCE loss of the backbone branch can also be expressed as:

[0169]

[0170] Among them, B(·) represents the BCE loss or the cross-entropy loss of binary classification, p s is the category vector of the source domain branch and the target domain branch, p t is the category vector of the backbone branch, M is the number of category vectors of the backbone branch, and m represents the category vector of the current backbone branch.

[0171] It should be noted that referring to Figure 6 , Figure 6 is a step diagram for image preprocessing of the finger vein recognition training method provided by the embodiments of the present application. This method includes at least one of the following steps:

[0172] Step S810: Perform edge detection on the finger vein image training set and remove false edges.

[0173] In some embodiments, Figure 13(a) is the original image of the finger vein training image. To perform edge detection on the finger vein image training set, Gaussian low-pass filtering is required to remove noise from the image. Since the edges of the fingers in the finger vein image are relatively clear, only the simple Prewitt operator is used. Also, because the edges of the collected image are in the horizontal direction, the vertical template of the slightly improved Prewitt operator is further used to detect the finger edges, obtaining as shown in Figure 13 (b) the image after edge detection.

[0174] In some embodiments, as shown in Figure 13 (b), it can be seen that in addition to detecting the upper and lower edges of the finger, there are also pseudo-edges formed by the structure of the acquisition device itself. Therefore, the contour information of all edges is printed. According to experience, the threshold is set to 400. When the contour area is less than the threshold, it is regarded as a pseudo-edge and then removed, obtaining Figure 13 (c) the result.

[0175] Step S820: Perform midline fitting and rotation correction on the finger vein image training set after removing pseudo-edges to unify the angles of the fingers in the finger vein image training set.

[0176] In some embodiments, first, the indices of the maximum values of the upper and lower edges of the finger vein training image after removing pseudo-edges are selected. According to its characteristics, the interval is set to 10 to obtain the coordinate points corresponding to the upper and lower edge indices. Then, the average coordinates of the maximum pixel points are calculated, obtaining as shown in Figure 13 (d) the image after midline fitting.

[0177] In some embodiments, rotation correction is completed by calculating the angle between the fitted line and the horizontal line. The missing pixels due to rotation are supplemented by bicubic interpolation, obtaining Figure 13 (e) the result.

[0178] Step S830: Perform black filling on the finger vein image training set after midline fitting and rotation correction and intercept the inscribed area of the finger.

[0179] In some embodiments, after rotation correction, bicubic interpolation causes black filling to appear in the background area of the rotated image. Although it has little impact on the extraction of the final sensing area, considering the fixed mechanism of the image sensor and near-infrared light, and the fact that the texture in this area is not rich, the black-filled part is directly removed to increase the processing speed, obtaining Figure 13 (f) the result.

[0180] In some embodiments, there are two ways to intercept the finger region, namely the inscribed region of the finger and the circumscribed region of the finger. Due to the irregularity of the finger edge, if the circumscribed region of the finger is intercepted, black filling will occur. Intercepting the inscribed region can avoid this situation. At the same time, the part of the vein pattern lost by the inscribed region compared to the circumscribed region is almost negligible. Therefore, the maximum value of the upper edge and the minimum value of the lower edge close to the midline are selected, and two inner tangent lines of the upper and lower edges are fitted. As Figure 13 shown in (g), the inscribed region of the finger intercepted by the inner tangent line of the edge is as shown in Figure 13 (h).

[0181] Step S840: According to the finger vein image training set after intercepting the inscribed region of the finger, find the positions of the finger joints.

[0182] In some embodiments, due to the different qualities of the collected vein images, not all finger regions contain rich vein patterns. Fewer patterns not only have no practical information but also affect the subsequent recognition and verification processes. First, based on the intercepted inscribed region of the finger, a window with the same height as the inscribed region slides step by step at one-thirtieth of the width, and the sum of the sliding pixels at each step is calculated to obtain the brightness change trend in the horizontal direction Figure 13 (i). It can be seen from the figure that the brightness shows peaks at the two finger joints. Therefore, this feature is used to find the positions of the finger joints.

[0183] Step S850: According to the positions of the finger joints, intercept the region of interest of the finger vein image training set, and update the region of interest as the finger vein image training set.

[0184] In some embodiments, since the joints near the finger tip are not always detectable, which is related to the dataset itself. Therefore, in the embodiments of the present application, the joints near the finger tip are used as a reference, and two-thirds of the regions on both the left and right are intercepted as the ROI region of the finger, as shown in Figure 13 (j). Finally, the final ROI region is obtained by intercepting this part, and the result is shown in Figure 13 (k).

[0185] It should be noted that referring to Figure 7 , Figure 7 is the step diagram of the finger vein recognition test method provided by the embodiments of the present application. The method includes but is not limited to the following steps:

[0186] Step S910: Obtain a finger vein test image and obtain a finger vein recognition model, where the finger vein recognition model is trained by a finger vein recognition training method.

[0187] Step S920: Pass the finger vein test image through the finger vein recognition model to obtain a finger vein feature image.

[0188] Step S930: Search for finger vein feature images in the multi-source database and determine whether there is user information corresponding to the finger vein test image.

[0189] It can be understood that when it is determined that there is user information corresponding to the finger vein test image in the multi-source database, the corresponding user information can be output; when it is determined that there is no user information corresponding to the finger vein test image in the multi-source database, it can be selected whether to enter the finger vein test image into the database as needed.

[0190] It can be understood that the finger vein recognition test method provided by the embodiments of the present application uses a multi-source database. When extracting the ROI and constructing the data set, it is necessary to adjust the pictures of the four data sets to the same size for training and verification.

[0191] In some embodiments, when outputting the corresponding recognition result according to the finger vein feature image, the embodiments of the present application use a conventional support vector machine (SVM) to complete the matching training and testing, and finally output the corresponding finger vein matching result according to the matching score.

[0192] It can be understood that by passing the finger vein removal image through the trained finger vein recognition model, the corresponding recognition result can be obtained quickly and accurately.

[0193] In addition, the present application provides an electronic device, including a memory, a processor, and a computer program stored on the memory and executable on the processor. When the processor executes the computer program, it performs the finger vein recognition training method such as steps S100 to S700 and / or the finger vein recognition test method such as steps S910 to S930.

[0194] The processor and the memory can be connected through a bus or other means.

[0195] As a non-transitory computer-readable storage medium, the memory can be used to store non-transitory software programs and non-transitory computer-executable programs. In addition, the memory can include high-speed random access memory, and can also include non-transitory memory, such as at least one magnetic disk storage device, a flash memory device, or other non-transitory solid-state storage devices. In some embodiments, the memory may optionally include a memory remotely set relative to the processor, and these remote memories can be connected to the processor through a network. Examples of the above networks include, but are not limited to, the Internet, an enterprise intranet, a local area network, a mobile communication network, and combinations thereof.

[0196] The non-transitory software programs and instructions required to implement the finger vein recognition training method and / or the finger vein recognition test method of the above embodiments are stored in the memory. When executed by the processor, they perform the finger vein recognition training method in the above embodiments. For example, they perform the above-described Figure 2The method steps S100 to S700 in, or execute the finger vein recognition test method in the above embodiments. For example, execute the Figure 7 method steps S910 to S930 in.

[0197] The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separated, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0198] In addition, an embodiment of the present application also provides a computer-readable storage medium. The computer-readable storage medium stores computer-executable instructions. When the computer-executable instructions are executed by a processor or a controller, the above processor can execute the finger vein recognition training method in the above embodiments. For example, execute the Figure 2 method steps S100 to S700 in, Figure 3 method steps S510 to S550 in, Figure 4 method steps S610 to S660 in, Figure 5 method steps S671 and S672 in, Figure 6 method steps S810 to S850 in. Again, when executed by a processor in the above embodiments, the above processor can execute the finger vein recognition test method in the above embodiments. For example, execute the Figure 7 method steps S910 to S930 in.

[0199] Those of ordinary skill in the art will understand that all or some of the steps and systems disclosed above can be implemented as software, firmware, hardware, and appropriate combinations thereof. Some or all of the physical components can be implemented as software executed by a processor, such as a central processing unit, a digital signal processor, or a microprocessor, or as hardware, or as an integrated circuit, such as an application specific integrated circuit. Such software can be distributed on a computer-readable medium, which can include a computer storage medium (or non-transitory medium) and a communication medium (or transitory medium). As is well known to those of ordinary skill in the art, the term computer storage medium includes volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer storage media includes, but is not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical disk storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and that can be accessed by a computer. In addition, it is well known to those of ordinary skill in the art that communication media typically embodies computer-readable instructions, data structures, program modules, or other data in a modulated data signal such as a carrier wave or other transport mechanism, and can include any information delivery media.

[0200] Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art can understand that various changes, modifications, substitutions, and variations can be made to these embodiments without departing from the principles and spirit of the present invention, and the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A finger vein recognition training method, characterized in that, it includes: Obtain multiple finger vein image training sets, where the multiple finger vein image training sets include multiple source domain finger vein image training sets and a target domain finger vein image training set; Obtain a lightweight multi-source domain adaptation network, which includes multiple source domain branches, a main branch, a target domain branch, a branch intermediate layer feature transfer module, and a domain transfer loss converter; Input the source domain finger vein image training set into the source domain branch for pre-training to obtain first feature data; Input the target domain finger vein image training set into the target domain branch for pre-training to obtain second feature data; Input the target domain finger vein image training set into the main branch for training, and during the training process of the main branch, perform feature transfer on the first feature data and the second feature data in the main branch through the branch intermediate layer feature transfer module to obtain a transfer training model; Calculate the domain transfer loss corresponding to the completion of the main branch training based on the transfer training model through the domain transfer loss converter; Optimize the main branch according to the domain transfer loss, and based on the optimized main branch, obtain a finger vein recognition model; Both the first feature data and the second feature data include multiple feature maps; the step of inputting the target domain finger vein image training set into the main branch for training, and during the training process of the main branch, performing feature transfer on the first feature data and the second feature data in the main branch through the branch intermediate layer feature transfer module to obtain a transfer training model includes: Respectively obtain intermediate layer feature maps from the first feature data and the second feature data, where the intermediate layer feature map is at least one of the corresponding multiple feature maps; Align the intermediate layer feature map with the feature map to be transferred in the main branch through a converter or a regression function; Calculate the feature transfer loss between the source domain branch and the target domain branch according to the aligned intermediate layer feature map; Adjust the loss coefficient of the feature transfer loss between the source domain branch and the target domain branch to obtain the feature transfer loss of the branch intermediate layer feature transfer module; According to the feature transfer loss of the branch intermediate layer feature transfer module, narrow the distance metric of the feature similarity between the feature map to be transferred in the main branch and the intermediate layer feature map, so that the feature distribution of the feature map to be transferred in the main branch approaches the source domain branch and the target domain branch, and obtain a transfer training model.

2. The finger vein recognition training method according to claim 1, characterized in that, the step of calculating the domain transfer loss corresponding to the completion of the main branch training based on the transfer training model through the domain transfer loss converter includes: Distill the first summary score output by the source domain branch and the second summary score output by the main branch through the source domain branch loss converter to obtain a vanilla KD loss, where the source domain branch loss converter is one of the domain transfer loss converters; Calculate the similarity between the output image of the source domain branch and the finger vein label, where the finger vein label is a preset finger vein image; Perform a weighted operation on the similarity and the vanilla KD loss to obtain a source domain loss function; Based on the target domain branch loss converter, obtain a target domain loss function according to the third summary score, the output image, and the second summary score output by the target domain branch, where the target domain branch loss converter is one of the domain migration loss converters; Adjust the loss coefficients of the source domain loss function and the target domain loss function to determine an intermediate loss; Perform a weighted operation on the intermediate loss and the supervision loss of the backbone branch to obtain a domain migration loss.

3. The finger vein recognition training method according to claim 2, wherein, before performing the weighted operation on the intermediate loss and the supervision loss of the backbone branch to obtain a domain migration loss, the method further includes: Obtain a first category vector and a second category vector, where the first category vector is the category vector of the source domain branch and the target domain branch, and the second category vector is the average value of all category vectors in the backbone branch; Calculate the cross-entropy loss between the first category vector and the second category vector, and use the calculation result as the supervision loss of the backbone branch.

4. The finger vein recognition training method according to claim 1, wherein, before obtaining the trained finger vein recognition model by passing the finger vein image training set through the lightweight multi-source domain adaptation network, at least one of the following is included: Perform edge detection on the finger vein image training set and remove false edges; Perform centerline fitting and rotation correction on the finger vein image training set after removing false edges to unify the angles of the fingers in the finger vein image training set; Perform black filling on the finger vein image training set after centerline fitting and rotation correction and intercept the inner cut region of the finger; According to the finger vein image training set after intercepting the inner cut region of the finger, find the positions of the finger joints; According to the positions of the finger joints, intercept the region of interest of the finger vein image training set, and update the region of interest as the finger vein image training set.

5. The finger vein recognition training method according to claim 1, wherein, Both the source domain branch and the target domain branch are SGUnetV1 network architectures, where the SGUnetV1 network architecture includes five layers of encoders and four layers of decoders.

6. The finger vein recognition training method according to claim 5, wherein, The backbone branch is the network architecture obtained by removing the two layers of encoders and two layers of decoders with the most channels from the SGUnetV1 network architecture.

7. A finger vein recognition testing method, wherein, includes: Obtain a finger vein test image and obtain a finger vein recognition model, where the finger vein recognition model is trained by the finger vein recognition training method according to any one of claims 1 to 6; Pass the finger vein test image through the finger vein recognition model to obtain a finger vein feature image; Search for the finger vein feature image in the multi-source database and determine whether there is user information corresponding to the finger vein test image.

8. A finger vein recognition device, characterized in that, it includes: A finger vein recognition training device, the finger vein pre-training device is used to process the finger vein image training set and obtain a trained finger vein recognition model, and the finger vein recognition model is trained by the finger vein recognition training method according to any one of claims 1 to 6; A finger vein recognition test device, the finger vein recognition test device is used to pass the finger vein test image through the finger vein recognition model and obtain a corresponding recognition result according to the image output by the finger vein recognition model.

9. A computer storage medium, characterized in that, it stores computer-executable instructions, and the computer-executable instructions are used to execute the finger vein recognition training method according to any one of claims 1 to 6 or the finger vein recognition test method according to claim 7.

Images