A hand vein recognition method and device fusing depth information

Abstract

The application discloses a hand vein recognition method and device fusing depth information and relates to the technical field of vein recognition. The method comprises the following steps: collecting a target image group; the target image group comprises two hand vein images with parallax; respectively pre-processing the two hand vein images to obtain two vein texture images; matching the two vein texture images to obtain a depth image; superimposing the depth image and the two vein texture images to obtain a three-channel image to be recognized; inputting the three-channel image to be recognized into a feature parameter extraction model to extract feature parameters, and obtaining feature parameters to be recognized; calculating target Euclidean distances between the feature parameters to be recognized and sample feature parameters corresponding to each sample data group; and determining an individual corresponding to the target image group according to the sizes of the target Euclidean distances. The application can improve the reliability of vein recognition and avoid cracking of a vein recognition device.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vein recognition, in particular to a hand vein recognition method and device fusing depth information. BACKGROUND

[0002] Vein recognition is a new identity recognition technology, which uses the absorption characteristics of hemoglobin in veins to near-infrared light, acquires original vein images through a near-infrared camera, and obtains vein texture images after region of interest extraction, filtering, image enhancement and image segmentation, and finally realizes identity recognition through feature extraction algorithms and matching algorithms. Vein recognition has the advantages of high accuracy, strong stability, safety and convenience, and because the vein features are located subcutaneously, they are not easy to be copied and forged. However, in actual application, vein recognition still has the risk of being forged and cracked. The simplest cracking method is to use a vein acquisition device to obtain original vein images, copy vein texture images, and crack them using pictures. Therefore, the existing vein recognition device has the problems of low recognition reliability and easy cracking. SUMMARY

[0003] The purpose of the present application is to provide a hand vein recognition method and device fusing depth information to improve the reliability of vein recognition and avoid cracking of the vein recognition device.

[0004] To achieve the above purpose, the present application provides the following solutions:

[0005] A hand vein recognition method fusing depth information, the method comprising:

[0006] acquiring a target image group; the target image group comprising two hand vein images with parallax;

[0007] preprocessing the two hand vein images respectively to obtain two vein texture images;

[0008] matching the two vein texture images to obtain a depth image;

[0009] superimposing the depth image and the two vein texture images to obtain a to-be-identified three-channel image;

[0010] inputting the to-be-identified three-channel image into a feature parameter extraction model for feature parameter extraction to obtain to-be-identified feature parameters; the feature parameter extraction model is obtained by training a twin capsule network using a sample data set; the sample data set comprises a plurality of sample data groups, and different sample data groups correspond to different individuals; one sample data group comprises a plurality of sample three-channel images taken from the same individual;

[0011] calculate a target Euclidean distance between the to-be-identified feature parameter and a sample feature parameter corresponding to each of the sample data sets; one of the sample feature parameters is obtained by inputting a sample three-channel image in the sample data set into the feature parameter extraction model to perform feature parameter extraction;

[0012] determine an individual corresponding to the target image group according to the target Euclidean distances.

[0013] Optionally, the determination method of the feature parameter extraction model comprises:

[0014] obtain a sample data set;

[0015] input sample three-channel images in the sample data set into a twin capsule network respectively to perform feature parameter extraction, and obtain corresponding sample feature parameters;

[0016] train the twin capsule network with the target that the Euclidean distance between sample feature parameters corresponding to sample three-channel images of the same individual is less than a first set value, and the Euclidean distance between sample feature parameters corresponding to sample three-channel images of different individuals is greater than a second set value, to obtain a trained twin capsule network; wherein the first set value is less than the second set value;

[0017] determine the trained twin capsule network as the feature parameter extraction model.

[0018] Optionally, the two hand vein images are respectively preprocessed to obtain two vein texture images, specifically comprising:

[0019] perform image data conversion processing on the two hand vein images respectively to obtain two grayscale images;

[0020] perform region of interest extraction processing on the two grayscale images respectively to obtain two region of interest images;

[0021] perform filter denoising processing on the two region of interest images respectively to obtain two filtered images;

[0022] perform normalization processing on the two filtered images respectively to obtain two normalized images;

[0023] perform image enhancement processing on the two normalized images respectively to obtain two enhanced images;

[0024] perform vein segmentation processing on the two enhanced images respectively to obtain two vein texture images.

[0025] Optionally, the vein segmentation processing on the two enhanced images respectively to obtain two vein texture images specifically comprises:

[0026] The center points of the hand veins are extracted from the two enhanced images respectively by using a maximum curvature algorithm to obtain two vein center point images;

[0027] All the center points of the hand veins in each of the vein center point images are connected to obtain two vein connection images;

[0028] The two vein connection images are binarized respectively to obtain two vein texture images.

[0029] Optionally, the sample three-channel images in the sample data set are input into the twin capsule network respectively to extract feature parameters to obtain corresponding sample feature parameters, and the method specifically comprises the following steps:

[0030] Each of the sample three-channel images is subjected to convolution operation to obtain a corresponding convolution result;

[0031] Each of the convolution results is subjected to capsule segmentation to obtain a plurality of corresponding capsules;

[0032] Each of the capsules is subjected to a nonlinear transformation to obtain a corresponding transformation result;

[0033] Each of the transformation results is multiplied by a weight matrix to obtain a corresponding vector result;

[0034] Each of the vector results is subjected to clustering, and the cluster center of each class is taken as a feature vector;

[0035] According to the plurality of feature vectors, a feature matrix corresponding to each of the sample three-channel images is determined;

[0036] According to each of the feature matrices, a sample feature parameter corresponding to each of the sample three-channel images is calculated.

[0037] Optionally, the target Euclidean distance between the to-be-identified feature parameter and the sample feature parameters corresponding to each of the sample data groups is calculated, and the method specifically comprises the following steps:

[0038] The Euclidean distances between the to-be-identified feature parameter and all the sample feature parameters are calculated to obtain a plurality of sample Euclidean distances;

[0039] The sample Euclidean distances of the same sample data group are averaged to obtain a plurality of target Euclidean distances.

[0040] Optionally, the individual corresponding to the target image group is determined according to the target Euclidean distances, and the method specifically comprises the following steps:

[0041] The sizes of the target Euclidean distances are compared to determine the minimum Euclidean distance;

[0042] comparing the minimum Euclidean distance with a size of a set threshold value;

[0043] if the minimum Euclidean distance is greater than or equal to the set threshold value, determining that the target image group does not belong to the individual corresponding to the sample data set;

[0044] if the minimum Euclidean distance is less than the set threshold value, determining that the target image group belongs to the individual corresponding to the sample data set, and determining the individual corresponding to the target image group as the individual corresponding to the minimum Euclidean distance.

[0045] A hand vein recognition device fusing depth information, the device is realized by the above method, and the device comprises:

[0046] a vein collection device for collecting a target image group; the target image group comprises two hand vein images with parallax;

[0047] a host computer connected with the vein collection device; the host computer comprises:

[0048] a vein segmentation module for respectively pre-processing the two hand vein images to obtain two vein texture images;

[0049] an image matching module for matching the two vein texture images to obtain a depth image;

[0050] an image superposition module for superimposing the depth image and the two vein texture images to obtain a three-channel image to be identified;

[0051] a feature parameter extraction module for inputting the three-channel image to be identified into a feature parameter extraction model to extract feature parameters to obtain feature parameters to be identified;

[0052] a target Euclidean distance calculation module for calculating target Euclidean distances between the feature parameters to be identified and sample feature parameters corresponding to each sample data group;

[0053] a category determination module for determining an individual corresponding to the target image group according to each target Euclidean distance.

[0054] Optionally, the vein collection device comprises:

[0055] two near-infrared cameras respectively connected with the host computer, for collecting a target image group of a hand part to be identified or a photo of the hand part to be identified.

[0056] Optionally, the vein collection device further comprises:

[0057] a support structure; the support structure comprises a fixed panel, a base and a support.

[0058] The base is opposite to the fixed panel, the support is between the fixed panel and the base and is perpendicular to the fixed panel and the base respectively, the base is provided with a collection area, two near-infrared cameras are arranged on the fixed panel respectively, and the collection area is used for placing a hand part to be identified or a photo of a hand part to be identified.

[0059] According to the specific embodiments provided by the present application, the following technical effects are disclosed:

[0060] The application provides a hand vein recognition method and device fusing depth information, two hand vein images with parallax are collected for preprocessing to obtain two vein texture images, the two preprocessed vein texture images are matched to obtain a depth image, and the depth image and the two vein texture images are superimposed to obtain a to-be-identified three-channel image, so that the vein depth information is fused in the to-be-identified three-channel image, the vein depth presented by a picture or other fraudulent means is significantly different from that of a living body, and the vein depth presented by the picture or other fraudulent means can be distinguished through the above processing, thereby improving the reliability of vein recognition. In addition, the application extracts feature parameters by using a trained twin capsule network, more image feature information can be retained in the capsule, and the accuracy of the recognition result is improved. BRIEF DESCRIPTION OF DRAWINGS

[0061] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced as follows. Obviously, the drawings in the following description only constitute some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor on the basis of these drawings.

[0062] Figure 1 The flowchart of the hand vein recognition method fusing depth information provided by the embodiments of the present application is shown in the figure.

[0063] Figure 2 The structure diagram of the vein collection device provided by the embodiments of the present application is shown in the figure.

[0064] Figure 3 The schematic diagram of the calibration square paper provided by the embodiments of the present application is shown in the figure.

[0065] Figure 4 The flowchart of the hand vein image preprocessing provided by the embodiments of the present application is shown in the figure.

[0066] Figure 5 The extraction effect diagram of the region of interest provided by the embodiments of the present application is shown in the figure.

[0067] Figure 6A training process schematic diagram of the twin capsule network provided by the embodiment of the present application is shown.

[0068] Symbol explanation: 1-fixed panel, 2-sliding groove, 3-near-infrared camera, 4-near-infrared filter, 5-circular near-infrared LED array, 6-bracket, 7-base, 8-acquisition area. DETAILED DESCRIPTION

[0069] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

[0070] The purpose of the present application is to provide a hand vein recognition method and device fusing depth information, so as to improve the reliability of vein recognition and avoid the cracking of the vein recognition device.

[0071] In order to make the above-mentioned purposes, characteristics and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0072] Figure 1 A flowchart of the hand vein recognition method fusing depth information provided by the embodiment of the present application is shown. Figure 1 As shown in the figure, the hand vein recognition method fusing depth information provided by the present application includes:

[0073] Step S1: acquiring a target image group; the target image group includes two hand vein images with parallax. The target image group is acquired from a to-be-recognized hand part of an individual or a photo of the to-be-recognized hand part of the individual (i.e. a counterfeit means).

[0074] As a specific embodiment, the target image group is acquired by a vein acquisition device. Before acquiring the target image group, the binocular camera of the vein acquisition device is calibrated to obtain various parameters of the camera.

[0075] Figure 2 A structural diagram of the vein acquisition device provided by the embodiment of the present application is shown. Figure 2As shown, the support structure of the venous collection device includes a fixed panel 1, a support 6 and a base 7. Among them, the height of the support 6 can be adjusted; the fixed panel 1 is fixed with a sliding groove 2 and a ring-shaped near-infrared LED array 5; the ring-shaped near-infrared LED array 5 is uniformly distributed with 12 near-infrared LEDs of 850nm as light sources; two near-infrared cameras 3 are fixed on the sliding groove 2, and the near-infrared cameras 3 can slide on the sliding groove 2 to change the camera distance; the two near-infrared cameras 3 are each installed with a near-infrared filter 4, which can ensure that the near-infrared camera 3 only receives near-infrared light near 850nm. The collection area 8 is located on the base 7. A 9x9 black and white grid paper is placed in the collection area 8 for camera calibration, and the grid paper used for calibration is as shown in Figure 3

[0076] Specifically, the calibration can be performed using Stereo CameraCalibration in the computer vision toolbox in Matlab, and after calibration, the matrix stereoParameters can be obtained, which contains various parameters including distortion parameters of the two cameras, which can be used for subsequent depth calculation. Calibration only needs to be performed once, and only when the camera is replaced, the support height is adjusted or the camera distance is adjusted, recalibration is needed.

[0077] In this embodiment, the part of the hand to be collected is placed in the collection area 8, and two hand vein images with parallax are collected by the two near-infrared cameras 3 at the same time, and the objects photographed by the two images are the same, only the angles are different.

[0078] Step S2: Preprocess the two hand vein images respectively to obtain two vein texture images. In step S2, the same preprocessing is performed on the two collected hand vein images to obtain two preprocessed hand vein images as two vein texture images.

[0079] Figure 4 A hand vein image preprocessing flowchart is provided for the embodiment of the present application. As shown in Figure 4 The two hand vein images are preprocessed respectively to obtain two vein texture images, which specifically includes:

[0080] Step S2.1: Image data conversion processing is performed on the two hand vein images respectively to obtain two gray scale images.

[0081] ​Step S2.2: performing region of interest extraction processing on the two gray-scale images respectively to obtain two region of interest images. In the embodiment, the region of interest extraction includes four steps: 1st, binarizing the hand vein image after image type conversion to obtain a binary image; 2nd, performing morphological opening operation on the binary image to smooth the edge and eliminate small spikes; 3rd, finding the largest connected domain of the binary image and constructing a circumscribed rectangle frame of the largest connected domain; 4th, determining the region of interest in the hand vein image according to the obtained circumscribed rectangle frame. The extraction effect of the region of interest is shown in FIG. 8, wherein, Figure 5 Figure 5 (a) is a gray-scale image of the hand vein image, (b) is a binary image, (c) is a circumscribed rectangle frame diagram of the largest connected domain, and (d) is a region of interest diagram extracted from (a).

[0082] Step S2.3: performing filter denoising processing on the two region of interest images respectively to obtain two filtered images. Preferably, the filter denoising processing adopts median filtering.

[0083] Step S2.4: performing normalization processing on the two filtered images respectively to obtain two normalized images. Specifically, the normalization processing includes gray-scale normalization processing and size normalization processing based on the bilinear interpolation method.

[0084] Step S2.5: performing image enhancement processing on the two normalized images respectively to obtain two enhanced images. In the embodiment, the image enhancement processing adopts a contrast limited adaptive histogram equalization (CLAHE) algorithm.

[0085] Step S2.6: performing vein segmentation processing on the two enhanced images respectively to obtain two vein texture images. The vein segmentation processing is specifically to extract the hand vein texture by using the maximum curvature algorithm, and specifically includes:

[0086] Step S2.6.1: extracting the center point of the hand vein, that is, extracting the center point of the hand vein from the two enhanced images respectively by using the maximum curvature algorithm to obtain two vein center point images.

[0087] Specifically, the hand vein image is convolved with a two-dimensional Gaussian template and its first and second order partial derivatives respectively to obtain the corresponding hand vein image partial derivatives. The two-dimensional Gaussian template is as follows:

[0088]

[0089] The first order partial derivative of the two-dimensional Gaussian template is as follows:​

[0090]

[0091]

[0092] The second-order partial derivatives of the two-dimensional Gaussian template are as follows:

[0093]

[0094]

[0095]

[0096] In the formula, (x, y) represents the coordinates of the template, the center position of the template is the origin, the size of the template is generally 28, and σ represents the standard deviation, which is generally taken as 3.5. By using the above partial derivatives to respectively convolve the hand vein image, the corresponding hand back vein image partial derivatives can be obtained, which are respectively denoted as H' x , H' y , H" xx , H" yy , H" xy and H" yx , wherein H' x represents the first-order partial derivative of the two-dimensional Gaussian template with respect to x, H' y represents the first-order partial derivative of the two-dimensional Gaussian template with respect to y, H" xx represents the second-order partial derivative of the two-dimensional Gaussian template with respect to x 2 , H" yy represents the second-order partial derivative of the two-dimensional Gaussian template with respect to y 2 , H" xy represents the second-order partial derivative of the two-dimensional Gaussian template with respect to xy, and H" yx represents the second-order partial derivative of the two-dimensional Gaussian template with respect to yx. When the convolution is performed, the size of the image is expanded by copying the values of the boundary pixel points, and the purpose of this is to process the calculation problem of the boundary pixel points of the image. From the above partial derivatives, the curvature K θ of the hand vein image in the θ direction can be calculated:

[0097]

[0098]

[0099] H' θ = H' x cos θ + H' y sin θ;

[0100]

[0101] ​The center point of the hand vein can be extracted by detecting the local maximum point at θ = 0°, 45°, 90° and 135°. θ The center point of the hand vein can be extracted by detecting the local maximum point at θ = 0°, 45°, 90° and 135°.

[0102] Step S2.6.2: connecting the center points of the hand veins, that is, connecting the center points of all the hand veins in each of the vein center point images to obtain two vein connection images.

[0103] Specifically, each center point extracted in step S2.6.1 and its left and right adjacent two pixel points are judged. If the values of the five pixel points are all large or the value of the center point is small and the values of the two adjacent four pixel points are all large, they are connected horizontally; if the value of the center point is large and the values of the two adjacent four pixel points are all small, it is considered that the center point is a noise point.

[0104] Step S2.6.3: marking the hand vein information, that is, performing binaryzation processing on the two vein connection images respectively to obtain two vein texture images.

[0105] Specifically, the OTSU algorithm is used to binaryzation the image obtained in the second step, and a vein segmentation image can be obtained, that is, the preprocessing of the hand vein image is completed.

[0106] Step S3: matching the two vein texture images to obtain a depth image.

[0107] Step S4: superimposing the depth image and the two vein texture images to obtain a three-channel image to be identified.

[0108] In this embodiment, the semi-global block matching algorithm (SGBM) is used to match the two hand vein images obtained in step S2, that is, the vein texture images, and finally a depth image is obtained. The size of the depth image is consistent with the two hand vein images obtained in step S2, and the three images are superimposed to obtain a three-channel image X, which will be input into the trained twin capsule network in the subsequent steps for vein recognition.

[0109] Before using the twin capsule network for vein recognition, the twin capsule network needs to be trained first to obtain a feature parameter extraction model. The determination method of the feature parameter extraction model comprises:

[0110] Step S8.1: obtaining a sample data set.

[0111] Step S8.2: inputting the sample three-channel images in the sample data set into the twin capsule network respectively to extract feature parameters, to obtain corresponding sample feature parameters, specifically including:

[0112] Step S8.2.1: performing convolution operation on each of the sample three-channel images respectively to obtain corresponding convolution results.

[0113] Step S8.2.2: performing capsule cutting on each of the convolution results respectively to obtain corresponding multiple capsules.

[0114] Step S8.2.3: performing nonlinear transformation on each of the capsules respectively to obtain corresponding transformation results.

[0115] Step S8.2.4: multiplying each of the transformation results with a weight matrix respectively to obtain corresponding vector results.

[0116] Step S8.2.5: clustering each of the vector results respectively, and taking the clustering center of each class as a feature vector.

[0117] Step S8.2.6: determining the feature matrix corresponding to each of the sample three-channel images according to multiple feature vectors.

[0118] Step S8.2.7: calculating the sample feature parameter corresponding to each of the sample three-channel images according to each of the feature matrices.

[0119] Step S8.3: training the twin capsule network with the Euclidean distance of the sample feature parameters of the sample three-channel images of the same individual being less than a first set value, and the Euclidean distance of the sample feature parameters of the sample three-channel images of different individuals being greater than a second set value as the target, to obtain a trained twin capsule network. Wherein, the first set value is less than the second set value.

[0120] Step S8.4: determining the trained twin capsule network as the feature parameter extraction model.

[0121] Specifically, the training process of the twin capsule network is as follows Figure 6As shown, the input of the network during training is two images, X1 and X2, both of which are 32x32x3. Generally, the two images are collected from the same part, such as the palm or the back of the hand. If the veins of the two images come from the same person, they are called homologous, and if they come from different people, they are called heterogeneous. The two images are subjected to the same operation in the network. First, a 256x9x9x3 convolution kernel is used to perform convolution operation on the image, obtaining a convolution result with a size of 24x24x256. Then, a 256x9x9x256 convolution kernel is used to perform convolution operation with a step size of 2, obtaining a convolution result with an output size of 8x8x256. This output is divided into 32 layers, obtaining 32 capsule layers, each with a size of 8x8x8, i.e., each layer contains 64 capsules with a size of 1x8, and 32 layers contain a total of 2048 capsules. After obtaining the capsules, a squash activation function is used to perform a nonlinear transformation on them:

[0122]

[0123] where S j is the input of the jth capsule, and V j is the output of the jth capsule. Each capsule is multiplied by a 8x16 weight matrix Wij to obtain a 16-dimensional vector. Therefore, after multiplying all the capsules by Wij, 2048 16-dimensional vectors are obtained. These vectors are clustered into 15 categories, and after iteration, the center of each category is selected as a feature, i.e., only 15 vectors are retained, obtaining a 15x16 matrix. This matrix is expanded into a 1x240 vector, and then a squash activation function and a softmax function are used for full connection to obtain a feature parameter with a size of 64x1. The feature parameters of X1 and X2 are f(X1) and f(X2), respectively. The Euclidean distance between the two feature parameters is calculated to adjust the convolution kernel and the weight matrix Wij of the network in reverse. MN images containing N categories (i.e., the database contains N people, and each person collects M hand vein images) X1, X2, …, X MN are trained in turn, and finally the Euclidean distance between the feature parameters of the homologous is as small as possible, and the Euclidean distance between the feature parameters of the heterogeneous is as large as possible.

[0124] Step S5: inputting the three-channel image to be identified into a feature parameter extraction model to extract a feature parameter to be identified; the feature parameter extraction model is obtained by training a twin capsule network using a sample data set; the sample data set includes a plurality of sample data groups, and different sample data groups correspond to different individuals; and one sample data group includes a plurality of sample three-channel images taken from the same individual.

[0125] Step S6: calculating target Euclidean distances between the to-be-identified feature parameter and sample feature parameters corresponding to each of the sample data sets; one of the sample feature parameters is obtained by inputting a sample three-channel image in the sample data set into the feature parameter extraction model to perform feature parameter extraction.

[0126] Further, the calculating of the target Euclidean distances between the to-be-identified feature parameter and the sample feature parameters corresponding to each of the sample data sets specifically includes:

[0127] Step S6.1: calculating Euclidean distances between the to-be-identified feature parameter and all sample feature parameters to obtain a plurality of sample Euclidean distances.

[0128] Step S6.2: calculating average values of the sample Euclidean distances of the same sample data set to obtain a plurality of target Euclidean distances.

[0129] Step S7: determining an individual corresponding to the target image set according to each of the target Euclidean distances.

[0130] Further, the determining of the individual corresponding to the target image set according to each of the target Euclidean distances specifically includes:

[0131] Step S7.1: comparing sizes of each of the target Euclidean distances to determine a minimum Euclidean distance.

[0132] Step S7.2: comparing the minimum Euclidean distance with a set threshold value.

[0133] Step S7.3: if the minimum Euclidean distance is greater than or equal to the set threshold value, determining that the target image set does not belong to the individual corresponding to the sample data set.

[0134] Step S7.4: if the minimum Euclidean distance is less than the set threshold value, determining that the target image set belongs to the individual corresponding to the sample data set, and determining the individual corresponding to the target image set as the individual corresponding to the minimum Euclidean distance.

[0135] Specifically, the image X obtained in step S4 is taken as an input of the twin capsule network at the end of training, and the feature parameter f(X) thereof is obtained through the same process. The Euclidean distances between f(X) and the M feature parameters obtained during training are calculated, and the average value of the Euclidean distances between the M feature parameters in each class and f(X) is taken as the average Euclidean distance (i.e., the target Euclidean distance) between X and the class. The class with the minimum average Euclidean distance less than the set threshold value ζ is the recognition result of the image X; if the average Euclidean distances are all greater than the set threshold value ζ, it is indicated that the image X is not in the N classes or is not a living body (i.e., a picture or other fake means).

[0136] The application further provides a hand vein recognition device with fused depth information, which is realized by the method and comprises the following components.

[0137] A vein collection device is configured to collect a target image set, wherein the target image set comprises two hand vein images with parallax.

[0138] An upper computer is connected to the vein collection device and configured to: pre-process the two hand vein images to obtain two vein texture images; match the two vein texture images to obtain a depth image; superimpose the depth image and the two vein texture images to obtain a three-channel image to be identified; input the three-channel image to be identified into a feature parameter extraction model to extract feature parameters to obtain feature parameters to be identified; the feature parameter extraction model is obtained by training a twin capsule network using a sample data set; the sample data set comprises a plurality of sample data groups, and different sample data groups correspond to different individuals; one sample data group comprises a plurality of sample three-channel images taken from the same individual; calculate target Euclidean distances between the feature parameters to be identified and sample feature parameters corresponding to each sample data group; one sample feature parameter is obtained by inputting a sample three-channel image in the sample data set into the feature parameter extraction model to extract feature parameters; and determine an individual corresponding to the target image set according to the target Euclidean distances.

[0139] Further, the upper computer comprises:

[0140] A vein segmentation module is configured to pre-process the two hand vein images to obtain two vein texture images.

[0141] An image matching module is configured to match the two vein texture images to obtain a depth image.

[0142] An image superimposition module is configured to superimpose the depth image and the two vein texture images to obtain a three-channel image to be identified.

[0143] A feature parameter extraction module is configured to input the three-channel image to be identified into a feature parameter extraction model to extract feature parameters to obtain feature parameters to be identified.

[0144] A target Euclidean distance calculation module is configured to calculate target Euclidean distances between the feature parameters to be identified and sample feature parameters corresponding to each sample data group.

[0145] A category determination module is configured to determine an individual corresponding to the target image set according to the target Euclidean distances.

[0146] As a specific embodiment, a structural diagram of the vein collection device is shown in Figure 2 The support structure comprises a fixed panel 1, a base 7 and a support 6.

[0147] The base 7 is arranged opposite to the fixed panel 1, and the support 6 is located between the fixed panel 1 and the base 7 and is perpendicular to the fixed panel 1 and the base 7 respectively. The base 7 is provided with a collection area 8. Two near-infrared cameras 3 are respectively arranged on the fixed panel 1 and are connected with the upper computer.

[0148] The collection area 8 is used for placing a hand part to be identified or a photo of the hand part to be identified. The two near-infrared cameras 3 are used for collecting a target image group of the hand part to be identified or the photo of the hand part to be identified.

[0149] Preferably, the vein collection device further comprises a ring-shaped near-infrared LED array 5 arranged on the fixed panel 1. The fixed panel 1 is provided with a sliding groove 2, and the two near-infrared cameras 3 are respectively fixed on the sliding groove 2, so that the distance between the two near-infrared cameras 3 can be changed by sliding on the sliding groove 2.

[0150] The existing vein recognition device has the problems of low recognition reliability and easy cracking. The present patent proposes a vein collection device containing binocular cameras to obtain vein images containing blood vessel depth information, and uses a twin capsule network to realize vein recognition, which is more efficient, more accurate and more resistant to cracking means such as pictures. Compared with the prior art, the present application has the following advantages:

[0151] High efficiency: the pre-processing algorithm of the hand vein image in the present application is fast; the twin capsule network used is a shallow convolutional neural network with fewer parameters and less calculation, so the training speed is fast and the recognition speed is also faster.

[0152] High accuracy: the present application can provide high-quality vein texture images by pre-processing the hand vein image, which helps the accurate identification of the twin capsule network; as a convolutional neural network, the twin capsule network can effectively extract the effective information in the three-channel image X for identification, and its pool layer is less than that of the general convolutional neural network, and the design of the capsule enables the 8 feature values in the capsule to store more image information, including position, size, direction, distortion, speed, albedo, hue, texture, etc., which helps more accurate identification.

[0153] Strong resistance to cracking means such as pictures: that is, stronger prosthesis attack detection capability. The present application calculates the depth of the hand vein image. Since the vein depth presented by the picture and other fake means is significantly different from the living body vein, the trained twin capsule network is used to extract the feature parameters, so as to perform vein recognition, and the veins presented by the picture and other fake means can be distinguished.

[0154] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the difference from other embodiments. The same or similar parts between the various embodiments can be referred to each other.

[0155] The principles and implementation modes of the present application are described by applying specific examples in this paper. The above description of the embodiments is only used to help understand the core idea of the present application. Meanwhile, for the general technical personnel in the field, according to the idea of the present application, the specific implementation mode and application range will be changed. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A hand vein recognition method fusing depth information, characterized in that, The method comprises: acquiring a target image group; the target image group comprises two hand vein images with parallax; respectively pre-processing the two hand vein images to obtain two vein texture images; matching the two vein texture images to obtain a depth image; superimposing the depth image and the two vein texture images to obtain a three-channel image to be identified; inputting the three-channel image to be identified into a feature parameter extraction model to extract feature parameters to obtain feature parameters to be identified; the feature parameter extraction model is obtained by training a twin capsule network using a sample data set; the sample data set comprises a plurality of sample data groups, and different sample data groups correspond to different individuals; one sample data group comprises a plurality of sample three-channel images taken from the same individual; calculating target Euclidean distances between the feature parameters to be identified and sample feature parameters corresponding to each sample data group; one sample feature parameter is obtained by inputting a sample three-channel image in the sample data set into the feature parameter extraction model to extract feature parameters; determining an individual corresponding to the target image group according to the target Euclidean distances. 2.The hand vein recognition method of fusing depth information according to claim 1, characterized in that, The method for determining the feature parameter extraction model comprises: obtaining a sample data set; inputting sample three-channel images in the sample data set into a twin capsule network to extract feature parameters to obtain corresponding sample feature parameters; training the twin capsule network to obtain a trained twin capsule network, with the target that the Euclidean distances between sample feature parameters of sample three-channel images of the same individual are less than a first set value, and the Euclidean distances between sample feature parameters of sample three-channel images of different individuals are greater than a second set value; wherein the first set value is less than the second set value; determining the trained twin capsule network as the feature parameter extraction model. 3.The hand vein recognition method of fusing depth information according to claim 1, characterized in that, The method for respectively pre-processing the two hand vein images to obtain two vein texture images comprises: respectively performing image data conversion processing on the two hand vein images to obtain two grayscale images; respectively performing region of interest extraction processing on the two grayscale images to obtain two region of interest images; respectively performing filter denoising processing on the two region of interest images to obtain two filtered images; respectively performing normalization processing on the two filtered images to obtain two normalized images; respectively performing image enhancement processing on the two normalized images to obtain two enhanced images; respectively performing vein segmentation processing on the two enhanced images to obtain two vein texture images. 4.The hand vein recognition method of fusing depth information according to claim 3, characterized in that, The method for respectively performing vein segmentation processing on the two enhanced images to obtain two vein texture images comprises: respectively extracting center points of hand veins from the two enhanced images using a maximum curvature algorithm to obtain two vein center point images; connecting all center points of hand veins in each vein center point image to obtain two vein connection images; respectively performing binarization processing on the two vein connection images to obtain two vein texture images. 5.The hand vein recognition method of fusing depth information according to claim 2, characterized in that, The sample three-channel images in the sample data set are respectively input into a twin capsule network for feature parameter extraction, and corresponding sample feature parameters are obtained, specifically including: Each of the sample three-channel images is subjected to convolution operation to obtain a corresponding convolution result; Each of the convolution results is subjected to capsule cutting to obtain a plurality of corresponding capsules; Each of the capsules is subjected to a nonlinear transformation to obtain a corresponding transformation result; Each of the transformation results is multiplied by a weight matrix to obtain a corresponding vector result; Each of the vector results is subjected to clustering, and the clustering center of each class is taken as a feature vector; According to the plurality of feature vectors, a feature matrix corresponding to each of the sample three-channel images is determined; According to each of the feature matrices, a sample feature parameter corresponding to each of the sample three-channel images is calculated. 6.The hand vein recognition method of fusing depth information according to claim 1, wherein, The target Euclidean distance between the to-be-identified feature parameter and the sample feature parameters corresponding to each of the sample data groups is calculated, specifically including: The Euclidean distances between the to-be-identified feature parameter and all sample feature parameters are calculated to obtain a plurality of sample Euclidean distances; The average value of each of the sample Euclidean distances of the same sample data group is calculated to obtain a plurality of target Euclidean distances. 7.The hand vein recognition method of fusing depth information according to claim 1, characterized in that, According to each of the target Euclidean distances, the individual corresponding to the target image group is determined, specifically including: The sizes of each of the target Euclidean distances are compared to determine the minimum Euclidean distance; The size of the minimum Euclidean distance and the set threshold value are compared; If the minimum Euclidean distance is greater than or equal to the set threshold value, it is determined that the target image group does not belong to the individual corresponding to the sample data set; If the minimum Euclidean distance is less than the set threshold value, it is determined that the target image group belongs to the individual corresponding to the sample data set, and the individual corresponding to the target image group is determined to be the individual corresponding to the minimum Euclidean distance.

8. A hand vein recognition apparatus that fuses depth information, characterized by, The device adopts the method of any one of claims 1-7, and the device comprises: A vein collection device for collecting a target image group; the target image group comprises two hand vein images with parallax; A host computer connected with the vein collection device; the host computer comprises: A vein segmentation module for respectively pre-processing the two hand vein images to obtain two vein texture images; An image matching module for matching the two vein texture images to obtain a depth image; An image superposition module for superimposing the depth image and the two vein texture images to obtain a to-be-identified three-channel image; A feature parameter extraction module for inputting the to-be-identified three-channel image into a feature parameter extraction model for feature parameter extraction to obtain a to-be-identified feature parameter; A target Euclidean distance calculation module for calculating the target Euclidean distance between the to-be-identified feature parameter and the sample feature parameters corresponding to each of the sample data groups; A category determination module for determining the individual corresponding to the target image group according to each of the target Euclidean distances. 9.The hand vein recognition apparatus with fused depth information of claim 8, wherein, The vein collection device comprises: Two near-infrared cameras are connected with the host computer respectively, and used for collecting target image groups of the hand parts to be identified or photos of the hand parts to be identified. 10.The hand vein recognition device with fused depth information of claim 9, wherein, The venous collection device further comprises: A support structure, comprising a fixed panel, a base and a support; The base is arranged opposite to the fixed panel, and the support is located between the fixed panel and the base and is perpendicular to the fixed panel and the base respectively; the base is provided with a collection area; the two near-infrared cameras are arranged on the fixed panel respectively; and the collection area is used for placing the hand parts to be identified or the photos of the hand parts to be identified.

Images