A palmprint image recognition method based on small sample metric network

Abstract

The application discloses a palmprint image recognition method based on a small sample metric network, belongs to the cross field of biometric recognition and computer vision, and comprises the following steps: dividing a training set and a test set in a palmprint dataset by using a small sample learning mode, and dividing the training set and the test set into several tasks; constructing a light small sample metric network, and extracting a feature vector by using the network; constructing a metric formula of the feature vector, and measuring the distance of the feature vector by using the formula; designing a loss function of a negative log likelihood based on a metric distance, and accumulating the distance calculated by using the distance formula into a corresponding loss.The application introduces small sample learning and an improved signal-to-noise ratio distance to train a model, solves the problem of few training samples, and obtains a high recognition rate.

Description

TECHNICAL FIELD

[0001] The application belongs to the cross field of biometric recognition and computer vision, and particularly relates to a palmprint image recognition method based on a small sample metric network. BACKGROUND

[0002] Biometric recognition technology is to identify the identity of a person by extracting inherent features of the human body, such as fingerprints, irises, faces, veins, palmprints or ears; among which, palmprints contain rich features such as main lines, ridge lines, wrinkles and other detailed features, and with the development of computer performance, big data and deep learning, palmprint recognition methods based on deep learning have gradually emerged.

[0003] For traditional palmprint recognition algorithms, a special feature extractor needs to be designed artificially, which usually requires a lot of work, and traditional deep learning-based methods require a large number of training samples and corresponding labels to ensure that the trained model does not overfit; most palmprint recognition algorithms based on deep learning require a large number of samples to train the model; from the actual point of view, considering the privacy problem of users, it is often difficult to collect a large number of samples, and it also requires a lot of work to label these samples; although some algorithms combine traditional palmprint recognition methods to reduce the dependence on labeled samples, these algorithms often require researchers' prior experience and have certain subjective factors, which often makes the model complex and increases the calculation difficulty. SUMMARY

[0004] The purpose of the present application is to provide a palmprint image recognition method based on a small sample metric network, which is a light neural network model, and the model is trained based on an improved signal-to-noise ratio distance to realize, in the case of small samples, deep learning and metric learning methods to reduce the computational complexity, extract stable palmprint recognition features for high-precision matching and recognition.

[0005] A palmprint image recognition method based on a small sample metric network is provided, comprising the following steps:

[0006] S1: Collect and extract palmprint image ROIs, organize corresponding data sets, and divide the training set and the test set in the palmprint data set using the small sample learning method, and divide them into several tasks;

[0007] S2: Construct a light small sample metric network, and use the network to extract a feature vector;

[0008] S3: Obtain the feature vector corresponding to each palmprint ROI image through S2;

[0009] S4: construct a distance formula of the feature vector, and use the formula to measure the distance of the feature vector;

[0010] S5: design a loss function based on the negative log-likelihood of the measured distance, update the model parameters in the training network through the loss function, and calculate the accuracy of the test set and the recognition rate of the query set;

[0011] S6: save and deploy the network model trained in S5 to the actual running environment.

[0012] As a further scheme of the application, S1 specifically comprises the following steps:

[0013] S1.1: collect palm image and use specific preprocessing and image segmentation method to extract palm print image ROI and arrange corresponding data set, data set is randomly divided into training set and test set according to 1:1 ratio, and the labels between training set and test set are not intersected;

[0014] S1.2: divide the divided training set and test set into h tasks according to n-shot k-way;

[0015] There are k classes in the support set, each class has n samples, and the value range of n is 1 to the maximum number of samples of any class, and the value range of k is 1 to the maximum number of classes;

[0016] The query set is q-shot k-way, and the query set is divided in the same way as the support set, except that the value of shot may be different, wherein the selection method of the divided task is to randomly select k classes in the training set or test set, and then randomly select n samples or q samples in the k classes.

[0017] As a further scheme of the application, S2 specifically comprises the following steps:

[0018] S2.1: the structure of the small sample measurement network is composed of 4 convolution modules and 1 dimension reduction layer;

[0019] S2.2: each module of the 4 convolution modules has corresponding input channel number and output channel number, the structure of each module includes a two-dimensional convolution layer, the size of the convolution kernel of the two-dimensional convolution layer is 3*3, the number of convolution kernels is 64, and the size of the padding pixel of the convolution is 1; followed by a batch normalization layer; then a ReLU activation function; the last layer is a maximum pooling layer, the window size is 2*2, and the step is 2;

[0020] S2.3: the dimension reduction layer converts the high-dimensional feature map after the 4 convolution modules into a one-dimensional feature vector;

[0021] S2.4: randomly initializing the network structure parameters according to the constructed network structure;

[0022] S2.5: the entire network model can be expressed as:

[0023] ;

[0024] wherein the input image is , the network model is f, the parameters in the network model are p, and the feature vector output by the network model is .

[0025] As a further scheme of the present application, the S3 specifically comprises the following steps:

[0026] S3.1: converting each palm print ROI image into an RGB image in the format of 128*128*3, and inputting the image into the small sample metric network of S2 to obtain a corresponding feature vector;

[0027] S3.2: according to the small sample learning method in S1, for each task, the support set can obtain feature vectors, and the query set can obtain feature vectors, that is, there are n feature vectors in each of the k or q classes.

[0028] As a further scheme of the present application, the S4 specifically comprises the following steps:

[0029] According to the feature vectors obtained in S3 and the small sample learning task mode defined in S1, the metric distance between the feature vectors in the query set of each task and the feature vectors in the support set is calculated, and the distance formula is as follows:

[0030] ;

[0031] wherein is the feature vector obtained in S3, and represent the distance coefficients, the function is used to represent the variance of a certain feature vector, and the function is defined as follows:

[0032] ;

[0033] wherein represents the mean of the feature vector , and n represents the dimension of the feature vector .

[0034] As a further scheme of the present application, the S5 specifically comprises the following steps:

[0035] S5.1: According to the metric distance calculated in S4 and the small sample learning method in S1 and the divided training set, the model parameters in the training network are updated using the following loss function:

[0036] ;

[0037] represents the support set in the training task, represents the query set in the training task, represents that the anchor picture x of the support set and the query picture y of the query set are of the same class, and represents that they are not of the same class, represents that the LogSoftmax function is calculated for the distance between the anchor picture and the query picture, the LogSoftmax function is calculated for the anchor picture in the support set and the query picture in the query set, and finally the loss value is accumulated, and the loss value is used to update and optimize the model parameters;

[0038] S5.2: According to the test set divided in S1, the accuracy on the test set needs to be calculated, and according to the distance formula in S4, the distance between all pictures in the query set and each picture in the support set is calculated, and the shortest distance principle is used to determine whether the matching is successful, and finally the recognition rate corresponding to the matching result is calculated.

[0039] As a further scheme of the present application, the optimizer used to optimize the model parameters in S5.1 is Adam optimizer, and the learning rate is 0.01.

[0040] As a further scheme of the present application, the S6 specifically comprises the following steps:

[0041] S6.1: According to the network model parameters trained in S5, the network model parameters are saved and deployed to the actual running environment;

[0042] First, a palm print database is established, and the corresponding feature vector is extracted using the trained model, and the feature vector is saved in the database or the memory, so as to facilitate calculation and matching;

[0043] S6.2: The anchor picture matched with x is obtained by the formula as follows:

[0044] ;

[0045] Where x is the picture to be matched, s represents the support set in the test task, a is the anchor picture, and a is the picture in the support set as described above, and p is the network model parameter after training.

[0046] Compared with the prior art, the present application has the beneficial effects that:

[0047] The present application introduces small sample learning and improved signal-to-noise ratio distance to train the model, and in the case of small sample, based on deep learning and metric learning method, reduces the computational complexity, extracts stable palmprint recognition features for high-precision matching and recognition, solves the problem of few training samples, and obtains high recognition rate. BRIEF DESCRIPTION OF DRAWINGS

[0048] In order to facilitate those skilled in the art to understand, the present application will be further described below in conjunction with the drawings.

[0049] Figure 1 A flowchart of a palmprint image recognition method based on a small sample metric network;

[0050] Figure 2 A palm acquisition flowchart provided by the present application;

[0051] Figure 3 A data set division flowchart provided by the present application;

[0052] Figure 4 A model training and testing flowchart provided by the present application;

[0053] Figure 5 A palmprint matching flowchart provided by the present application. DETAILED DESCRIPTION

[0054] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the drawings and examples; it should be understood that the specific examples described herein are only used to explain the present application, and are not used to limit the present application, that is, the described examples are only a part of the embodiments of the present application, but not all the embodiments; the components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0055] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed application, but only represents selected embodiments of the present application; based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0056] Please refer to Figures 1-5 The present application specifically includes the following steps:

[0057] Step one: collection and division of palmprint dataset; the dataset of the application is divided by small sample learning, small sample learning refers to learning or understanding new concepts or categories through a small number of samples, and is carried out through one task after another, the data set can be divided into support set and query set in the task, the support set refers to the data set used for matching in the task, the query set refers to the data set used for querying in the task, the data in the query set needs to be matched with the data in the support set; the support set used in the training task is defined as , the query set used in the training is defined as , the support set used in the test task is defined as S, and the query set is defined as Q; each task can be n-shot and k-way, k-way refers to k classes of samples in the support set, n-shot refers to n samples in each class in the support set, so the support set has samples; in the query set, the number of samples in each class is defined as q, i.e. q-shot, so the query set has samples.

[0058] It should be noted that in small sample learning, the data and used for training and the data S and Q used for testing are independent label spaces, and the two are mutually exclusive; palm image is collected and specific preprocessing and image segmentation method is used to extract palmprint image ROI and arrange corresponding data set, the data set is randomly divided into training set and test set according to the ratio of 1:1; the divided training set and test set are further divided according to the small sample learning method, i.e. divided into h tasks according to the n-shot k-way method, each task is divided into support set and query set, the support set has k classes, each class has n samples, the value range of n is 1 to the maximum number of samples of any class, the value range of k is 1 to the maximum number of classes, the query set is q-shot k-way, and the division method is the same as that of the support set, only the value of shot may be different, wherein the selection method of the divided task is to randomly select k classes in the training set or test set, and then randomly select n samples or q samples in the k classes.

[0059] Step two: establish a small sample measurement network; most small sample learning models use deep convolutional layers to extract image features, and the application reduces the number of convolutional layers while ensuring stable feature extraction; the structure of the small sample measurement network of the application is composed of 4 convolutional modules and 1 dimension reduction module, each module of the 4 convolutional modules has a corresponding input channel number and output channel number, the structure of each module includes a two-dimensional convolutional layer, the size of the convolution kernel is 3x3, the number of convolution kernels is 64, the size of the padding pixels is 1, followed by a batch normalization layer, then a ReLU activation function, and the last layer of the convolutional module is a maximum pooling layer; the function of the dimension reduction module is to convert the high-dimensional feature map after the 4 convolutional modules into a one-dimensional feature vector.

[0060] Step three: palmprint image feature extraction; for each palmprint ROI image, input the 128x128x3 format RGB image into the network of step two to obtain the corresponding feature vector; according to the small sample learning method of step one, for each task, the support set can obtain feature vectors, and the query set can obtain feature vectors, that is, there are n or q feature vectors in each of the k classes; assuming that the input image is , the network model is f, the parameters in the network model are p, and the feature vector output by the network model is , then the following formula is obtained: .

[0061] Step four: feature measurement calculation; in the theory of statistics, the definition of signal-to-noise ratio is the ratio of signal variance to noise variance, according to the definition of signal-to-noise ratio in statistics, the signal-to-noise ratio distance between feature vectors can be defined as:

[0062] ;

[0063] Wherein , represents the variance of the feature vector , represents the mean of the feature vector , and n represents the dimension of the feature vector .

[0064] Similar to the traditional Euclidean distance, if the palmprint images of the same person correspond to feature vectors with a relatively close signal-to-noise ratio distance, and if the palmprint images of different people correspond to feature vectors with a relatively far signal-to-noise ratio distance, according to the above constraints, the following distance function formula is obtained:

[0065] ;

[0066] But compared with the traditional Euclidean distance, the signal-to-noise ratio distance does not satisfy the commutative law, that is The signal-to-noise ratio distance is sensitive to which feature vector is an anchor point; in small sample learning, the pictures in the query set need to be matched one by one with the pictures in the support set, so the feature vectors corresponding to the pictures in the support set can be used as anchor points, and then the distance between the feature vectors of each image in the query set and the feature vectors of each image in the support set can be calculated; in order to improve the applicability of the signal-to-noise ratio distance, according to the characteristics of the signal-to-noise ratio distance, the present application proposes to use the feature vectors of each image in the query set and the support set as anchor points, that is, when calculating the anchor points of two feature vectors, the signal-to-noise ratio distance when one of the two feature vectors is used as an anchor point is calculated respectively, and then multiplied by the corresponding distance coefficient and added as the distance between the two vectors; according to the feature vectors obtained in step four and the small sample learning task mode defined in step one, for each task, the feature vectors in the query set are calculated with the feature vectors in the support set to measure the distance, and the total formula is as follows:

[0067] ;

[0068] That is

[0069] ;

[0070] Wherein, and represent the distance coefficient.

[0071] Step five: network training and testing; according to the measurement distance calculated in step four and the small sample learning method in step one and the training set divided out, in order to update the model parameters in the training network, the present application uses the following loss function to update the model:

[0072] ;

[0073] Wherein

[0074] ;

[0075] In the loss L, indicates that the anchor point picture x in the support set and the query picture y in the query set are of the same class, and indicates that they are not of the same class; indicates that the LogSoftmax function is calculated for the distance between the anchor point picture and the query picture, and the LogSoftmax function is calculated for the anchor point picture in the support set and the query picture in the query set, and the loss value can be obtained by accumulating the LogSoftmax function, and the loss value is used to update the optimization model parameters; the optimizer used in the present application is Adam optimizer, and the learning rate is 0.01.

[0076] According to the test set divided in step one, the accuracy on the test set needs to be calculated, and according to the distance formula in step four, the distance between all pictures in the query set and each picture in the support set is calculated, and the shortest distance principle is used to determine whether the matching is successful, and finally the corresponding recognition rate is calculated according to the matching result.

[0077] Step six: palmprint matching; in step three, the network model parameters calculated in step two are used, and the shortest distance in the support set is selected as the matching object during matching, and the matching formula is as follows:

[0078] ;

[0079] Where x is the input picture, since it is selected from the query set , is the anchor picture, as described above is the element in the support set, and p is the parameter of the model after training, and the formula finally obtains the anchor picture matched with x.

[0080] In one embodiment:

[0081] S1: Collect palm image, extract palmprint ROI area through specific palmprint segmentation method, classify and count palmprint ROI area image samples, wherein the size of the palmprint ROI area image is 128x128, and the process is as shown in Figure 2 .

[0082] S2: Divide the collected palmprint image into training set and test set according to the ratio of 1:1, and the classes are not crossed.

[0083] S3: The training set and the test set are divided in the manner of 1-shot 10-way, 1-shot 15-way, 5-shot 10-way, and 5-shot 15-way, that is, 10 classes or 15 classes are randomly selected on the training set and the test set, and 1 or 5 samples are randomly selected in each class as the support set, and the remaining samples are selected as the query set; wherein in each iteration period, the training set is divided into 100 tasks, and the test set is divided into 200 tasks, and there are 50 iteration periods in total, and the process is as shown in Figure 3 .

[0084] S4: Input each task into the small sample metric network, and sequentially pass through 4 layers of convolution module and 1 layer of dimension reduction module, the convolution module is composed of 64 3x3 convolution kernels, batch normalization layer, ReLU activation function layer, and 2x2 maximum pooling layer, and then the high-dimensional feature map is converted into a feature vector through the dimension reduction module.

[0085] S5: training and testing of the model; for each task, the feature vectors obtained after inputting the support set and query set pictures into the network are used to calculate the corresponding metric distance according to the above formula, the corresponding loss value is calculated according to the calculated metric distance and the above formula, and the model parameters are optimized using the Adam algorithm and a learning rate of 0.01, and after 50 cycles, the optimized model parameters are obtained, and the process is as shown in Figure 4

[0086] For each task, the feature vectors obtained after inputting the support set and query set pictures into the network are used to calculate the corresponding metric distance according to the above formula, and the shortest distance principle is used to calculate the recognition accuracy, and the process is as shown in Figure 4

[0087] S6: deploy the trained model to the corresponding production environment, establish the corresponding palmprint database, use the images of the palmprint database as the support set, and use the palmprint images to be recognized as the query set, and calculate the recognition result according to the above matching formula, that is, the shortest distance principle, and the process is as shown in Figure 5

[0088] Through experiments, the data experimental results of the palmprint dataset of the present application are as follows:

[0089]

[0090] The above content is only an example and description of the structure of the present application, and those skilled in the art can make various modifications or supplements to the described specific embodiments or use similar ways to replace them, as long as they do not deviate from the structure of the present application or exceed the scope defined by the present application, and they should belong to the protection scope of the present application.​​​

Claims

1. A palmprint image recognition method based on a few-sample metric network, characterized in that, Includes the following steps: S1: Collect and extract the ROI of palm print images, organize them into corresponding datasets, and use few-shot learning to divide the training set and test set in the palm print dataset into several tasks. S2: Construct a lightweight few-sample metric network and use this network to extract feature vectors; S3: Obtain the feature vector corresponding to each palmprint ROI image through S2; S4: Construct the distance formula for feature vectors and use this formula to measure the distance between feature vectors; S5: Design a loss function based on the negative log-likelihood of distance, update the model parameters in the training network through the loss function, and calculate the accuracy of the test set and the recognition rate of the query set. S6: Based on the network model parameters trained in S5, save the network model and deploy it to the actual operating environment; S1 specifically includes the following steps: S1.1: Collect palm images and use specific preprocessing and image segmentation methods to extract the ROI of the palm print images and organize them into corresponding datasets. The datasets are randomly divided into training and test sets in a 1:1 ratio, and the labels of the training and test sets are disjoint. S1.2: Divide the already partitioned training and test sets into n-shot k-way sets. There are 10 tasks, and each task is divided into a support set and a query set. Support set includes There are classes, each class has One sample, The value range is from 1 to the maximum number of samples in any class. The value ranges from 1 to the maximum number of categories; The query set is a q-shot k-way set, and the way the query set is divided is the same as that of the support set, except that the value of the corresponding shot may be different. The task for partitioning is selected randomly from the training set or the test set. This class, Randomly select from each category A sample or One sample; S3 specifically includes the following steps: S3.1: Convert each palmprint ROI image into a 128×128×3 RGB image and input it into the few-sample metric network in S2 to obtain the corresponding feature vector; S3.2: Based on the few-shot learning method in S1, the support set can be obtained for each task. The query set can obtain feature vectors. 1 eigenvector, i.e. or Each of the classes has 1 eigenvector.

2. The palmprint image recognition method based on a few-sample metric network according to claim 1, characterized in that, S2 specifically includes the following steps: S2.1: The structure of the small sample metric network consists of 4 convolutional layers and 1 dimensionality reduction layer; S2.2: Each of the four convolutional modules has a corresponding number of input channels and output channels. The structure of each module includes a two-dimensional convolutional layer with a kernel size of 3×3 and 64 kernels, and a padding pixel size of 1. Next is a batch normalization layer; followed by a ReLU activation function; and finally, a max pooling layer with a window size of 2×2 and a stride of 2. S2.3: The dimensionality reduction layer transforms the high-dimensional feature map, which has passed through 4 convolutional modules, into a 1-dimensional feature vector; S2.4: Randomly initialize the network structure parameters according to the constructed network structure; S2.5: The entire network model can be represented as: ； The input image is The network model is The parameters in the network model are The feature vector output by the network model is .

3. The palmprint image recognition method based on a few-sample metric network according to claim 2, characterized in that, S4 specifically includes the following steps: Based on the feature vectors obtained in S3 and the few-shot learning task method defined in S1, the query set for each task... Each feature vector and the support set The distance metric is calculated from the feature vectors, using the following formula: ； in The feature vector obtained in S3, and Represents the distance coefficient, function The function used to represent the variance of a certain eigenvector The definition is as follows: ； in Representing the eigenvector The mean, Representing the eigenvector Dimensions.

4. The palmprint image recognition method based on a few-sample metric network according to claim 3, characterized in that, S5 specifically includes the following steps: S5.1: Based on the metric distance calculated in S4 and the few-shot learning method and the partitioned training set in S1, the model parameters in the training network are updated using the following loss function: ； This represents the support set in the training task. This represents the query set in the training task. Anchor image representing the support set Query images of query sets They belong to the same category, and This indicates that they are not of the same type. This represents the LogSoftmax function used to calculate the distance between the anchor image and the query image. The LogSoftmax function is calculated between the anchor images in the support set and the query images in the query set, and finally accumulated as the loss value. This loss value is used to update and optimize the model parameters. S5.2: Based on the test set defined in S1, the accuracy on the test set needs to be calculated. Then, according to the distance formula in S4, the distance between all images in the query set and each image in the support set is calculated. The principle of shortest distance is used to determine whether the match is successful. Finally, the corresponding recognition rate is calculated based on the matching results.

5. The palmprint image recognition method based on a few-sample metric network according to claim 4, characterized in that, The optimizer used to optimize the model parameters in S5.1 is the Adam optimizer, with a learning rate of 0.

01.

6. The palmprint image recognition method based on a few-sample metric network according to claim 4, characterized in that, S6 specifically includes the following steps: S6.1: Based on the network model parameters trained in S5, save the network model parameters and deploy them to the actual operating environment; First, a palmprint database needs to be established. The trained model is then used to extract the corresponding feature vectors. These feature vectors are stored in the database or in memory for easy calculation and matching. S6.2: Obtain the result through the formula. The matching formula for anchor point images is as follows: ； in These are the images that need to be matched. This represents the support set in the test task. It is an anchor point image, and as described in the formula above. It supports centralized images. These are the parameters of the network model after training.

Images