Multispectral palmprint recognition method, system and device based on fusion extreme learning machine

Abstract

The application discloses a multispectral palmprint recognition method, system and device based on a fusion extreme learning machine, relates to the palmprint recognition technical field, and comprises the following steps: inputting a multispectral palmprint image to be recognized into a multispectral palmprint recognition network model to obtain palmprint identity information; wherein the multispectral palmprint recognition network model is obtained by training a fusion extreme learning machine through a joint loss function and multiple training data; the fusion extreme learning machine comprises multiple extreme learning machine network models and an identity recognition module connected with the output end of each extreme learning machine network model; different extreme learning machine network models are used for inputting palmprint images of different spectral bands; and the joint loss function is a loss function composed of a network structure loss function, a multispectral fusion prediction loss function and a single-spectrum prediction loss function. The application can recognize palmprint identity information with high precision.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of palmprint recognition, in particular to a multispectral palmprint recognition method, system and device based on a fusion extreme learning machine. BACKGROUND

[0002] As a new biometric feature recognition method, palmprint recognition has been widely used in access control, electronic payment and attendance management fields, and is considered to be one of the most effective identity authentication schemes. Palmprint is composed of main lines, wrinkles and ridges, and has very stable structure and basic information for identity recognition. Compared with other biometric feature recognition technologies, palmprint recognition has the advantages of low distortion, user-friendly, non-invasive and high uniqueness, and is therefore more convenient, safe and reliable in real application, and has formed a relatively independent research field.

[0003] Most of the traditional palmprint researches use a natural light imaging system to obtain the palmprint in grayscale format. However, in actual application scenarios, the palmprint information collected under a single spectrum is not rich enough. Moreover, in some extreme application scenarios, when the palmprint collection process is disturbed by noise, such as uneven illumination, Gaussian noise and salt and pepper noise, the model is prone to calculation deviation, which causes the palmprint recognition system to have poor recognition performance in the test scenario of unknown identity. SUMMARY

[0004] The purpose of the present application is to provide a multispectral palmprint recognition method, system and device based on a fusion extreme learning machine, which can accurately recognize palmprint identity information.

[0005] To achieve the above purpose, the present application provides the following solutions.

[0006] In a first aspect, the present application provides a multispectral palmprint recognition method based on a fusion extreme learning machine, comprising:

[0007] obtaining a multispectral palmprint image to be recognized;

[0008] inputting the multispectral palmprint image to be recognized into a multispectral palmprint recognition network model to obtain palmprint identity information;

[0009] The multispectral palmprint recognition network model is obtained by training a fusion extreme learning machine through a joint loss function and multiple training data.

[0010] In a second aspect, the present application provides a multispectral palmprint recognition system based on a fusion extreme learning machine, comprising:

[0011] A multispectral palmprint image to be identified acquisition module is configured to acquire a multispectral palmprint image to be identified.

[0012] A palmprint identity information recognition module is configured to input the multispectral palmprint image to be identified into a multispectral palmprint recognition network model to obtain palmprint identity information.

[0013] The multispectral palmprint recognition network model is obtained by training a fusion extreme learning machine through a joint loss function and multiple training data.

[0014] In a third aspect, the present application provides an electronic device comprising a memory and a processor, wherein the memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to perform the multispectral palmprint recognition method based on a fusion extreme learning machine according to the first aspect.

[0015] According to the embodiments of the present application, the following technical effects are achieved:

[0016] Since the multispectral fusion prediction loss function can optimize learning the complementary relationship of multispectral palmprint features, the single-spectrum prediction loss function can enable the model to extract more fine palm feature information, and the network structure loss function can enable the model to improve the generalization ability, therefore, the multispectral palmprint recognition network model obtained by training the fusion extreme learning machine through the joint loss function including the network structure loss function, the multispectral fusion prediction loss function and the single-spectrum prediction loss function has better robustness, so that the multispectral palmprint recognition network model has good recognition performance in the test scene of unknown identity. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed to be used in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0018] Figure 1 The execution flowchart of the multispectral palmprint recognition method based on the fusion extreme learning machine provided by the embodiments of the present application is shown in the figure.

[0019] Figure 2 The structural schematic diagram of the extreme learning machine network model provided by the embodiments of the present application is shown in the figure.

[0020] Figure 3 The overall flowchart of the multispectral palmprint recognition framework based on the fusion extreme learning machine provided by the embodiments of the present application is shown in the figure.

[0021] Figure 4 The flowchart of the multispectral palmprint recognition method based on the fusion extreme learning machine provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0022] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some embodiments of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the scope of protection of the present application.

[0023] The main work of the present application is to design a limit learning machine fusion framework for multispectral palmprint recognition. The core includes designing an optimization objective based on a joint loss, which includes a structure loss, a multispectral fusion prediction loss, and a single spectrum prediction loss. With the above joint loss, the multispectral limit learning machine model uses the collaborative optimization of the network model to extract the complementary information of the multispectral palmprint and the discriminative palmprint features in the single spectrum to comprehensively predict the object identity information.

[0024] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below in combination with the drawings and specific embodiments.

[0025] Embodiment one

[0026] For the problems involved in the background art, the existing solution strategy mostly uses multispectral images to replace single spectrum images, increases different spectral information, so as to improve the recognition accuracy. The core problem of applying multispectral image recognition palmprint information is how to fully utilize multi-source information. The current method can be divided into two directions, i.e. performing spectral information fusion at the image level or performing spectral information fusion at the matching score level. In the spectral information fusion strategy based on image level, multispectral images are integrated into a single image, and palm features of the fusion image are extracted. In the spectral information fusion strategy at the matching score level, palmprint features are extracted from different spectral bands respectively, the matching scores are obtained by the comparator, and then the fusion is performed according to different fusion rules, and finally the fusion result is verified. Compared with the former strategy, the spectral information fusion strategy at the matching score level can more fully utilize multispectral information, and flexibly integrate features by setting fusion rules. Therefore, according to the spectral information fusion strategy at the matching score level, the present embodiment provides a robust limit learning machine fusion model, i.e. a multispectral palmprint recognition network model, specifically: the present embodiment generates spectral contribution weights according to the hidden layer output features of the limit learning machine network model, and then reconstructs the multi-source matching scores and fuses them according to the spectral contribution weights, so as to comprehensively train the limit learning machine network model by fusing prediction errors and spectral prediction errors, obtain the multispectral palmprint recognition network model, and apply the multispectral palmprint recognition network model to recognize identity information, so as to improve the robust performance of the recognition system to noise.

[0027] The execution process of the multispectral palmprint recognition method based on the fusion limit learning machine provided by the present embodiment is as follows Figure 1The limit learning machine network model is constructed for different spectral bands of the multispectral palmprint image, and the palmprint image is inputted to calculate the hidden layer output features of the limit learning machine network model. According to the distribution characteristics of the hidden layer output features, the spectral contribution weight is generated. According to the single-spectrum prediction loss of the limit learning machine, the multispectral fusion prediction loss and the network structure loss, the network parameters of the limit learning machine network model are calculated in a multispectral collaborative manner, so as to obtain a multispectral palmprint recognition network model. Finally, the multispectral palmprint recognition network model is used for fusion prediction of the palmprint identity.

[0028] Specifically, the limit learning machine (ELM) network model is used as the basic model of the embodiment. The ELM network model is a learning method for single-hidden layer feedforward neural networks (SLFNs) and is widely used due to its fast learning speed and significant generalization ability. Unlike the traditional neural network which needs to repeatedly adjust parameters, the ELM network model only needs to set the number of hidden nodes, and does not need to adjust the weights of the input layer and the bias of the hidden layer. The weights of the output layer are determined by the least square algorithm.

[0029] The ELM network model can approximate any function and is more likely to obtain a global optimal solution rather than a local optimal solution. The structure of the ELM network model is as shown in Figure 2 wherein n, L and m are the number of input nodes, hidden nodes and output nodes respectively. α l = [α l1 , α l2 ,..., α ln ] T is the weight vector connecting the input nodes to the lth hidden node. b l is the bias of the lth hidden node. Similarly, β l = [β l1 , β l2 ,..., β lm ] T is the weight vector connecting the lth hidden node to the output node.

[0030] For N training data D = {(x i , t i ), i = 1, 2,..., N}, x i = {x i1 , x i2 ,..., x in} T ∈ R n is an input vector with a length of n, and t i = {t i1 , t i2 ,..., t im} T ∈ R mis one-hot encoding with length m. The mapping from input x i to output t i is represented as follows:

[0031]

[0032] where g(x) is the activation function of the hidden layer, the hidden layer output feature h i and the network weight β are defined as follows:

[0033]

[0034] β = [β1, β2,..., β L ] T (3).

[0035] The above formula is matrixed as:

[0036] Hβ = T (4).

[0037] where:

[0038]

[0039] T = [t1, t2,..., t N ] T (6).

[0040] H represents the output matrix of the hidden layer when N inputs {x i , i = 1, 2,..., N} enter the hidden layer.

[0041] The training of the ELM network model is to search for appropriate network parameters Each network parameter is defined as follows:

[0042]

[0043] The ELM network model theory shows that if the activation function g(x) is infinitely differentiable in any interval, then the solution of the above formula can be realized in three steps:

[0044] Step 1: Randomly set network parameters where l = 1, 2,..., L;

[0045] Step 2: Obtain the hidden layer output matrix H according to formula (5);

[0046] Step 3: Determine the output weight matrix β by where is the Moore-Penrose generalized inverse matrix of H.

[0047] The original ELM network model shows low generalization ability to the change of test data, therefore, the matrix norm of β is used to represent the complexity of the ELM network model, the smaller the value is, the stronger the generalization ability of the model is. Thus, the parameter optimization target is adjusted as follows:

[0048]

[0049] where C is a trade-off parameter between prediction error and generalization ability.

[0050] Since is a random allocation, the parameter optimization target is converted into a quadratic convex optimization problem which can be solved by , so the output weight is calculated as follows:

[0051]

[0052] where I N and I L are unit matrices with dimensions N and L respectively.

[0053] For the multispectral palmprint recognition problem, the embodiment provides a multispectral palmprint recognition framework based on a fusion extreme learning machine as shown in Figure 3 The framework optimizes the joint loss composed of network structure loss, multispectral fusion prediction loss and single spectrum prediction loss. The multispectral fusion prediction loss is used to optimize the complementary relationship of multispectral palmprint features, the single spectrum prediction loss is used to make the model extract more fine palm feature information, and the network structure loss is used to improve the generalization ability of the model. By combining the three kinds of losses, the multispectral palmprint recognition framework provided in the embodiment realizes the balance between the complementarity and uniqueness of multispectral features.

[0054] Specifically, the embodiment distinguishes different spectral bands by superscript (s), s = 1, 2,..., S, S is the number of spectral bands. is a multispectral palmprint image training set, where, represents the i-th palmprint sample of the s-th spectral band, t i is the corresponding one-hot identity label vector. According to the additive weighting rule, the matching score level fusion output is as follows:

[0055]

[0056] where, is a hidden layer output feature vector, is a predicted matching score based on a single ELM network model, is a normalized fusion weight of each spectral band, which is defined as follows:

[0057]

[0058] k(·) is the spectral weight generating function, which can be implemented by the absolute density of the k-neighborhood of the sample, high density indicates that the spectrum is representative and should be given a larger weight in multispectral classification, and low density indicates that the spectrum data is far from the core area and should be given a smaller weight in multispectral classification. The fusion matching score is generated according to the contribution weight of each spectral band, and the spectrum with a high contribution weight is enhanced, otherwise it is suppressed.

[0059] The multispectral fusion prediction loss is defined as the mean square error (MSE) between the multispectral fusion output and the target output:

[0060]

[0061] After replacing, the multispectral fusion prediction loss can be matrixed as:

[0062]

[0063] wherein is a diagonal matrix that allocates the contribution weight of each spectral band. Minimizing this loss will make the fusion output consistent with the expected output as much as possible, while effectively alleviating the influence of spectral pollution on the model and integrating the complementary features of multispectral palmprints.

[0064] The single-spectrum prediction loss is defined as the weighted mean square error (WMSE) between the output of a single-spectrum extreme learning machine model and the target output:

[0065]

[0066] Similarly, the single-spectrum-based loss is matrixed as:

[0067]

[0068] Spectra with low contribution weights are more likely to be disturbed by noise, and their prediction loss contribution is small. This single-spectrum prediction loss optimization can enable each ELM model to learn high-discriminative palmprint features from noise-free training samples.

[0069] The network structure loss is the sum of the matrix norms of all β (s) , which represents the complexity of the fusion ELM framework. Minimizing this loss can make the model have good generalization ability, and is defined as follows:

[0070]

[0071] ​Finally, the embodiment provides a joint loss that considers the complementarity, uniqueness and generalization ability of the model, where λ1 and λ2 are balance parameters, and are defined as follows:

[0072]

[0073] It can be seen that L joint is a convex quadratic function of β (s) , and the solution of the optimization problem is as follows:

[0074]

[0075] From the above formula, it can be found that the value of β (i) is closely related to other network parameters β (s) ,s≠i. The balance parameter λ1 is used to control the degree of dependence between β (i) . For example, when λ1 = 0, β (i) is determined by a single spectral band, and the extracted features are only for that spectral band. Therefore, selecting a suitable λ1 can ensure the complementarity and uniqueness of different spectral information, and consider all cases

[0076]

[0077] The optimal solution β (i) ,i = 1, 2, …, S of the above model can be represented by the following formula:

[0078]

[0079] Where:

[0080]

[0081] Therefore, the output weight of the fusion network can be defined as:

[0082]

[0083] Where I N and I L are unit matrices with dimensions N and L, respectively.

[0084] The learning process of the fusion extreme learning machine model is summarized as follows:

[0085] Step 1: Randomly set Where l = 1, 2, …, L;

[0086] Step 2: Obtain the hidden layer output matrix H (s) according to formula (5);

[0087] Step 3: Calculate the diagonal matrix of the spectral band allocation contribution weight according to formula (11)

[0088] Step 4: Calculate the output weight of the fusion network according to formula (21)

[0089] Finally, the unknown multispectral sample is given The prediction result can be calculated by the following formula:

[0090]

[0091] Taking the CASIA multispectral palmprint dataset as an example, the effectiveness of the embodiment is verified, and the single-spectrum prediction loss identification, the multispectral fusion prediction loss based on the network structure loss optimization, the network structure loss and the single-spectrum prediction loss optimization, and the joint loss optimization method proposed in the embodiment are considered respectively. The accuracy, precision, sensitivity, and F1 index are used as evaluation (the larger the value, the better), and the results of the three methods are compared, as shown in Table 1. It can be found that the embodiment is better than the comparison method in the four indicators.

[0092] Table 1 Comparison results of different optimization strategies

[0093]

[0094] Embodiment two

[0095] As Figure 4 shown, the multispectral palmprint recognition method based on the fusion extreme learning machine provided by the embodiment comprises:

[0096] Step 100: Obtain a multispectral palmprint image to be identified.

[0097] Step 200: Input the multispectral palmprint image to be identified into a multispectral palmprint recognition network model to obtain palmprint identity information.

[0098] The multispectral palmprint recognition network model is obtained by training the fusion extreme learning machine based on the joint loss function and multiple training data; the training data comprises a multispectral palmprint image as a sample and a one-hot identity label vector corresponding to a palmprint image of each spectral band in the multispectral palmprint image as a sample; the fusion extreme learning machine comprises multiple extreme learning machine network models and an identity recognition module connected to the output end of each extreme learning machine network model; different extreme learning machine network models are used to input palmprint images of different spectral bands; and the joint loss function is a loss function composed of a network structure loss function, a multispectral fusion prediction loss function, and a single-spectrum prediction loss function.

[0099] In the embodiment, the extreme learning machine network model is configured to obtain a hidden layer output feature vector according to the input corresponding spectral band palmprint image; and the identity recognition module is configured to generate a palmprint identity information vector according to the hidden layer output feature vector of each extreme learning machine network model.

[0100] Further, the identity recognition module is configured to generate an identity matching score weight according to the hidden layer output feature vector of each extreme learning machine network model, perform a spectral information weighted fusion strategy on the identity matching scores predicted by different extreme learning machine network models, and generate a palmprint identity information vector.

[0101] In the embodiment, the multi-spectral fusion prediction loss function is a function of calculating the mean square error between a multi-spectral fusion output and a target output; the multi-spectral fusion output is a palmprint identity information vector output by the identity recognition module; and the target output is a corresponding one-hot identity label vector.

[0102] In the embodiment, the single-spectral prediction loss function is a function of calculating a weighted mean square error between a single-spectral extreme learning machine network model output and a target output. The extreme learning machine network model output is a palmprint identity information vector output by the extreme learning machine network model.

[0103] In the embodiment, the network structure loss function is a function of calculating the sum of matrix norms of all target network parameters; and the target network parameter is a weight vector connecting a hidden node to an output node in the extreme learning machine network model.

[0104] Embodiment Three

[0105] In order to perform the method corresponding to the above-mentioned embodiment two to achieve the corresponding functions and technical effects, the following provides a multi-spectral palmprint recognition system based on a fusion extreme learning machine.

[0106] The multi-spectral palmprint recognition system based on the fusion extreme learning machine provided in the embodiment comprises:

[0107] A to-be-identified multi-spectral palmprint image acquisition module is configured to acquire a to-be-identified multi-spectral palmprint image.

[0108] A palmprint identity information recognition module is configured to input the to-be-identified multi-spectral palmprint image into a multi-spectral palmprint recognition network model to obtain palmprint identity information.

[0109] The multispectral palmprint identification network model is obtained by training a fusion extreme learning machine through a joint loss function and multiple training data; the training data include multispectral palmprint images as samples, and one-hot identity label vectors corresponding to the palmprint images of each spectral band in the multispectral palmprint images as samples; the fusion extreme learning machine includes multiple extreme learning machine network models and an identity recognition module connected to the output end of each extreme learning machine network model; different extreme learning machine network models are used to input palmprint images of different spectral bands; and the joint loss function is a loss function composed of a network structure loss function, a multispectral fusion prediction loss function, and a single-spectrum prediction loss function.

[0110] Embodiment four

[0111] The electronic device includes a memory and a processor, the memory is used to store a computer program, and the processor runs the computer program to enable the electronic device to execute the multispectral palmprint identification method based on the fusion extreme learning machine according to the embodiment one.

[0112] Optionally, the electronic device can be a server.

[0113] In addition, the embodiment of the present application also provides a computer readable storage medium, which stores a computer program, and the computer program is executed by a processor to realize the multispectral palmprint identification method based on the fusion extreme learning machine according to the embodiment one.

[0114] In the specification, each embodiment is described in a progressive manner, and each embodiment focuses on the difference from other embodiments, and the same or similar parts between each embodiment can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the related parts can be referred to the method part.

[0115] The principles and implementation manners of the present application are described by using specific examples in the present application, and the above embodiment description is only used to help understand the method of the present application and its core idea; meanwhile, for those skilled in the art, according to the idea of the present application, the specific implementation manner and application range will be changed. In conclusion, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A multispectral palmprint recognition method based on fusion extreme learning machine, characterized in that, The method comprises the following steps: obtaining a multispectral palmprint image to be identified; inputting the multispectral palmprint image to be identified into a multispectral palmprint identification network model to obtain palmprint identity information; the multispectral palmprint identification network model is obtained by training a fusion extreme learning machine through a joint loss function and a plurality of training data; the training data comprises a multispectral palmprint image as a sample and a one-hot identity label vector corresponding to a palmprint image of each spectral band in the multispectral palmprint image as a sample; the fusion extreme learning machine comprises a plurality of extreme learning machine network models and an identity recognition module connected to the output end of each extreme learning machine network model; different extreme learning machine network models are used to input palmprint images of different spectral bands; the joint loss function is a loss function composed of a network structure loss function, a multispectral fusion prediction loss function and a single spectral prediction loss function; the network structure loss function is a function of calculating the sum of matrix norms of all target network parameters; the target network parameters are weight vectors connecting hidden nodes to output nodes in the extreme learning machine network model; the multispectral fusion prediction loss function is a function of calculating the mean square error between multispectral fusion output and target output; the multispectral fusion output is a palmprint identity information vector output by the identity recognition module; the target output is a corresponding one-hot identity label vector; the single spectral prediction loss function is a function of calculating the weighted mean square error between the output of a single spectral extreme learning machine network model and the target output; the extreme learning machine network model output is a palmprint identity information vector output by the extreme learning machine network model, and the target output is a corresponding one-hot identity label vector.

2. The multispectral palmprint recognition method based on fusion extreme learning machine according to claim 1, characterized in that, The extreme learning machine network model is used to obtain a hidden layer output feature vector according to the input corresponding spectral band palmprint image; the identity recognition module is used to generate a palmprint identity information vector according to the hidden layer output feature vector of each extreme learning machine network model.

3. The multispectral palmprint recognition method based on fusion extreme learning machine according to claim 1 or 2, characterized in that, The identity recognition module is used to generate an identity matching score weight according to the hidden layer output feature vector of each extreme learning machine network model, perform a spectral information weighted fusion strategy on the identity matching scores predicted by different extreme learning machine network models, and generate a palmprint identity information vector.

4. A multispectral palmprint identification system based on fusion extreme learning machine, characterized in that, The method comprises the following steps: a multispectral palmprint image to be identified acquisition module is used to obtain a multispectral palmprint image to be identified; a palmprint identity information identification module is used to input the multispectral palmprint image to be identified into a multispectral palmprint identification network model to obtain palmprint identity information; The multispectral palmprint identification network model is obtained by training a fusion extreme learning machine through a joint loss function and multiple training data; the training data includes multispectral palmprint images as samples, and one-hot identity label vectors corresponding to palmprint images of each spectral band in the multispectral palmprint images as samples; the fusion extreme learning machine includes multiple extreme learning machine network models and an identity recognition module connected to the output end of each extreme learning machine network model; different extreme learning machine network models are used to input palmprint images of different spectral bands; the joint loss function is a loss function composed of a network structure loss function, a multispectral fusion prediction loss function, and a single-spectrum prediction loss function; The network structure loss function is a function of calculating the sum of matrix norms of all target network parameters; the target network parameters are weight vectors connected from hidden nodes to output nodes in the extreme learning machine network model; The multispectral fusion prediction loss function is a function of calculating the mean square error between multispectral fusion output and target output; the multispectral fusion output is a palmprint identity information vector output by the identity recognition module; the target output is a corresponding one-hot identity label vector; The single-spectrum prediction loss function is a function of calculating the weighted mean square error between the output of a single-spectrum extreme learning machine network model and the target output; the extreme learning machine network model output is a palmprint identity information vector output by the extreme learning machine network model, and the target output is a corresponding one-hot identity label vector.

5. An electronic device, comprising: An electronic device includes a memory for storing a computer program and a processor for running the computer program to enable the electronic device to perform a multispectral palmprint identification method based on a fusion extreme learning machine according to any one of claims 1 to 3.

Images