A dual-modal recognition method based on canonical correlation analysis of finger vein and finger knuckle print

Abstract

A kind of finger vein and finger knuckle print dual-mode identification method based on canonical correlation analysis, comprising: Daubechies wavelet pre-processing is carried out to the finger vein image and finger knuckle print image for training, to obtain the low-frequency image of finger vein image and finger knuckle print image respectively, K-L transform is used to extract features from two groups of low-frequency images respectively, then correlation criterion function between two groups of feature matrices is established, two sets of canonical projection vectors are obtained according to criterion function, finally the canonical correlation characteristics are fused by given feature fusion strategy and applied in classification identification.The beneficial effects of the present application are: the present application obtains the feature information of two modes by dimension reduction, reduces the recognition calculation amount, improves the recognition accuracy, introduces more class information into the criterion function more fully, not only reaches the purpose of information fusion, but also effectively eliminates the redundant information between features, and makes the classification performance greatly improved.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of biometric recognition, and particularly relates to a finger vein and finger knuckle print dual-mode recognition method based on canonical correlation analysis. BACKGROUND

[0002] The finger vein and the finger knuckle print are both biometric information located on the surface of the finger, and contain rich texture information. Because of their compactness and easy-to-acquire characteristics, they are often used as research objects of finger multi-modal fusion technology.

[0003] The finger vein belongs to subcutaneous tissue, and has higher anti-spoofing performance compared to other biometric features. However, the single finger vein mode has the problem that the vein features of some people are not clear enough, which is particularly evident in female users, thereby reducing the accuracy of finger vein recognition. In addition, the clarity of the finger vein image is also affected by factors such as environmental temperature. The finger knuckle print, as a kind of wrinkle texture, is basically stable after a person reaches adulthood, and there are obvious differences between different individuals. However, compared to other biometric features, the finger knuckle print contains less feature information. Therefore, the respective defects of the single modal cause the reduction of the recognition accuracy and reliability.

[0004] The influence of traditional simple serial and parallel feature fusion on recognition accuracy is also particularly evident. Each modal is affected by different types and degrees of noise, resulting in that the fused features cannot accurately represent the original feature information. In addition, with the increase of the feature dimension of the single modal, the feature dimension of the fused features increases exponentially, causing the curse of dimensionality, increasing the computational complexity, and affecting the recognition accuracy. Therefore, the quality of feature fusion directly affects the final recognition effect. SUMMARY

[0005] The present application solves the problem of high feature dimension, a large amount of redundant information and low computational efficiency after traditional simple fusion in the prior art, and proposes a feature fusion method for finger veins and finger knuckle prints based on improved traditional canonical correlation analysis.

[0006] In order to achieve the above purpose, the present application provides a feature fusion method for finger veins and finger knuckle prints based on canonical correlation analysis. The method comprises the following steps:

[0007] S1, performing Daubechies wavelet preprocessing on the extracted ROI images of the finger veins and the finger knuckle prints, wherein the sequence N=2, and low-frequency images and high-frequency images in horizontal, vertical and diagonal directions are obtained through wavelet decomposition. The present application only retains the low-frequency images of the images, respectively as and for input images for subsequent feature extraction, wherein, is the low-frequency image of the finger vein image after db2 wavelet decomposition, is the low frequency image after db2 wavelet decomposition of the knuckle line image.

[0008] S2, Eigenface is used to extract the feature information of the finger vein dataset and the knuckle line dataset respectively. The specific steps include:

[0009] S21, the low frequency image after db2 wavelet decomposition and are input as the training set, wherein, the pixel size of m x ×n x , the pixel size of m y ×n y .

[0010] S22, connect the columns of each and to form column vectors of dimension respectively. Get I X ={X1, X2, X3, …, X i} as the formed finger vein vector set, wherein X i is the finger vein vector of the i-th image, I Y ={Y1, Y2, Y3, …, Y i} is the formed knuckle line vector set, wherein Y i is the knuckle line vector of the i-th image.

[0011] S23, calculate the average finger vein image and the knuckle line image The average image calculation formula is as follows:

[0012]

[0013] S24, calculate the difference between each image and the average image and and obtain the difference vector set of the finger vein and the knuckle line respectively as The difference calculation formula is as follows:

[0014]

[0015] S25, use K-L transformation to extract features of the finger vein difference vector set A x and the knuckle line difference vector set A Y , and the specific steps include the following:

[0016] S251, construct vector set A X and A YThe transpose matrix of the covariance matrix of the i x i dimensional C AX and the i x i dimensional C AY The specific matrix calculation formula is:

[0017]

[0018] S252, respectively, calculate and obtain the matrix C AX and C AY The first M eigenvalues and eigenvectors γ x , v x and γ y , v y , where M < i, and the eigenvector of the corresponding covariance matrix is μ x and μ y , wherein the eigenvector calculation formula is:

[0019]

[0020] S253, the characteristic information vector set of the finger vein and the knuckle ridge is obtained by using the above calculation:

[0021]

[0022] Where ω MX and ω MY are the characteristic information vector sets of the finger vein and the knuckle ridge, and the dimensions are both M x i, where i represents the number of pictures contained in the finger vein data set and the knuckle ridge data set, and M represents the dimension of the two features.

[0023] S3, the extracted finger vein feature and knuckle ridge feature are improved discriminant canonical correlation analysis, the mapping matrix α and β are solved by feature decomposition method, and the new features u i = α T ω MX and v i = β T ω MY represent the input features, where α T is the transpose matrix of the mapping matrix α, β T is the transpose matrix of the mapping matrix β, and the specific steps include:

[0024] S31, standardize the two information features ω MX and ω MY , eliminate the adverse effects of possible numerical or dimensional non-uniformity of the two groups of features, and obtain the standard matrix X and Y with mean value 0 and variance 1. The standardization formula is:

[0025]

[0026] where μ X = E(x) and μ Y = E(y) are the mean vectors of the training samples ω MX and ω MY , respectively.

[0027] σ xj and σ yj are the standard deviations of the jth feature component of the finger vein and the finger-ridge pattern samples, respectively.

[0028] S32, extract the canonical correlation features between X and Y, denoted as and represent the feature components after the dth canonical correlation analysis, and thus the two sets of feature vectors after transformation are:

[0029]

[0030]

[0031] The specific steps for calculating the canonical projection vectors and the canonical correlation features are as follows:

[0032] S321, construct the criterion function of the improved discriminant canonical correlation analysis as follows:

[0033]

[0034] where S w represents the within-class correlation matrix, and S b represents the between-class correlation matrix. The within-class correlation matrix S w can be expressed as:

[0035]

[0036]

[0037] where c is the number of finger vein and finger-ridge categories,

[0038]

[0039] I n = {1, …, 1} (13)

[0040] A = diag(I n1×n1 , …, I nc×nc ) (14)

[0041] Therefore, the criterion function is transformed into the following function:

[0042]

[0043] S322、Consider J d The extreme value of (a, b) is only related to the direction of a and b, and is not related to their size. Add the constraint condition:

[0044] a T XAX T a = b T YAY T b = 1 (16)

[0045] S323, define the Lagrange function:

[0046]

[0047] S324, on the basis of the Lagrange function L, respectively, the derivative of a and b, and organize to get:

[0048] XAY T b = vXAX T a (18)

[0049] YAX T a = vYAY T b (19)

[0050] S325, joint solution S324 derived from the two equations, the final maximum value of the problem is transformed into solving the following two generalized eigen equation problem:

[0051] XAY T YAY T-1 YAX T a = v 2 XAX T a (20)

[0052] YAX T XAX T-1 XAY T b = v 2 YAY T b (21)

[0053] S326, find the maximum of the equation in S325 d first generalized eigenvalue v1≥v1≥…≥v d , the corresponding eigenvector is the discriminant type of the typical correlation projection vector set.

[0054] S4, the following fusion strategy is used to obtain the fusion representation of the typical correlation features of the finger vein and the knuckle pattern:

[0055]

[0056] Wherein, Z 1i and Z 2i Both represent the characteristics of the i-th finger vein sample and the finger joint vein sample after feature fusion.

[0057] S5, the original image L1 of the finger vein and the original image P1 of the finger joint vein of any individual are preprocessed by the method of S1 to obtain low-frequency images L2 and P2, and the feature information L3 of the finger vein and the finger joint vein is extracted by using the feature vector obtained in S262, L3 = μ x L2, P3 = μ y P2, the mapping matrix alpha and beta obtained by S3 are used to obtain the typical correlation feature vector x * =(alpha1, alpha1, …, alpha d ) T L3, y * =(beta1, beta1, …, beta d ) T P3, and the corresponding fusion features Z1 * and Z2 * are obtained by using the fusion strategy of S41.

[0058] S6, the Euclidean distance of Z1 * to Z 1i , the Euclidean distance of Z2 * to Z 2i is calculated, if the Euclidean distance d of the to-be-identified fusion feature to all samples is greater than the set threshold, the matching fails, otherwise the matching succeeds.

[0059]

[0060] Compared with the prior art, the beneficial effects of the present application are as follows:

[0061] The finger vein and finger joint vein dual-mode identification method based on canonical correlation analysis provided by the present application can effectively fuse the two features of finger vein and finger joint vein, and the method of the present application can analyze the internal relationship between the two features, seek the correlation between them, obtain new features with maximum correlation while retaining most of the original feature information, and reduce the redundant information between the original features.

[0062] Compared with the traditional canonical correlation analysis which does not consider the category information between data, the application introduces the correlation between the same samples and the correlation between different categories into the criterion function more fully on the basis of discriminant canonical correlation analysis, so that the classification performance is greatly improved. Compared with the traditional finger vein and knuckle print feature extraction method, the application obtains the low frequency image of the finger vein and knuckle print image by multi-Besse wavelet, and extracts the feature of the denoised image by K-L transformation, so that the calculation amount is greatly reduced and the calculation efficiency is improved. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 The implementation flowchart of the embodiment of the application is shown in the figure.

[0064] Figure 2 The low frequency image of the finger vein image denoised by db2 wavelet in the example of the application is shown in the figure.

[0065] Figure 3 The low frequency image of the knuckle print image denoised by db2 wavelet in the example of the application is shown in the figure. DETAILED DESCRIPTION

[0066] The embodiment of the application will be described below with reference to the accompanying drawings.

[0067] The data set used in the embodiment is the zjutFVR finger vein data set and the zjutKPR knuckle print data set collected by the applicant. The finger vein and the knuckle print are both 396 categories, and each single mode has a total of 2376 pictures. The resolution of the finger vein picture is 980*400, and the resolution of the knuckle print picture is 220*110.

[0068] The flowchart of the finger vein and knuckle print dual-mode recognition method based on canonical correlation analysis proposed in the embodiment is shown in the figure. Figure 1 The method includes the following steps:

[0069] S1, the extracted ROI image of the finger vein and the knuckle print is preprocessed by Daubechies wavelet, wherein the sequence N=2, the low frequency image and the high frequency image in the horizontal, vertical and diagonal directions are obtained by wavelet decomposition, and the application only retains the low frequency image of the image respectively as and The input image for subsequent feature extraction, wherein, is the low frequency image of the finger vein image after db2 wavelet decomposition, is the low frequency image of the knuckle print image after db2 wavelet decomposition.

[0070] S2, the characteristic information of the finger vein data set and the knuckle print data set is extracted by using the idea of Eigenface. The specific steps include:

[0071] S22, low-frequency image after db2 wavelet decomposition and as the training set input, wherein, the pixel size of each image is 202x491, the pixel size of each image is 56x111.

[0072] S23, connecting the columns of each and to form a column vector of dimension, respectively, to obtain the set of finger vein vectors I X ={X1, X2, X3, …, X i}, where X i is the finger vein vector of the i-th image, and I Y ={Y1, Y2, Y3, …, Y i} is the set of knuckle line vectors formed, where Y i is the knuckle line vector of the i-th image.

[0073] S24, calculating the average finger vein image and the knuckle line image

[0074] S25, calculating the difference between each image and the average image and and obtaining the respective difference vector sets of the finger veins and knuckle lines as

[0075] S26, using K-L transformation to extract features from the finger vein difference vector set A x and the knuckle line difference vector set A Y , the specific steps including the following:

[0076] S261, constructing the transpose matrix of the covariance matrix of the vector sets A X and A Y to obtain the i x i-dimensional C AX and the i x i-dimensional C AY , i = 396.

[0077] S262, calculating and obtaining the first M eigenvalues and the corresponding eigenvectors γ AX , v AY and γ x , v x and γ y , v y of the matrices C x and C y , respectively, where M < i, and the eigenvectors of the corresponding covariance matrix are μ x and μ y .

[0078] S263. Using the characteristic finger veins and characteristic finger joint patterns obtained above, label the original finger vein training set and finger joint pattern training set to obtain ω. MX and ω MY Both features have dimensions M×i, where i represents the number of images contained in the finger vein dataset and the knuckle print dataset, and M represents the dimensions of the two features.

[0079] S3. Improved canonical correlation analysis is performed on the extracted finger vein and knuckle features. The mapping matrices α and β are solved using feature decomposition, and the new feature u is used. i =α T ω MX and v i =β T ω MY Let α represent the input features. T Let β be the transpose of the mapping matrix α. T To obtain the transpose of the mapping matrix β, the specific steps include:

[0080] S31, combine the two information features ω MX and ω MY Standardization is performed to eliminate the adverse effects of potential numerical or dimensional imbalances between the two sets of features, resulting in standard matrices X and Y with a mean of 0 and a variance of 1.

[0081] S32. Extract the canonical correlation features between X and Y, denoted as... and Let X represent the eigencomponents after the d-th pair of classical correlation analysis. Thus, we can obtain two sets of eigenvectors after transformation, X. * and Y * .

[0082] The specific steps for calculating the typical projection vectors and typical related features are as follows:

[0083] S321. The improved criterion function for discriminant canonical correlation analysis is constructed as follows:

[0084]

[0085] S322. Determine the constraints:

[0086] α T XAX T α=β T YAY T β=1 (16)

[0087] S323. Using the Lagrange function, the problem of finding the maximum value is transformed into solving the following two generalized eigenvalue equations:

[0088] XAY T YAY T-1 YAX T α = v 2 XAX T α (20)

[0089] YAX T XAX T-1 XAY T β=v 2 YAY T β (21)

[0090] S324. Find the first d largest generalized eigenvalues ​​of the equation: v1≥v1≥…≥v d The corresponding feature vector This is the desired discriminative canonical correlation projection vector set.

[0091] S4. The following fusion strategy is used to obtain a fusion representation of typical related features of digital veins and knuckles:

[0092]

[0093] Among them, Z 1i and Z 2i Both represent the features after fusing the features of the i-th finger vein sample and the finger joint print sample.

[0094] S5. Preprocess the original image L1 of the finger vein and the original image P1 of the finger joint print of any individual using the method in S1 to obtain low-frequency images L2 and P2. Extract the feature information L3 = μ of the finger vein and finger joint print using the feature vector obtained in S262. x L2, P3 = μ y P2, using the mapping matrices α and β obtained from S3, yields the canonical correlation feature vector x. * =(α1,α1,……,α) d ) T L3, y * =(β1,β1,……,β) d ) T P3, using the S41 fusion strategy, yields the corresponding fusion feature Z1. * and Z2 * .

[0095] S6, Calculate Z1 * To Z 1i Euclidean distance, Z2 * To Z 2i If the Euclidean distance d between the fused feature to be identified and all samples is greater than the set threshold, the matching fails; otherwise, the matching succeeds.

[0096] The application obtains feature information of two modes by dimension reduction, reduces recognition calculation amount, improves recognition precision, introduces more class information into criterion function more fully, achieves information fusion, effectively eliminates redundant information between features, and greatly improves classification performance.

[0097] The content described in the embodiments of the present specification is only a list of implementation forms of the inventive concept, and the protection scope of the present application should not be regarded as being limited to the specific forms described in the embodiments, and the protection scope of the present application also extends to equivalent technologies that can be thought of by those skilled in the art according to the inventive concept.

Claims

1. A dual-modal recognition method for finger veins and knuckle prints based on canonical correlation analysis, characterized in that, Includes the following steps: S1. Preprocess the finger vein images and knuckle print images used for training to obtain low-frequency finger vein images and knuckle print images. S2. Feature extraction based on the eigenface method is performed on the preprocessed low-frequency images of finger veins and knuckle patterns. The specific process of feature extraction is as follows: S21, Low-frequency image after db2 wavelet decomposition and As input to the training set, where The pixel size is , The pixel size is ; S22, each and The columns are connected to form = , = A column vector of dimension; to obtain I X = { X 1 ，X 2 ，X 3 ，……，X i } represents the formed set of finger vein vectors, where X i For the first i The finger vein vector of the image, I Y ={ Y 1 ， Y 2 ，Y 3 ，……，Y i } represents the resulting set of knuckleprint vectors, where Y i For the first i The knuckle vector of the image; S23. Calculate the average digital vein image. and knuckle print images The formula for calculating the average image is as follows: ， (1) S24. Calculate the difference between each image and the average image. and And obtain the difference vector sets of finger veins and finger joint patterns respectively. A X ={ , , ..., }, A Y ={ , , ..., The formula for calculating the difference is as follows: ， (2) S25. Using KL transform on the finger vein difference vector set A x and the set of differences in knuckle prints A Y Feature extraction involves the following steps: S251, Constructing a vector set A X and A Y The transpose of the covariance matrix is ​​obtained i×i Vie and i×i Vie The specific matrix calculation formula is as follows: ， (3) S252. Calculate and obtain the matrices respectively. and The former M eigenvalues ​​and eigenvectors γ x 、v x and γ y 、v y ,in M< i The eigenvectors of its corresponding covariance matrix are μ x and μ y The formula for calculating the eigenvector is: ， (4) S253. Use the characteristic finger veins and characteristic finger joint patterns obtained from the above calculations to label the original finger vein training set and finger joint pattern training set: ， (5) in, and This refers to the desired feature information vector set of finger veins and knuckle prints, with each vector having a dimension of 1. M×i ,in i This indicates the number of images contained in the finger vein dataset and the knuckle print dataset. M Dimensions representing two features; S3. Establish the criterion function for canonical correlation analysis using the finger vein feature matrix and the finger joint pattern feature matrix as input matrices, and solve for the projection space and the discriminant canonical correlation projection vector set; S4. Two fusion strategies are used to obtain the fused feature vector set; the following fusion strategy is used to obtain the fused representation of typical related features of finger veins and knuckle patterns: ， (22) S5. Based on the obtained projection space, perform fusion recognition on the finger vein image and the knuckle print image to be identified; combine the original finger vein image of any individual. L 1 Original image of knuckle print P 1 Low-frequency images are obtained by preprocessing using method S1. L 2 and P 2 The feature vectors obtained from S252 are used to extract the feature information of finger veins and knuckle patterns. , The mapping matrix obtained using S3 α and β Derive the canonical correlation feature vector , The corresponding fusion features are obtained by using the S4 fusion strategy. Z 1 and Z 2 ; calculate Z 1 arrive Z 1i European distance, Z 2 arrive Z 2i The Euclidean distance, if the Euclidean distance of the fused feature to be identified is equal to the Euclidean distance of all samples. d If all values ​​are greater than the set threshold, the match fails; otherwise, the match succeeds.

2. The method for dual-modal recognition of finger veins and knuckle prints based on canonical correlation analysis according to claim 1, characterized in that, The specific process of preprocessing the finger vein image and knuckle print image in step S1 is as follows: The extracted ROI images of finger veins and knuckle prints were preprocessed using Daubechies wavelet preprocessing, where the sequence... N= 2. Wavelet decomposition yields low-frequency images and high-frequency images in the horizontal, vertical, and diagonal directions. Only the low-frequency images are retained. and The input image used for subsequent feature extraction, where, This is the low-frequency image of the finger vein image after db2 wavelet decomposition. This is the low-frequency image of the knuckle print image after db2 wavelet decomposition.

3. The method for dual-modal recognition of finger veins and knuckle prints based on canonical correlation analysis according to claim 2, characterized in that, The specific process of solving for the projection space and the discriminative canonical correlation projection vector set is as follows: S31, combine the two information features and Standardization is performed to eliminate the adverse effects of potential numerical or dimensional imbalances between the two sets of features, resulting in a standard matrix with a mean of 0 and a variance of 1. X and Y The standardization formula is: ， (6) in, and Training samples and The mean vector; , , and These are the first samples corresponding to the finger veins and finger joint prints. j The standard deviation of each characteristic component; S32, Extraction X and Y The typical correlation characteristics between them are denoted as and Indicates the first d After classical correlation analysis, the eigencomponents are transformed into two sets of eigenvectors: (7) (8) The specific steps for calculating the typical projection vectors and typical related features are as follows: S321. The improved criterion function for discriminant canonical correlation analysis is constructed as follows: (9) in, , S w Represents the intraclass correlation matrix. S b Represents the inter-class correlation matrix; and the intra-class correlation matrix S w It can be represented as: (10) (11) in, c For the number of categories of finger veins and finger joint prints, ，(12) ，(13) (14) Therefore, the criterion function is transformed into the following function: (15) S322, considering The extreme values ​​are only related to α and β The direction of the numbers is related to their orientation, but not their magnitude. Add the following constraint: (16) S323. Define the Lagrange function: (17) S324, In the Lagrange function L Based on the above, respectively α and β Taking the derivative and simplifying, we get: (18) (19) Solving S325 and S324 together yields two equations, ultimately transforming the maximum value problem into solving the following two generalized eigenvalue equations: (20) (21) S326. Find the largest first equation in S325. d generalized eigenvalues The corresponding feature vector This is the desired discriminative canonical correlation projection vector set.

Images