A face recognition method based on multi-layer cascaded boundary median discriminant projection

Abstract

The application discloses a face recognition method based on multi-layer cascading boundary median discriminant projection, blocks and removes the mean value of a face image, calculates an intra-class dispersion matrix and an inter-class dispersion matrix according to the distance between a to-be-recognized sample and each median sample; calculates the convolution kernel of a first layer convolution layer and the input image of a second layer, carries out second convolution to obtain an output feature map; binarizes the output feature map, blocks the feature map, calculates a corresponding histogram and splices the histogram into a complete vector. The feature output vector is input into a classifier to be trained and output a classification result. The application reflects the centralized trend of sample distribution by using the median of the sample, and has strong discriminant ability; the sample is trained in a multi-layer cascading mode, the intra-class dispersion matrix is calculated, the role of the sample farthest from the median is emphasized, and other samples of the same class are considered, and the discriminant ability is strong; only twice training is needed, and the calculation speed is fast; when the inter-class dispersion matrix is calculated, the distance between different-class samples after projection is maximized, and the classification precision is high.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of image recognition, and relates to a face recognition method based on multi-layer cascading boundary median discriminant projection. BACKGROUND

[0002] Face recognition is an important identity authentication method, and has a broad application prospect in police archives management system, bank security verification system and man-machine interaction field. Feature extraction is an important step of face recognition, and through dimension reduction, noise in the face image can be eliminated, and the essential features of data can be mined. The extracted features can better complete the classification task.

[0003] In recent years, image feature extraction methods based on lightweight convolutional neural networks have been widely used in face recognition. The well-known image feature extraction methods based on lightweight convolutional neural networks include: (1) Multi-level cascaded PCA network (PCANet), recorded in "PCANet: A Simple Deep Learning Baseline for Image Classification?" published in IEEE transactions on image processing, vol. 24, no. 12, pp. 5017-5023, 2015. This method uses unsupervised dimension reduction method PCA to train convolution kernel, gets output feature map after two convolutions, binarizes the output feature map, blocks the feature map, calculates the corresponding histogram and splices it into a complete vector, inputs these feature output vectors into the classifier for training, and outputs the classification results of the test images; (2) Multi-level cascaded LDA network (LDANet), recorded in "PCANet: A Simple Deep Learning Baseline for Image Classification?" published in IEEE transactions on image processing, vol. 24, no. 12, pp. 5017-5023, 2015. This method uses supervised dimension reduction method LDA to train convolution kernel, gets output feature map after two convolutions, binarizes the output feature map, blocks the feature map, calculates the corresponding histogram and splices it into a complete vector, inputs these feature output vectors into the classifier for training, and outputs the classification results of the test images; (3) Multi-level cascaded LPP network (LPPNet), recorded in "Image Feature Extraction and Recognition Method Based on LPPNet" by Xiao Meihong in 2018 in Henan University Master's Thesis. This method uses supervised dimension reduction method LPP to train convolution kernel, gets output feature map after two convolutions, binarizes the output feature map, blocks the feature map, calculates the corresponding histogram and splices it into a complete vector, inputs these feature output vectors into the classifier for training, and outputs the classification results of the test images.The three methods have similar basic ideas, all of which use traditional feature extraction methods as the training method of the convolution kernel, but each has defects: the PCANet uses an unsupervised learning method PCA to reconstruct the convolution kernel, without fully utilizing the class information of the sample; the LDANet uses a supervised learning method LDA to reconstruct the convolution kernel, which utilizes the class information of the sample, but ignores the spatial structure between samples, and due to the limited separation ability of LDA for positive and negative samples, the discriminant ability of LDANet is limited compared with the PCANet model; the LPPNet adopts an unsupervised dimension reduction method LPP to reconstruct the convolution kernel, which fully considers the class information and local spatial structure between samples in the process of reconstructing the convolution kernel, but ignores the global spatial structure. SUMMARY

[0004] To solve the above problems, the present application provides a face recognition method based on multi-layer cascading boundary median discriminant projection, which performs blocking and mean removal processing on the to-be-recognized sample, and constructs an intra-class dispersion matrix and an inter-class dispersion matrix according to the distance information of the to-be-recognized sample and each class of median sample, redefines the distance of the sample boundary to obtain low-dimensional data features with better performance, then calculates the convolution kernel of the first layer of convolution layer and the input image of the second layer, performs second layer convolution by imitating the first layer convolution process, and obtains the output feature map. The feature map is blocked, the corresponding histogram is calculated and spliced into a complete vector through binarization of the output feature map. Finally, these feature output vectors are input into the classifier for training, and the classification result of the test image is output.

[0005] To achieve the above purpose, the present application provides the following technical scheme:

[0006] A face recognition method based on multi-layer cascading boundary median discriminant projection, comprising the following steps:

[0007] Step 1: blocking and mean removal processing of face image;

[0008] Step 2: calculating the median sample in each class of sample;

[0009] Step 3: calculating the distance between the farthest same class sample and the nearest different class sample of the sample;

[0010] Step 4: calculating the intra-class dispersion matrix and the inter-class dispersion matrix;

[0011] Step 5: calculating the convolution kernel of the first layer of convolution layer and the input image of the second layer;

[0012] Step 6: second convolution to obtain the output feature map;

[0013] Step 7: binarizing the output feature map

[0014] Step 8: Block the feature map, calculate the corresponding histogram and splice into a complete vector;

[0015] Step 9: Input the feature output vector into the classifier for training, and output the classification result of the test image.

[0016] Further, the step 1 includes the following process:

[0017] Suppose that the size of the C class is m x n N sample images Take the block of the i-th face image as the center point of each pixel and the block size is k1 x k2, and collect all the blocks to obtain Here x i,j is the j-th block of the image I i , and The symbol is the maximum integer not exceeding z; the x i,j is de-meaned to obtain:

[0018]

[0019] Among them is the block after de-meaning, and 1 is a full 1 vector corresponding to the size; combine to form a sample matrix to obtain:

[0020]

[0021] Further, the step 2 includes the following process:

[0022] Let M i be the median vector of the i-th class sample, and the calculation process is as follows:

[0023] Let the i-th class sample be:

[0024]

[0025] Among them represents the a-th sample in the b-th class, represents the c-th block of the a-th sample in the b-th class, and n i is the number of training samples in each class sample; then the j-th row data of M i is The process of calculating is to sort in ascending order.

[0026] Further, when calculating , it is assumed that If n i is odd, If n iis even, then

[0027] Further, the step 3 includes the following process:

[0028] The distance formula of the ith sample to the median sample of the class j is:

[0029]

[0030] Suppose the number of same-class samples farthest from the class median is G1, and the number of different-class samples closest to the class median is G2. According to the distance formula, the set of same-class samples farthest from the class median is calculated, and is defined as The set of different-class samples closest to the class median is calculated, and is defined as

[0031] Further, the step 4 includes the following process:

[0032] The intra-class dispersion matrix and the inter-class dispersion matrix S are calculated by the following formula respectively w and S b :

[0033]

[0034]

[0035] wherein is used to depict the importance of other samples in the same class, t is a thermal kernel coefficient, and λ is a coefficient.

[0036] Further, the step 5 includes the following process:

[0037] The optimization objective of the algorithm is represented as

[0038] maxtr(V T (S b -S w )V)s.t.V T V=I (7)

[0039] Solve the set of eigenvectors: perform eigenvalue decomposition on the matrix (S b -S w ), and solve the eigenvectors v1, v2,..., v r corresponding to the first r largest eigenvalues to form a matrix v 1 ={v1,v2,...,v r};

[0040] These eigenvectors reconstruct the two-dimensional convolution kernel matrix in the first layer, and the formula is

[0041]

[0042] where L i represents the number of convolution kernels of the i-th layer, W i j represents the i-th convolution kernel in the j-th convolution layer, represents that the dimension of the matrix is k1xk2; represents that the vector is converted into a convolution kernel; the image feature convolved by the first layer of convolution kernels is represented as:

[0043]

[0044] where ° represents the convolution operation; the image is subjected to a four- side zero padding operation, and in the second layer of convolution layers, the j-th non-overlapping vectorized patch in the feature map is defined as Normalization is performed on these vectorized patches to obtain where represents the normalized image block patch vector, Y l represents the image feature after convolution by the l-th convolution kernel, and all the feature maps are represented as:

[0045]

[0046] Further, the step 6 includes the following process:

[0047] The second layer of convolution layers is trained in the same way as the first layer; the second layer of convolution kernel vectors is obtained by formula (7), and then it is reconstructed into a convolution kernel matrix This process is represented as

[0048]

[0049] Suppose that the number of convolution kernels of the second layer is L2, and for N original training images, a total of L2 feature maps are obtained after the second convolution.

[0050] Further, the step 7 includes the following process:

[0051] The Heaviside step function is used to binarize these output feature maps, and L2 feature maps are weighted and added to map into one feature map, which can be represented as:

[0052]

[0053] where, represents the Heaviside step function, which converts all values greater than 0 to 1 and all values less than or equal to 0 to 0; represent the weighted sum mapping feature map, each pixel in The value range of

[0054] Further, the step 8 includes the following process:

[0055] For each mapped feature map Divide it into B blocks and calculate the corresponding histogram for each block, and finally all The corresponding histogram vector Spliced into a complete vector; after completing the process, the final original input image I i The corresponding feature is represented as:

[0056]

[0057] Where f i The final feature output vector of the i-th input image, The corresponding histogram vector of the feature map B is the number of blocks.

[0058] Compared with the prior art, the present application has the following advantages and beneficial effects:

[0059] (1) Only two training is needed, without multiple iterations, compared with other multiple iteration training neural network, the calculation speed is faster.

[0060] (2) The multi-level cascade method is used to train the sample, and the discrimination and representation ability of the proposed model is enhanced.

[0061] (3) The median is used to reflect the central tendency of the sample distribution, which enhances the discrimination ability of the proposed model and avoids the problem that different images of the same person have large differences due to changes in light, occlusion and posture.

[0062] (4) When calculating the intra-class dispersion matrix, the role of the sample farthest from the median is emphasized, and other samples of the same class are also considered, which enhances the discrimination ability of the proposed model.

[0063] (5) When calculating the inter-class dispersion matrix, the distance between different class samples is maximized after projection, which improves the classification accuracy of the proposed model. BRIEF DESCRIPTION OF DRAWINGS

[0064] Figure 1 The flow chart of the face recognition method based on multi-level cascade boundary median discriminant projection provided by the present application.

[0065] Figure 2 The picture sample in the AR face library. DETAILED DESCRIPTION

[0066] The technical solutions provided by the present application will be described in detail below in combination with specific embodiments, and it should be understood that the following specific embodiments are only used to illustrate the present application and not to limit the scope of the present application.

[0067] The present application provides a face recognition method based on multi-layer cascading boundary median discriminant projection, and the specific process is as shown in Figure 1 , which comprises the following steps.

[0068] Step 1, block and de-meaning processing of face image

[0069] Suppose that there are N samples of class C with size m x n Take the i-th face image as the center point, take a block with size k1 x k2, and collect all the blocks to obtain Here x i,j is the j-th block of image I i , and Here the symbol is the maximum integer not exceeding z. After de-meaning processing of x i,j , the following can be obtained:

[0070]

[0071] Wherein is the block after de-meaning, and 1 is a full 1 vector with corresponding size. Combine to form a sample matrix, and the following can be obtained:

[0072]

[0073] Step 2, calculate the median sample of each class sample

[0074] Let M i be the median vector of the i-th class sample, and its calculation process is as follows:

[0075] Let the i-th class sample be:

[0076]

[0077] Wherein represents the a-th sample in the b-th class, represents the c-th block of the a-th sample in the b-th class, and n i is the number of training samples in each class sample. The j-th row data of M i is The process of calculating is to sort in ascending order. For convenience of calculation, it is assumed that If n iis odd, then If n i is even, then

[0078] Step 3, calculate the distance between the farthest same-class sample and the nearest different-class sample of the sample

[0079] The distance formula of the ith sample to the median sample of the class j where the sample is located is:

[0080]

[0081] Suppose the number of same-class samples farthest from the class median sample is G1, and the number of different-class samples nearest to the class median sample is G2. According to the distance formula, the set of same-class samples farthest from the class median sample in each class sample is calculated and defined as The set of different-class samples nearest to the class median sample in each class sample is calculated and defined as

[0082] Step 4, calculate the intra-class dispersion matrix and inter-class dispersion matrix

[0083] S w and S b represent the intra-class dispersion matrix and inter-class dispersion matrix respectively. The formulas are respectively:

[0084]

[0085]

[0086] wherein is used to describe the importance of other samples in the same class, t is the heat kernel coefficient, taking the value of 1.326, and λ is the coefficient, taking the value range of (0, 1].

[0087] Step 5, calculate the convolution kernel of the first layer convolution layer and the input image of the second layer

[0088] The optimization objective of the algorithm can be represented as

[0089] maxtr(V T (S b -S w )V)s.t.V T V=I (7)

[0090] The set of eigenvectors is solved by: performing eigenvalue decomposition on the matrix (S b -S w ), and solving the eigenvectors v1, v2,..., v r corresponding to the first r largest eigenvalues to form the matrix v 1 ={v1,v2,...,v r}.

[0091] These feature vectors reconstruct the two-dimensional convolution kernel matrix in the first layer, which is

[0092]

[0093] where L i denotes the number of convolution kernels in the i-th layer, denotes the i-th convolution kernel in the j-th convolution layer, denotes that the dimension of the matrix is k1x k2; denotes that the vector is converted into a convolution kernel. The image features convolved by the first layer of convolution kernels can be represented as:

[0094]

[0095] where ° denotes the convolution operation. In order to keep the dimension of the feature map after convolution the same as that of the image before convolution, a four- corner zero padding operation is performed on the image. In the second convolution layer, the j-th non-overlapping vectorized patch in the feature map is defined as Normalization of these vectorized patches can obtain where denotes the normalized image patch vector. Y l denotes the image features after convolution by the l-th convolution kernel, and all the feature maps can be represented as:

[0096]

[0097] Step 6, the second convolution obtains the output feature map

[0098] For training the second convolution layer, the process is exactly the same as that of the first layer, and the training of the second convolution layer is completed.

[0099] The second layer of convolution kernel vectors can be obtained by formula (7), and then they are reconstructed into the convolution kernel matrix This process can be represented as

[0100]

[0101] Assuming that the number of convolution kernels in the second layer is L2, for N original training images, a total of L2 feature maps can be obtained after the second convolution.

[0102] Step 7, binarize the output feature map

[0103] The Heaviside step function is used to binarize the feature maps after the outputs, and L2 feature maps are weighted and added to map into one feature map, which can be represented as:

[0104]

[0105] wherein, represents the Heaviside step function, which is used to convert all values greater than 0 into 1 and all values less than or equal to 0 into 0. represents the feature map after the weighted and added mapping, and each pixel in The value range of

[0106] Step 8, the feature map is divided into blocks, the corresponding histogram is calculated and spliced into a complete vector

[0107] For each mapped feature map it is divided into B blocks and the corresponding histogram of each block is calculated, and finally all corresponding histogram vectors are spliced into a complete vector. After the process is completed, the final original input image I i corresponding features can be represented as:

[0108]

[0109] wherein, f i represents the final feature output vector of the i-th input image, represents the histogram vector corresponding to the feature map .

[0110] Step 9, then the feature output vectors are input into the classifier for training, and the classification results of the test images are output.

[0111] To verify the effectiveness of the present application, experiments are performed on the commonly used AR face library, which has a total of 3120 pictures, including 120 people, each having 26 pictures (all taken under different light, expressions and occlusion environments), and each picture has a resolution of 50x40, Figure 2 is an example of 26 pictures of a person in the face library.

[0112] In the experiment, the first 2, 4, 6, and 8 images of each person are selected as training samples, and the remaining images are used as to-be-identified samples for testing. The parameter λ is set to 0.5, t is set to 1.326, G1 is set to 1, and G2 is set to 30. The comparison algorithms are boundary discriminant projection (MDP), multi-level cascade PCA network (PCANet), VGG neural network (VGG16), and Mobilenet neural network (Mobilenet v2). Table 1 lists the face recognition rates of the present application and the comparison algorithms:

[0113]

[0114]

[0115] Table 1

[0116] Obviously, compared with other algorithms in the art, the present application can achieve the best effect for different training sample quantities, and effectively improves the recognition accuracy.

[0117] The technical means disclosed in the present application scheme are not limited to the technical means disclosed in the above embodiments, but also include technical solutions composed of any combination of the above technical features. It should be noted that for those skilled in the art, without departing from the principles of the present application, a number of improvements and refinements can be made, and these improvements and refinements are also considered within the protection scope of the present application.

Claims

1. A face recognition method based on multi-layer cascaded boundary median discriminant projection, characterized in that, Includes the following steps: Step 1: Perform block segmentation and mean-reduction processing on the face image; Step 2: Calculate the median sample in each class; Step 3: Calculate the distance between the farthest sample of the same class and the nearest sample of a different class; Step 4: Calculate the intra-class scatter matrix and the inter-class scatter matrix; Step 5: Calculate the convolution kernel of the first convolutional layer and the input image of the second layer; specifically, the following process is included: The optimization objective of the algorithm is expressed as: maxtr(V T (S b -S w )V) s.t.V T V=I (7) Solving for the set of eigenvectors: for matrix (S) b -S w Perform eigenvalue decomposition to find the eigenvectors v1, v2, ..., v1 corresponding to the r largest eigenvalues. r The matrix v formed 1 ={v1,v2,...,v r }; These feature vectors are reconstructed into a two-dimensional convolutional kernel matrix in the first layer, as shown in the formula. Where L i Let W be the number of convolutional kernels in the i-th layer. i j This represents the i-th convolutional kernel in the j-th convolutional layer; As a reconstruction function, the vector Convert to convolution kernel The image features obtained after convolution by the first convolutional kernel are represented as follows: in, This represents a convolution operation; zero-padding is performed around the image. In the second convolutional layer, the feature map F is defined. i l The j-th non-overlapping vectorized patch in the data is Normalize these vectorized patches to obtain in Represented as a normalized image patch vector, Y l Let represent the image features after convolution by the l-th convolution kernel. Then, all feature maps are represented as follows: Step 6: The second convolution yields the output feature map; specifically, it includes the following process: The second convolutional layer is trained in the same way as the first layer; the second convolutional kernel vector is obtained by applying formula (7), and then it is reconstructed into the convolutional kernel matrix W. l 2 This process is represented as Assuming the second layer has L2 convolution kernels, for N original training images, a total of L2 feature maps are obtained after the second convolution. Step 7: Binarize the output feature map; Step 8: Divide the feature map into blocks, calculate the corresponding histograms, and concatenate them into a complete vector; Step 9: Input these feature output vectors into the classifier for training, and output the classification results for the test image.

2. The face recognition method based on multi-layer cascaded boundary median discriminant projection according to claim 1, characterized in that, Step 1 includes the following process: Assume there are N sample images of size m×n in class C. Using each pixel as the center point, extract blocks of size k1×k2 from the i-th face image, and collect all blocks to obtain the result. x here i,j Image I i The j-th block, and symbol It takes the largest integer not exceeding z; for x i,j After removing the mean, we get: in This refers to the block after demeaning, where 1 represents a vector of all 1s of the corresponding size; Combined, forming a sample matrix, we obtain:

3. The face recognition method based on multi-layer cascaded boundary median discriminant projection according to claim 1, characterized in that, Step 2 includes the following process: Let M i Let be the median vector of the i-th class of samples, and its calculation process is as follows: Let the i-th class of samples be: in This represents the a-th sample in class b. This represents the c-th block of the a-th sample in class b, where n i M is the number of training samples in each class; then M i The j-th row of data is calculate The process is to Sort in ascending order.

4. The face recognition method based on multi-layer cascaded boundary median discriminant projection according to claim 3, characterized in that, calculate Time assumption If n i If it is an odd number, then If n i If it is an even number, then 5. The face recognition method based on multi-layer cascaded boundary median discriminant projection according to claim 1, characterized in that, Step 3 includes the following process: The formula for calculating the distance from the i-th sample to the median sample of its class j is: Assuming G1 is the number of samples from the same class that are farthest from the class median, and G2 is the number of samples from different classes that are nearest neighbors of the class median, the set of samples from the same class that are farthest from the class median in each class is calculated using the distance formula, and is defined as follows: Calculate the set of out-of-class samples that are closest to the class midpoint in each class of samples, defined as...

6. The face recognition method based on multi-layer cascaded boundary median discriminant projection according to claim 1, characterized in that, Step 4 includes the following process: The intra-class scatter matrix and the inter-class scatter matrix S are calculated using the following formulas. w With S b : in Used to characterize the importance of other samples in the same class, t is the heat kernel coefficient, and λ is the coefficient.

7. The face recognition method based on multi-layer cascaded boundary median discriminant projection according to claim 1, characterized in that, Step 7 includes the following process: The heaviside stage function is used to binarize these output feature maps, and the L2 feature maps are weighted and summed to map them into a single feature map. This process can be represented as: in, This represents the Heaviside phase function, which converts all values ​​greater than 0 to 1 and all values ​​less than or equal to 0 to 0. This represents the feature map after weighted summation mapping, where each pixel is in... The range of values ​​is 8. The face recognition method based on multi-layer cascaded boundary median discriminant projection according to claim 1, characterized in that, Step 8 includes the following process: For each mapped feature map Divide it into B blocks and calculate the corresponding histogram for each block, and finally combine them into a single block. The corresponding histogram vectors are concatenated into a complete vector; after this process, the final original input image I is obtained. i The corresponding features are represented as follows: Among them, f i This represents the final feature output vector of the i-th input image.

Images