Robust semi-supervised multi-view face recognition system

Abstract

The invention discloses a robust semi-supervised multi-view face recognition system (RS2 Mv), and belongs to the field of face recognition. The method comprises the following steps: firstly, representing a collected face image through various types of features, and describing the face image from different feature levels; secondly, decomposing a feature matrix corresponding to each type of features into a pure data matrix and a noise matrix; then, learning self-representation subspaces corresponding to view angle features based on pure data obtained through decomposition; and finally, respectively optimizing each parameter through a coordinate descent method in an iterative optimization process until the system converges, and spreading label information of a labeled face image to a non-labeled face image through an image fused by the self-representation subspaces of multiple visual angles to complete face recognition. According to the method, the recognition precision of the face recognition system on the face image doped with noise data can be effectively improved, and accurate recognition of the non-label face image can be completed under the condition that only a small number of labeled face images exist.

Description

Technical Field

[0001] The present invention relates to the field of face recognition, and in particular to a robust semi-supervised multi-view face recognition system. Background Art

[0002] With the development of machine learning, machine learning related technologies have become the mainstream technical means for face recognition. However, due to the widespread use of optical sensors, the amount of face data that face recognition systems need to process has increased dramatically, and it costs a lot of money to manually label these massive face data. Semi-supervised learning has become the main technical means for face recognition systems because it only needs a few labeled face images to complete all face image recognition. In addition, since optical sensors are affected by the aging of internal components and interference from the external working environment, the face image data they collect is prone to contain various types of noise, which further affects the recognition performance of the face recognition system. Therefore, it is of great application value to design a face recognition system that can work in a scenario with only a few labeled face images and is robust to noise.

[0003] On the other hand, a large number of studies have shown that compared with using only a single type of feature to describe the facial image information for processing, more accurate face recognition results can be obtained by describing and representing the collected facial image information through multiple types of features and making comprehensive use of them. Therefore, when designing a face recognition system, we should also consider how to better utilize multiple types of features. In multi-view learning, each type of feature can be regarded as a perspective of the facial image sample. Under the guidance of the complementary principle and consensus principle of multi-view learning, a face recognition system with better recognition performance can be designed. Among them, the complementary principle means that each perspective of multi-view data contains specific knowledge that other perspectives do not have, thus playing a complementary role; the consensus principle means that although the representation forms of each perspective of multi-view data are different, since each perspective describes the same sample, the output results of each perspective should maintain consensus, that is, consistent face recognition results should be obtained through different perspective features. Summary of the invention

[0004] In view of this, the present invention provides a robust semi-supervised multi-view face recognition system to improve the robustness of existing face recognition systems to noisy images, and further enhance the ability to utilize multi-view features to improve face recognition performance.

[0005] The technical solution of the present invention is as follows:

[0006] Firstly, the n collected images are represented by V different types of feature matrices, each type of feature can be regarded as a perspective of the face image, where the columns of the matrix represent the features of the corresponding face image; secondly, the original feature matrix is ​​decomposed into a pure data matrix and a noise matrix; then, the self-representation subspace of each perspective is learned based on the self-representation idea of ​​the pure data matrix of the data; then, the adjacency graph of each perspective is constructed based on the learned self-representation subspace, and the multi-perspective consensus graph is obtained by fusion with the same average weight; then, the label information of the labeled face image is propagated to the unlabeled face image through the multi-perspective consensus graph, and the label indicator matrix of all face images is constructed; finally, the above steps are integrated into a unified learning framework, and the coordinate descent method is used to iteratively optimize each parameter until convergence to obtain the final recognition result of the unlabeled face image.

[0007] Compared with the prior art, the beneficial effects of the present invention are:

[0008] In the learning process of the self-representative subspace, only the subspace corresponding to the pure data obtained by decomposition is learned, which can effectively cope with the noise influence caused by external working conditions or internal interference of the sensor, improve the robustness of the face recognition system to noise, and thus greatly improve the recognition accuracy of the face recognition system.

[0009] Since this system adopts the idea of ​​self-representative subspace to learn the subspace of each perspective, compared with the conventional graph-based semi-supervised face recognition system, it reduces the high manual parameter adjustment cost of adjusting the Gaussian kernel width parameter.

[0010] The noise decomposition module, self-representation subspace learning module and semi-supervised classification module of this system are integrated into a unified learning framework in a mutually coupled manner. Each parameter can obtain the optimal solution under the influence of other parameters to ensure the performance of the face recognition system. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The overall flow chart of the present invention is shown in the attachment Figure 1 . DETAILED DESCRIPTION

[0012] In order to more clearly describe the content of the present invention, the present invention is described in detail below in combination with the embodiments and drawings, but the examples mentioned below are not intended to limit the scope of the present invention. The present invention relates to a robust semi-supervised multi-view face recognition system, comprising the following steps:

[0013] Step 1: The collected labeled face images and unlabeled face images are represented by V different types of features: where d v represents the feature dimension of the v-th perspective, They represent labeled face images and unlabeled face images, and the corresponding sample numbers are nl 、n u And n l +n u =n.

[0014] Step 2: Convert the original feature matrix X v Decompose into a clean data matrix and the noise matrix E v ,Right now And the noise matrix is ​​calculated by l 2,1 The norm is constrained, which is specifically expressed as: in:

[0015] Step 3: The clean data matrix obtained by decomposing the original data from each perspective Adopt the idea of ​​self-representation to learn its self-representation subspace S v , specifically expressed as: Among them, diag(S v ) indicates S v The diagonal elements of , 1 represents a vector whose elements are all 1, λ is a regularization parameter greater than 0, |||| F represents the F-norm of the corresponding matrix.

[0016] Step 4: Self-representation subspace S of each learned view v Basic construction of the adjacency graph W of each perspective v , and then fuse the adjacency graphs W of each view with the same average weight v The final multi-view consensus graph W is obtained, namely:

[0017] Step 5: Propagate the label information of the labeled face image to the unlabeled face image through the multi-view consensus graph W, and construct the label indicator matrix of the unlabeled face image Finally, the label indicator matrix of all face images is formed Where c is the number of face categories, F (l) =Y (l) , Based on n l A one-hot label matrix constructed from the label information of labeled face images.

[0018] Step 6: Combine the above steps into a unified learning framework, expressed as: Among them, Y i is the i-th row of the one-hot label matrix Y of the labeled face image, β and γ are regularization parameters greater than 0 for the corresponding items, L W is the Laplacian matrix of the multi-view consensus graph W, defined as: L W =DW, (7) D is the diagonal element. The diagonal matrix of .

[0019] Step 7: Use the coordinate descent method to optimize and solve each parameter in the system until the system converges. The specific steps are:

[0020] Step 7.1: Optimize the noise matrix E v and clean data matrix

[0021] Step 7.1.1: Optimize the noise matrix E, treating other parameters as constants v , then formula (6) is about E v The optimization problem becomes: Where A=(IS v )(IS v ) T ,

[0022] Step 7.1.2: Let (8) be about E v The derivative of is 0, then we can get E v The closed-form solution at the current optimization iteration is: E v =X v A(A+2γB) -1 . (9)

[0023] Step 7.1.3: Find E v After the closed solution of the current iteration, the pure data of the current iteration is further obtained:

[0024] Step 7.2: Optimize the self-representation subspace S v :

[0025] Step 7.2.1: Connect with S v If the irrelevant parameters are regarded as constants, then Equation (6) is v The optimization problem becomes:

[0026] Ignore S vThe constraint condition of S v The optimization problem becomes: Since each S in formula (11) v The optimization is performed independently, so for simplicity, the superscript v is omitted in the expression of formula (12), and

[0027] Step 7.2.2: Let the derivative of equation (12) with respect to S be 0, and we can find the closed solution of S in the current iteration without considering the constraints:

[0028] Step 7.2.3: Further consider the constraints of S, then: Formula (14) can be used to obtain the optimal solution of S under the constraints by constructing its Lagrangian function and combining it with the KKT condition: Where S ii =0, is the Lagrange multiplier.

[0029] Step 7.3: When other variables are known, equation (6) becomes:

[0030] According to the labeled face images and unlabeled face images in the face dataset, the Laplace matrix L W Block processing: Where l and u are the number of labeled face images and unlabeled face images, respectively. l 、n u , then the optimal solution of formula (17) is: where Y (l) It is a one-hot label matrix constructed based on the real label information of labeled face images.

[0031] Step 7.4: Repeat steps 7.1-7.3 until equation (6) converges and the system optimization is completed.

[0032] Step 8: After the system is optimized, the recognition process of unlabeled face images is: System Verification

[0033] The present invention uses the face image benchmark dataset ORL to verify the face recognition effect of the present invention. The dataset contains 400 face images of 40 different volunteers. Similar to the general practice, the present invention uses face image pixel values, LBP features and Gabor features as the three perspectives of the ORL image dataset during the verification process.

[0034] The present invention includes three regularization parameters λ, β and γ, respectively, wherein the search range of λ and β is [10 -3 ; 5*10 -3 ; 10 -2 ; 5*10 -2 ; 10 -1 ], the search range of γ is [10 -2 ; 10 -1 ; 10 0 ; 10 1 ; 10 2 ].

[0035] In order to verify the recognition accuracy of the system for unlabeled face images under different proportions of labeled face images, the present invention completes the recognition of unlabeled face images when the proportion of labeled face images (lr) is 10%, 15%, 20%, 25% and 30%, and uses the accuracy rate ACC as the evaluation index. The higher the ACC, the more accurate the face image recognition. In order to avoid errors caused by randomness in the verification process, the average value of 10 recognition results will be displayed, and these 10 verification processes are carried out independently.

[0036] Comparison model: The system involved in the present invention is named RS2Mv, and the models used for performance comparison with the present invention are AMGL, MVAR, MLAN, SSC_IHIC, and LACK, which are all current mainstream multi-view semi-supervised face recognition models.

[0037] Table 1 shows the performance comparison results of the present invention and the current mainstream semi-supervised face recognition model algorithm, where the standard deviation of the running results is shown in brackets, and the best face recognition result is marked in bold.

[0038] Table 1:

[0039] The face recognition results in Table 1 show that the present invention can achieve the best face recognition performance under different proportions of labeled face images. The reason for this phenomenon is that the present invention can decompose the noise in the original data to obtain pure noise during the learning process of the model, and further construct a self-representation subspace based on the learned pure noise, thereby eliminating the influence of noise and obtaining more accurate face recognition results. The above experimental phenomena and analysis verify the effectiveness and practicality of the present invention.

[0040] The above describes the specific implementation of the present invention according to the drawings, which is not limited to the above examples. Those skilled in the art can make several equivalent substitutions based on understanding the principles of the present invention, but they should all fall within the protection scope of the present invention.

Claims

1. A robust semi-supervised multi-view face recognition system, characterized by: The specific steps are: 1) Preprocessing: The collected n face images are represented by V different types of features as where d v represents the feature dimension of the v-th perspective, They represent labeled face images and unlabeled face images, and the corresponding sample numbers are n l 、n u And n l +n u =n; 2) The first step of training: the original feature data X v Decompose into a clean data matrix and the noise matrix E v ,Right now 3) The second step of training: the pure data matrix obtained by decomposing the original data from each perspective Adopt the idea of ​​self-representation to learn its self-representation subspace S v ; 4) The third step of training: self-representation subspace S of each learned perspective v Based on the adjacency graph W of each perspective v , and then fuse the adjacency graphs W of each view with the same average weight v Get the final multi-view consensus graph W; 5) Training Step 4: Propagate the label information of the labeled face image to the unlabeled face image through the multi-view consensus graph W, and construct the label indicator matrix of the unlabeled face image Finally, the label indicator matrix of all face images is formed Where c is the number of face categories, F (l) =Y (l) , Based on n l A one-hot label matrix constructed from the label information of labeled face images; 6) Face recognition process: The above steps are combined in a unified face recognition system, and the coordinate descent method is used to optimize each parameter in the system until the system converges. Finally, the label indicator matrix F of the unlabeled face image is obtained. (u) The position where the maximum value of each row appears is the category to which the corresponding face image belongs.

2. A robust semi-supervised multi-view face recognition system according to claim 1, characterized in that: The noise matrix E v By l 2,1 The norm is constrained and expressed as in:

3. A robust semi-supervised multi-view face recognition system according to claim 1, characterized in that: The clean data matrix The self-expressive subspace S v The learning process is expressed as: Where diag(S v ) indicates S v The diagonal elements of , 1 represents a vector whose elements are all 1, λ is a regularization parameter greater than 0, ||·|| F represents the F-norm of the corresponding matrix.

4. A robust semi-supervised multi-view face recognition system according to claim 1, characterized in that: The adjacency graph W of each perspective v The construction method is:

5. A robust semi-supervised multi-view face recognition system according to claim 1, characterized in that: The adjacency graphs W of each view are fused with the same average weight. v The method to obtain the multi-view consensus graph W is:

6. A robust semi-supervised multi-view face recognition system according to claim 1, characterized in that: The objective function of the unified face recognition system is: Among them, Y i is the i-th row of the one-hot label matrix Y of the labeled face image, β and γ are the regularization parameters of the corresponding items, both greater than 0.

7. The face recognition system formula according to claim 6, characterized in that: The L W is the Laplacian matrix of the multi-view consensus graph W, defined as: L W =D-W, Among them, D is the diagonal element. The diagonal matrix of .

8. According to the face recognition process described in 6) of claim 1, the method for determining the category to which the unlabeled face image belongs is specifically:

Images