A small sample multi-pose face recognition method based on hypergraph and multi-task collaboration

Abstract

The present application relates to a kind of based on hypergraph and multi-task cooperation small sample multi-pose face recognition method, belong to artificial intelligence face recognition technical field.The present application utilizes hypergraph and non-negative matrix decomposition to obtain image similar to frontal image, designs a kind of multi-pose face recognition framework based on hypergraph deflection.The framework is first separated to no pose deflection image.On this basis, a kind of feature coding method based on improved support vector description is proposed, the feature of no pose deflection image is extracted, and is optimized with the classifier based on dictionary learning, for feature extraction and feature classification.Feature coding method utilizes improved support vector data description and triangular coding, so that the feature extracted is more discriminative.At the same time, an effective feature extraction and feature classification optimization model is established, easy to obtain more close to global optimal solution, improves the recognition performance.

Description

TECHNICAL FIELD

[0001] The present application relates to a face recognition method, in particular to a small sample multi-pose face recognition method based on hypergraph and multi-task cooperation, and belongs to the technical field of artificial intelligence face recognition. BACKGROUND

[0002] Face recognition as a new identity recognition technology, under the wave of artificial intelligence, develops rapidly in recent years and becomes a popular research field. Among them, multi-pose face recognition mainly studies the influence of face angle change on face recognition effect. How to solve the influence of pose on face recognition and improve the multi-pose face recognition effect will help the face recognition technology to be applied to more identity recognition scenes.

[0003] The existing method based on pose correction mainly converts multi-pose face into frontal face. At present, the method of fitting frontal face includes two kinds: one is affine transformation method, which transforms multi-pose face into frontal face, the transformation relationship is simple, but it will cause the loss of face classification feature information, and the recognition effect is not ideal. The other is virtual fusion method, which fuses multiple multi-pose face image information to construct frontal face image, which corrects the pose to a certain extent, and then puts the classification error into the objective function, so as to realize the classification ability of face.

[0004] Although the method based on pose correction has achieved good results in processing face recognition and some scenes, it can better solve the recognition problem of multi-pose small sample face image. But in practical application, because the pose deflection image is difficult to reveal the relationship between multiple samples, these multi-pose images of the same kind pose great challenge to these methods.

[0005] In summary, the existing small sample multi-pose face recognition method has defects and limitations. First, the image features without pose deflection are not easy to extract. Second, the extracted features are not high in discriminability. Therefore, the existing method cannot reveal the relationship between the same samples, and it is very difficult to efficiently learn the dictionary. SUMMARY

[0006] The purpose of the present application is to effectively solve the technical problems of multi-pose small sample face recognition, and creatively propose a small sample multi-pose face recognition method based on hypergraph and multi-task cooperation.

[0007] Inspired by non-negative matrix factorization, each image involving pose changes can be decomposed by non-negative matrix factorization, and one of the matrices obtained by decomposition is used as an image without pose deflection, and the other matrix is used as a pose change matrix. Finally, the image without pose deflection is obtained through multiple iterative decompositions. Compared with other single-resolution image recognition methods, these methods can better process multi-resolution image recognition problems. Hypergraph can represent the complex relationship between samples. Inspired by hypergraph, each image is regarded as a node in the hypergraph, and the hypergraph formed by multiple images is subjected to non-negative matrix factorization to extract images with good performance and without pose deflection.

[0008] On this basis, the application first proposes a small sample multi-pose face recognition method based on hypergraph and multi-task cooperation.

[0009] The innovation of the application lies in: a hypergraph deflection and multi-task cooperative optimization method is used, and hypergraph and non-negative matrix factorization are used to obtain images similar to front images. On this basis, an improved support vector description based feature encoding method is proposed, and a dictionary learning based classifier is jointly optimized for feature extraction and feature classification.

[0010] (1) A multi-pose face recognition framework based on hypergraph deflection is proposed. The framework first separates the images without pose deflection, then uses the improved support vector data description based feature encoding method to extract the features of the images without pose deflection, and recognizes the extracted features.

[0011] (2) An improved support vector description based feature encoding method is proposed. The feature encoding method uses improved support vector data description and triangular coding to make the extracted features more discriminative.

[0012] (3) An effective feature extraction and feature classification optimization model is established, which is easy to obtain a solution closer to the global optimum, and helps to improve the recognition performance of the algorithm.

[0013] Advantages

[0014] Compared with the prior art, the application has the following advantages:

[0015] 1. The application introduces hypergraph and embeds it into the non-negative matrix factorization process, which more comprehensively reveals the relationship between samples of the same type.

[0016] 2. The application designs an improved support vector data description based feature encoding method to improve the discriminative ability of the extracted features. DETAILED DESCRIPTION

[0017] Figure 1 The figure is a hypergraph deflection and multi-task cooperative optimization schematic diagram in the application.

[0018] Figure 2 The flow chart of the face recognition process in the present application.

[0019] Figure 3 The schematic diagram of the process of extracting the frontal image.

[0020] Figure 4 The principle diagram of coding. DETAILED DESCRIPTION

[0021] The present application will be further described in conjunction with the embodiments, so as to make the technical content more clear and convenient to understand. The described embodiments are only representative embodiments of the present application, but not all embodiments. Based on the embodiments in the present application, all belong to the scope of protection of the present application.

[0022] A small sample multi-pose face recognition method based on hypergraph and multi-task cooperation, including image frontalization, feature extraction, feature classification and joint optimization, specifically as follows:

[0023] Step 1: image frontalization processing.

[0024] As shown in Figure 1 , based on non-negative matrix factorization and hypergraph embedding, feature discrimination enhancement is carried out. Including the following steps.

[0025] Step 1.1: non-negative matrix factorization.

[0026] Given any non-negative matrix , it is decomposed into the sum of two non-negative matrices , , as follows:

[0027]

[0028] Wherein, , is the base matrix, is the sub-matrix, F represents the Frobenius norm, T represents the matrix transpose, denotes the dimension symbol, m represents the dimension of the sample , n represents the sample number of the source data, r represents the sample number of the target data. s.t. is used to prompt that the following is a constraint term.

[0029] Then, update and :

[0030]

[0031] Wherein, denotes the source data, denotes the transformation matrix, ij denotes the ith iteration, and jk denotes the jkth iteration.

[0032] Step 1.2: Perform hypergraph embedding.

[0033] Hypergraph is an ordered binary group . is a non-empty set of nodes / vertices, called vertex set. is a non-empty subset of the vertex set, whose elements are called hyperedges.

[0034] Unlike ordinary graphs, each edge of a hypergraph not only connects two vertices, but also can connect more vertices.

[0035] Given a hypergraph , is a finite set of data points, where ( ) is a vertex. is a hyperedge, and the hyperedge set satisfies the following conditions:

[0036]

[0037]

[0038] where t represents the hyperedge number.

[0039] Each hyperedge has a corresponding weight , and the vertex hyperedge will form an incidence matrix , the elements of which are calculated as follows:

[0040]

[0041] In the hypergraph , the degree of each vertex is defined as the sum of the weights of the hyperedges it belongs to. The degree of a hyperedge is defined as the number of nodes it belongs to, which is calculated as follows:

[0042]

[0043] where denotes the hyperedge weight.

[0044] Let be a diagonal matrix whose main diagonal elements are , . Let and be the matrices obtained by and diagonal matrix generated respectively, .

[0045] Then, the unregularized hypergraph Laplacian matrix of is calculated by :

[0046]

[0047] where, denotes the inverse of the diagonal matrix .

[0048] Further, the present application proposes a new feature coding method, which can obtain features of each image with little pose deflection, and the features have good class resolution. The specific implementation is as follows:

[0049] The given data set is represented as ,where each column represents an image sample.

[0050] First, for each image, remove the noise in the image (a Gaussian filter can be applied to remove the noise).

[0051] Then, check whether the pixels of each image are negative, assign the negative values to 0, and keep the original values of the positive values, to obtain ,which represents the preprocessed image set.

[0052] Then, the unregularized hypergraph Laplacian matrix of is constructed . Let the number of hypergraph edges be , the number of hypergraphs be , = , and the number of vertices contained in each hyperedge be . The vertices contained in each hyperedge are generated by itself and its nearest neighbors, denotes the column of .

[0053] Specifically, the calculation method of is as follows:

[0054]

[0055]

[0056] where exp() represents the exponential function.

[0057] obtain and​​ Then, the objective function is as follows:

[0058]

[0059] in, , , , . express The error caused by the nonnegative decomposition. The regularization term for the hypergraph preserves the local geometry of the data and improves algorithm performance. Since the above is difficult to solve, an iterative solution method is directly used to solve the problem. The Lagrange function corresponding to the above equation... for:

[0060]

[0061] in, for correspond A matrix composed of Lagrange multipliers express The mkth iteration. For the reason correspond A matrix composed of Lagrange multipliers express The nkth iteration; Tr represents the coefficients, and Tr represents the trace of the matrix.

[0062] For ease of calculation, the above formula... Rewritten as:

[0063]

[0064] in, . This represents an example matrix.

[0065] By taking separately and The partial derivatives yield:

[0066]

[0067]

[0068] According to the KKT conditions (Karush-Kuhn-Tucker conditions). and ,get:

[0069]

[0070] In the above formula, the subscript of each variable represents the iteration number of the variable. and Update by the following way:

[0071]

[0072]

[0073] where, denotes the element-wise multiplication of two matrices. The output is the set of images without pose deflection. denotes the diagonal matrix.

[0074] Figure 3 The process of extracting near-frontal images from images involving pose variation is shown. denotes the set of original images with pose deflection, denotes the set of images obtained after preprocessing , denotes the set of images of approximate frontal images obtained by decomposition and iteration, denotes the pose transformation matrix. In Figure 3 , first, each image in the original image set is preprocessed to obtain a non-negative image set without noise pollution. Then, the hypergraph is embedded into the non-negative matrix decomposition to maintain the structure of the decomposed images. Finally, through matrix decomposition and iterative update, a set of images with almost no deflection is obtained.

[0075] Step 2: Feature extraction. Including the following steps.

[0076] Step 2.1: Feature encoding based on improved support vector data description.

[0077] Firstly, the present application proposes an improved support vector data description, which is used to obtain the spherical center and radius of each cluster. Then, the radius and center of the sphere corresponding to each cluster are used for feature encoding.

[0078] The existing support vector data description considers that each data point plays the same role when calculating the radius of each cluster, but this does not conform to the actual situation. Therefore, the present application assigns a learned weight to each data in model learning, and proposes an improved support vector data description, as follows:

[0079]

[0080] where, denotes the radius of the sphere, denotes the a sample, denotes the weight of the i-th element of the vector denotes the center of the ball, denotes the number of samples, denotes the slack variable, is a parameter.

[0081] For convenience of solving, the above equation is written in the form of Lagrange function:

[0082]

[0083] wherein, denotes the Lagrange function, denotes the corresponding Lagrange factor, , denote the i-th element of the vector

[0084] Let and , we get:

[0085]

[0086] wherein, , , , , is the i-th sample and the j-th sample after removing the pose deflection. , , is obtained using a linear algorithm, and T denotes transposition.

[0087] denotes the vector composed of the radii of the SVDD clustering ball, and is obtained by the following equation:

[0088]

[0089] wherein, is the set of support vectors, i.e., the sample points used in the above equation are support vectors. Whether the sample point is a support vector or not, it needs to satisfy the condition: if the sample point is a support vector, then its corresponding is not zero. , is the number of clusters in the data set.

[0090] Step 2.2: Triangular coding.

[0091] For each image after removing the pose deflection, it is decomposed into blocks, and each block is coded. ​​​

[0092] In particular, for an image without pose deflection , it is decomposed into blocks.

[0093] For any block , , denotes the number of decomposed blocks included in the image, is encoded as :

[0094]

[0095] where , , . and are obtained by triangular encoding.

[0096] , denotes the distance from to , denotes the center of the SVDD ball formed by the th cluster, is the mean of all . , , is the mean of all .

[0097] Figure 4 shows the schematic diagram of encoding. denotes the image the th block, the image is divided into blocks. denotes the center of the SVDD ball formed by the th cluster, a plurality of sample points are clustered into a cluster, denotes the radius of the SVDD ball formed by the th cluster, denotes the distance between and . denotes the center of the SVDD ball formed by the th cluster. is the radius of the SVDD ball formed by the th cluster. denotes the distance between and .

[0098] Therefore, the image is encoded as , and the expression is as follows:

[0099]

[0100] Step 3: Feature classification. Including the following steps.

[0101] By the deflection and feature coding operations of steps 1 and 2, the influence of pose variation on face recognition is greatly reduced. On this basis, in order to further improve the recognition rate of the algorithm, the test sample is represented by learning the dictionary and using the learned dictionary, and the category of the test sample is determined according to the representation residual.

[0102] Specifically, the model of the dictionary learning-based classifier is as follows:

[0103]

[0104] Wherein, is the training sample, is the learned dictionary, is the representation coefficient, represents the th atom in . F represents the Frobenius norm.

[0105] Step 4: Joint optimization stage. Including the following steps.

[0106] In order to obtain the global optimal solution of HDMCO (HDMCO, which represents the abbreviation of the proposed algorithm), the feature extraction and feature classification are jointly optimized, which is as follows:

[0107]

[0108] Thus, , and are obtained. The representation coefficient.

[0109] Wherein, is obtained by the following formula:

[0110]

[0111] Using a linear algorithm, the value of is obtained.

[0112] is obtained by the following formula:

[0113]

[0114] The above transformation is solved as follows:

[0115]

[0116] where J represents an intermediate matrix, represents a coefficient, represents the i-th atom of the matrix V.

[0117] The variable in the above formula is solved by iteration.

[0118] The solution is obtained by solving the following formula:

[0119]

[0120] The solution of the above formula is as follows:

[0121]

[0122] wherein, x represents an example variable. represents a coefficient.

[0123] Step 5: using the learned classifier to predict the classification result, classifying the sample, and realizing face recognition.

[0124] The information and data involved in the present application are all authorized by the user and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Claims

1. A small sample multi-pose face recognition method based on hypergraph and multi-task collaboration, characterized in that, The method comprises image frontalization, feature extraction, feature classification and joint optimization. Step 1: image frontalization processing; Based on non-negative matrix decomposition and hypergraph embedding, feature discrimination enhancement is performed, including the following steps. Step 1.1: non-negative matrix decomposition is performed; Given any nonnegative matrix X0, factorizes into two nonnegative matrices P T as follows: wherein, is a base matrix, is a sub-matrix, F denotes a Frobenius norm, T denotes a matrix transpose, denotes a dimension symbol, m denotes a dimension of a sample X0, n denotes a sample number of source data, r denotes a sample number of target data; s.t. is used to prompt that a constraint term follows. Then, update and P T : Wherein, X represents source data, P represents a transformation matrix, ij represents the ith iteration, and jk represents the jk iteration; Step 1.2: hypergraph embedding is performed; The hypergraph G is an ordered binary group G=(V, e); V is a non-empty set with nodes / vertices as elements, which is called the vertex set; e is a cluster of non-empty subsets, and its elements are called hyperedges; Each edge of the hypergraph not only connects two vertices, but also connects more vertices; Given a hypergraph G = (V, e), V = {v1, v2, …, vk} is a set of finite data points, where v k} is a set of finite data points, where v i (i = 1, 2, …, k) is a vertex; e j is a hyperedge, and the hyperedge set e satisfies the following conditions: e1∪e2∪e3…∪e t =V Wherein, t represents the hyperedge number; Each hyperedge e i has a corresponding weight w j , and the vertex hyperedges will form an incidence matrix The elements of the matrix are computed by the following formula: In a hypergraph G, the degree d of a vertex is the number of hyperedges it belongs to i is defined as the sum of the weights of the hyperedges it belongs to; the degree p of a hyperedge is the number of vertices it belongs to i is defined as the number of vertices it belongs to, which is calculated as follows: where w j denotes the super-edge weight; Let D v be a diagonal matrix whose main diagonal elements are Let D e and W be diagonal matrices generated by ρ j and w j , respectively, j = 1, 2, …, t; Then, the un-regularized hypergraph Laplacian L is calculated by H : wherein denotes the inverse of the diagonal matrix D e ; Step 2: feature extraction; Step 3: feature classification; Step 4: joint optimization; Step 5: using the learned classifier to predict the classification result, classifying the samples, and realizing face recognition.

2. The method of claim 1, wherein the method is based on hypergraph and multi-task collaboration. In step 1, a feature coding method is used to obtain the features of each image with little pose deflection, and the features have good class discrimination, as follows: The given dataset is represented as Each column in Y represents an image sample; First, for each image, remove the noise in the image; Then, check whether the pixel of each image is negative, assign the negative value as 0, and the positive value remains the original value, to obtain Y W , Y W represents the pre-processed image set; After that, construct Y W The non-regularized hypergraph Laplacian matrix L H of Y n W , Y n W represents the n-th column of Y W ; w j The calculation method is as follows: Wherein, exp() represents the exponential function; Y is obtained W and L H After that, the objective function is as follows: wherein represents Y W the error resulting from the non-negative factorization of P T L H P) represents a regularizer term for the hypergraph Using the iterative solution method, the Lagrange function Δ corresponding to the above formula is: where Ψ is Ψ mk corresponding to the Lagrange multipliers λ is a matrix composed of the Lagrange multipliers λ mk denotes the mth iteration of Ψ; Φ is a matrix composed of the Lagrange multipliers λ nk corresponding to P mk ≥ 0, Φ nk denotes the nth iteration of Φ; λ denotes a coefficient; and Tr denotes a trace of a matrix. Rewrite Δ of the above formula as: wherein B represents, for example, a matrix; By taking the partial derivatives of Δ and respectively, we obtain: According to and Φ nk P nk = 0, we get: In the above formula, the subscript of each variable indicates the iteration number of the variable; and P nk by updating: wherein, denotes element-wise multiplication of two matrices; output is a set of images without pose deflection; D v denotes a diagonal matrix.

3. The method of claim 1, wherein the method is based on hypergraph and multi-task collaboration. The implementation method of step 2 includes: Step 2.1: feature coding based on improved support vector data description; A support vector data description is used to obtain the center and radius of each cluster; then, the radius and center of each cluster corresponding to the ball are used for feature coding; A learned weight is assigned to each data in model learning, and a support vector data description is proposed, as follows: where r denotes the radius of the ball, y i denotes the i-th sample, p(y i ) denotes the weight of y i , b denotes the center of the ball, num denotes the number of samples, x i denotes the relaxation variable, is a parameter; Write the above formula into the form of Lagrange function: wherein denotes the Lagrangian function, a, b denote the respective Lagrange factors, a i , b i denote the i-th element of a and b, respectively; Let and resulting in: s.t.α T e=1 where Q = ( <x i , x j > ) num×num , Ω = ( <x i , x j > ) num×1 , e = (1, 1, 1, …, 1) T , x i , x j are the i-th sample and the j-th sample without pose deflection; α = [α1, α2, …, α num ], α is obtained using a linear algorithm, and T represents transposition. r represents the radius of the SVDD clustering ball, which is obtained by the following formula: Where Υ is the set of support vectors, meaning the sample points used in the above formula are support vectors; regardless of whether a sample point is a support vector, the condition must be satisfied: if sample point y i If it is a support vector, then its corresponding α i Non-zero; r = [r1, r2, ..., r C ], where C is the number of clusters in the dataset; Step 2.2: triangular coding is performed; For each image removed from the pose deflection, it is decomposed into blocks, each of which is encoded; For an image q with pose deflection removed, it is decomposed into blocks; for any block q j , denotes the number of decomposed blocks the image comprises, is encoded as U(q j ): U(q j ) = [U1(q j ) U2(q j )... Un(q C )... Un(q j )] T where U i (q j ) = [U i,1 (q j ) U i,2 (q j )], i = 1, 2,..., C, U i,1 (q j ) and U i,2 (q j ) are obtained by triangular coding; U i,1 (q j ) = max{0, d(s) - s i (q j )}, s i (q j ) = ||q j - o i ||2, s i (q j ) denotes the distance from q j to o i , o i denotes the center of the SVDD ball formed by the jth cluster, d(s) is the mean of all s i (q j ); U i,2 (q j ) = max{0, A(m) - m i (q j )}, is the mean of all m i ; where q j denotes the jth block of image q, image q is divided into blocks; O i denotes the center of the SVDD ball formed by the ith cluster, a plurality of sample points are clustered into a cluster, r i denotes the radius of the SVDD ball formed by the ith cluster; O j denotes the center of the SVDD ball formed by the jth cluster; r j is the radius of the SVDD ball formed by the jth cluster; The image is encoded as F q , the expression is as follows: The implementation method of step 3 is as follows: By learning the dictionary and using the learned dictionary to represent the test sample, the class of the test sample is determined according to the residual representation; The model of the dictionary learning-based classifier is as follows: where X is the training sample, D is the learned dictionary, Z is the representation coefficients, d i denotes the i-th atom in D; F denotes the Frobenius norm; The implementation method of step 4 is as follows: Joint optimization of feature extraction and feature classification is performed, as follows: Thus, α, D and Z are obtained; Z represents the coefficient; Wherein, α is obtained by the following formula: s.t. a T e = 1 The value of α is obtained by using a linear algorithm; D is obtained by the following formula: Convert the above formula into a solving form, as follows: J=J+D-V where J denotes an intermediate matrix, Θ denotes a coefficient, and v i denotes the i-th atom of the matrix V; D is obtained by iteratively solving the variable in the above formula; D is obtained by solving the following formula: The solution of the above formula is as follows: Wherein, shrink(x, a)=signmax(|x|-a, 0), x represents an example variable; η represents a coefficient.

Images