Multi-dimensional feature fusion clustering method for visual angle missing face recognition

Abstract

A multi-dimensional feature fusion clustering method for visual angle missing face recognition belongs to the field of face image data clustering processing in computer vision, and comprises the following steps: firstly, constructing a kernel matrix for incomplete multi-visual angle face image data, and performing multi-kernel completion; secondly, mapping the kernel matrix under each view angle to different dimension spaces through feature decomposition to obtain feature matrixes of different dimensions, and fusing the feature matrixes into a consistent representation matrix of a fixed dimension through a rotation matrix; and performing tensor subspace learning by taking the consistent representation as substitution of original data, calculating an affinity matrix according to a learned tensor subspace matrix, and performing spectral clustering on the affinity matrix to obtain a final clustering result. Compared with other methods, the method is higher in accuracy and more stable in performance.

Description

Technical Field

[0001] The present invention belongs to the field of computer vision and biometric feature recognition in artificial intelligence, and in particular relates to a multi-dimensional feature fusion clustering method for perspective-missing face recognition. Background Art

[0002] With the rapid development of computer vision and pattern recognition in artificial intelligence, people can identify complete person data and information from facial images. With the application of face recognition technology in different fields, including security monitoring, identity authentication, and social networks, it is particularly important to accurately identify people based on their facial images. Traditional face recognition systems usually rely on frontal or near-frontal images. However, in practical applications, faces may appear at different angles and perspectives, which puts higher requirements on the recognition system. Multi-view face recognition technology overcomes the limitations of features under a single perspective by fusing feature information from different perspectives. Existing multi-view clustering methods based on feature fusion can only handle complete data clustering problems and cannot be applied to images with missing perspective information. This also brings huge challenges to the problem of face recognition with missing perspectives. Effective and novel technical methods are needed to solve the application of incomplete multi-view face images. Summary of the invention

[0003] In order to solve the above problems, the present invention provides a multi-dimensional feature fusion clustering method for perspective-missing face recognition, the method comprising the steps of:

[0004] A kernel matrix is ​​constructed for the data in the original incomplete multi-view face image dataset, and multi-core completion is performed on it to obtain a complete kernel matrix at each viewpoint.

[0005] The completed data core matrix is ​​decomposed into feature matrices of different dimensions, and automatically weighted according to the contribution of each dimension. The feature matrices of different dimensions are fused into a consistent representation matrix with a fixed dimension through the rotation matrix.

[0006] The learned consistent representation matrix is ​​used as a substitute for the original data matrix for subspace learning to obtain a view-specific subspace representation matrix.

[0007] The subspace representation matrix at each perspective is stacked into a third-order tensor, and a tensor low-rank constraint is imposed on it to obtain a subspace representation matrix with high-order information.

[0008] The affinity matrix is ​​calculated using each view subspace representation, and spectral clustering is performed on it to obtain the clustering results.

[0009] According to the obtained clustering results, the clustering accuracy of the face image data set is calculated.

[0010] Furthermore, the kernel matrix in the incomplete multi-view face image dataset is defined as:

[0011]

[0012] Among them, i,j∈{1,2,…,n} represents the instance sample index, Yes X (v) In order to partially complete the incomplete multi-view data We use the average of the corresponding parts in other view kernels. The compensation K of the vth view kernel (v) Defined as

[0013]

[0014] Where m represents the number of missing rows and columns in the kernel, and V represents the total number of viewpoints.

[0015] Furthermore, the complete kernel is subjected to feature decomposition to decompose it into feature matrices of different dimensions, which are defined as follows:

[0016]

[0017] in, is the feature matrix of the data in the pth dimension, is the kernel matrix obtained from the original data, Tr(·) represents the trace of the matrix, I represents the identity matrix, and d p is the dimension of the pth dimension, and m is the total number of each perspective dimension. The feature matrices of different dimensions are fused into a consistent representation matrix of fixed dimension through the rotation matrix, and the above learning process is combined to obtain a consistent representation learning model:

[0018]

[0019] in, is the feature matrix of the pth dimension under the vth perspective, is the rotation matrix of the pth dimension, U (v) is the consistent representation matrix of the vth perspective, c is the consistent dimension after feature fusion, β p for Weights in the feature fusion stage.

[0020] Furthermore, the consistent representation matrix U (v) The subspace representation matrix learned as a replacement for the original data matrix is ​​defined as follows:

[0021]

[0022] U (v) U (v)Τ=I

[0023] Among them, Z (v) is the subspace representation matrix of the vth view, I represents the identity matrix, λ is the regularization parameter, ||·|| F is the Frobinus norm.

[0024] Furthermore, the tensor low-rank constraint is defined as follows:

[0025]

[0026] Among them, 0<p≤1, yes The Fast Fourier Transform (FFT) along the third dimension, i.e. Therefore, the inverse Fourier transform is yes The i-th frontal slice of j yes The jth largest singular value of ω. j Represents the j-th element of the weight vector ω.

[0027] Furthermore, based on the obtained clustering results, a clustering accuracy rate on the incomplete multi-view face image dataset is calculated.

[0028] The present invention provides a multi-dimensional feature fusion clustering method for perspective-missing face recognition, which has the following advantages:

[0029] (1) The method adopts a multi-view clustering framework, which makes full use of the view consistency and view difference information of multi-view face image data and can effectively mine the potential cluster structure of the data.

[0030] (2) The method adopts multi-dimensional feature fusion technology, and uses a consistent matrix fused from feature matrices in different dimensions as a substitute for the original data to perform subspace self-representation learning, so that the model can obtain more comprehensive knowledge and more discriminative information, thereby achieving better clustering performance.

[0031] (3) The method adopts the weighted tensor low-rank constraint technology, which integrates high-order information from different perspectives into the learned representation while utilizing feature fusion, achieving optimal performance and robustness under different missing rates. BRIEF DESCRIPTION OF THE DRAWINGS

[0032] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0033] Figure 1 It is a flow chart of a multi-dimensional feature fusion clustering method for perspective-missing face recognition provided by the present invention;

[0034] Figure 2 This is a schematic diagram of an incomplete multi-view face image dataset ORL-40; DETAILED DESCRIPTION

[0035] In order to make the purpose, technical scheme and advantages of the present invention clearer, the present invention is further described in detail below in conjunction with specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are only exemplary and are not intended to limit the scope of the present invention. In addition, in the following description, the description of well-known structures and technologies is omitted to avoid unnecessary confusion of the concept of the present invention.

[0036] Exemplary Methods

[0037] like Figure 1 The present invention provides a multi-dimensional feature fusion clustering method for perspective-missing face recognition, and the method steps are as follows:

[0038] Step S110: First, the original data matrix of the incomplete multi-view face image data is input, and a kernel matrix is ​​constructed for each view. The kernel matrix is ​​defined as follows:

[0039]

[0040] Among them, i,j∈{1,2,…,n} represents the instance sample index, Yes X (v) The i-th column data of . The missing part of each perspective kernel is filled with the average value of the corresponding part in other perspective kernels. The compensation K of the v-th perspective kernel (v) Defined as

[0041]

[0042] Where m represents the number of missing rows and columns in the kernel, and V represents the total number of viewpoints.

[0043] Step S120: Decompose the complete kernel into feature matrices of different dimensions by performing feature decomposition on the kernel, which are defined as follows:

[0044]

[0045] in, is the feature matrix of the data in the pth dimension, is the kernel matrix obtained from the original data, Tr(·) represents the trace of the matrix, I represents the identity matrix, and d p is the dimension of the pth dimension, and m is the total number of dimensions of each perspective. The feature matrices of different dimensions are fused into a consistent representation matrix of fixed dimension through the rotation matrix, and the projection-fusion process under each perspective mentioned above is combined to obtain a consistent representation learning model:

[0046]

[0047] in, is the feature matrix of the pth dimension under the vth perspective, is the rotation matrix of the pth dimension, U (v) is the consistent representation matrix of the vth perspective, c is the consistent dimension after feature fusion, β p for Weights in the feature fusion stage.

[0048] Step S130: Use the learned consistent representation matrix as a substitute for the original data matrix to perform subspace learning, and obtain a subspace representation matrix specific to a viewpoint, which is expressed as follows:

[0049]

[0050] U (v) U (v)Τ =I

[0051] Among them, Z (v) is the subspace representation matrix of the vth view, I represents the identity matrix, λ is the regularization parameter, ||·|| F is the Frobinus norm.

[0052] Step S140: Eliminate the regularization term, stack the representation matrices of each view subspace into a third-order tensor, and impose low-rank constraints on it to obtain the tensor subspace matrix Z, which is expressed as follows:

[0053]

[0054] in, Indicated by Z (v) The stacked third-order tensor, represents the Schatten-p norm of the weighted tensor, and γ represents the reconstruction loss weight parameter.

[0055] Step S150: Utilize Calculate the affinity matrix F( express The vth slice along the viewing direction) is sent to the spectral clustering algorithm to obtain the clustering result.

[0056] Step S160: Calculate the clustering accuracy on the incomplete multi-view face image dataset according to the obtained clustering results.

[0057] Through this implementation, a kernel matrix is ​​first constructed for the incomplete original data of each perspective, and the local completion method is used to complete it to obtain a complete data kernel matrix. Then, the kernel matrix under each perspective is mapped to different dimensional spaces through feature decomposition to obtain feature matrices of different dimensions, and then they are fused into a consistent representation matrix of fixed dimension through a rotation matrix. The consistent representation is used as a substitute for the original data to perform tensor subspace learning, and the affinity matrix is ​​calculated based on the learned tensor subspace matrix, and spectral clustering is performed on it to obtain the final clustering result. After obtaining the clustering result, the clustering accuracy on the data set is calculated.

[0058] To further illustrate, assuming that an incomplete multi-view face image dataset is clustered according to this embodiment, a clustering result with higher accuracy than most methods will be obtained.

[0059] Detailed description of the invention Results

[0060] This implementation uses a publicly available multi-view face image dataset and simulates an incomplete multi-view face image dataset according to different missing view ratios. The details of the dataset are described as follows:

[0061] The ORL-40 dataset is a dataset commonly used for face recognition. It consists of 400 facial images of 40 people of different genders, each of which has 10 facial images taken from different angles. All images are in grayscale PGM format with a size of 92x112 pixels. The dataset has the following three perspectives:

[0062] The first perspective is the grayscale intensity (Gray) feature, which directly measures the brightness value of each pixel and is often used in basic image processing tasks such as image enhancement and simple object detection, but is not suitable for capturing complex texture or shape information of the image.

[0063] The second perspective is the local binary pattern (LBP) feature, which generates a binary number by comparing the grayscale values ​​of the central pixel with the neighboring pixels. It is used to capture the local texture features of the image and is suitable for tasks such as face recognition and texture classification. It can effectively distinguish different texture patterns.

[0064] The third perspective is the Gabor feature, which extracts the texture and edge information of the image through Gabor filters of different frequencies and directions. It is suitable for complex image analysis tasks such as texture classification, edge detection and feature extraction, and can process multi-scale and multi-directional texture details.

[0065] In order to verify the superiority of this implementation (Ours), this implementation is compared with several existing incomplete multi-view face image data clustering methods, including IMV_AGL, ANIMC, TMBSD, DAIMC, FIMVC_VIA and other methods. Five incomplete view ratios are simulated on the public multi-view face image dataset ORL-40, and the clustering accuracy (ACC) of the above comparison methods is tested. The specific experimental results are shown in Table 1.

[0066] Table 1 Clustering accuracy of ORL-40 dataset ACC

[0067]

[0068] By comparing the data in the above table, we can clearly see that Ours achieves the best performance and significantly improves the clustering accuracy of incomplete multi-view face image data. The quantitative results fully demonstrate the superiority of Ours, because Ours can better capture the perspective consistency information and perspective difference information in incomplete multi-view face image data. While performing multi-dimensional feature fusion, Ours integrates more discriminative and comprehensive information into the learned consistent representation, achieving the best performance and stability under different missing rates. A large number of experiments show that this method is superior to existing methods.

[0069] This implementation proposes a multi-dimensional feature fusion clustering method for perspective-missing face recognition, which is used to perform clustering analysis on incomplete multi-perspective face image data commonly seen in life. It integrates multi-dimensional feature fusion representation, perspective-specific subspace representation, and weighted tensor low-rank representation into a unified framework, optimizes each other, and finally obtains clustering from the learned unified representation. The experimental results on five incomplete perspective ratios of the public dataset ORL-40 show that this implementation has a higher clustering accuracy and better superiority than other methods.

[0070] It should be understood that the above specific embodiments of the present invention are only used to illustrate or explain the principles of the present invention, and do not constitute a limitation of the present invention. Therefore, any modifications, equivalent substitutions, improvements, etc. made without departing from the spirit and scope of the present invention should be included in the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all changes and modifications that fall within the scope and boundaries of the appended claims, or the equivalent forms of such scope and boundaries.

Claims

1. A multi-dimensional feature fusion clustering method for view-missing face recognition, characterized in that: The method comprises the steps of: S110. construct a kernel matrix for the data in the original incomplete multi-view face image dataset, perform multi-core completion on it, and obtain a complete kernel matrix under each view; S120. Decomposing the completed data core matrix into feature matrices of different dimensions, and automatically weighting them according to the contribution of each dimension, and fusing the feature matrices of different dimensions into a consistent representation matrix of a fixed dimension through a rotation matrix; S130. Using the learned consistent representation matrix as a substitute for the original data matrix to perform subspace learning, and obtain a subspace representation matrix specific to a viewpoint; S140. stacking the subspace representation matrix at each perspective into a third-order tensor, and applying a tensor low-rank constraint thereon, so as to obtain a subspace representation matrix with high-order information; S150. Calculate the affinity matrix using each view subspace representation, and perform spectral clustering on it to obtain a clustering result; S160. Calculate the clustering accuracy of the face image dataset based on the obtained clustering results; In S110, the kernel matrix constructed from the original data is defined as: Among them, i,j∈{1,2,…,n} represents the instance sample index, Yes X (v) The i-th column data; in order to partially complete the incomplete multi-view data We use the average of the corresponding parts in other view kernels, and the compensation K of the v-th view kernel (v) Defined as Where m represents the missing rows and columns in the kernel, and V represents the total number of views; In S120, the complete kernel is subjected to feature decomposition to decompose it into feature matrices of different dimensions, which are defined as follows: in, is the characteristic matrix of the data in the pth dimension, is the kernel matrix obtained from the original data, Tr(·) represents the trace of the matrix, I represents the identity matrix, and d p is the dimension of the pth dimension, and m is the total number of dimensions for each view; The feature matrices of different dimensions are fused into a consistent representation matrix of fixed dimension through the rotation matrix, and the above learning process is combined to obtain a consistent representation learning model: in, is the feature matrix of the pth dimension under the vth view, is the rotation matrix of the pth dimension, U (v) is the consistent representation matrix of the vth view, c is the consistent dimension after feature fusion, β p for Weights in the feature fusion stage; In S130, the consistent representation matrix U is used (v) The subspace representation matrix learned as a replacement for the original data matrix is ​​defined as follows: s.t.U (v) U (v)Τ =I Among them, Z (v) is the subspace representation matrix of the vth view, I represents the identity matrix, λ is the regularization parameter, ||·|| F is the Frobinus norm; In S140, the tensor low-rank constraint is defined as follows: Among them, 0<p≤1, yes The Fast Fourier Transform (FFT) along the third dimension, i.e. Therefore, the inverse Fourier transform is yes The i-th frontal slice of j yes The jth largest singular value of ω. j Represents the j-th element of the weight vector ω.

2. According to claim 1, a multi-dimensional feature fusion clustering method for view-missing face recognition is characterized in that: The representation matrices of each view subspace are stacked into a third-order tensor and a low-rank constraint is imposed on it. The expression formula is: in, Indicated by Z (v) The stacked third-order tensor, represents the Schatten-p norm of the weighted tensor, and γ represents the reconstruction loss weight parameter. After obtaining the tensor subspace matrix Afterwards, we used Calculate the affinity matrix F( express The vth slice along the viewing direction) is sent to the spectral clustering algorithm to obtain the clustering result.

Images