A multi-view clustering image recognition method based on structure graph

Abstract

The present invention provides a multi-view clustering image recognition method based on a structure graph, which belongs to the fields of computer vision, image clustering, multi-view learning and machine learning. The method includes the following modules: a feature weight learning module, a dynamic feature matrix learning module, a multi-graph learning module, a view weight learning module and a consistency structure graph learning module. Compared with the prior art, the present invention designs a multi-view clustering image recognition method based on a structure graph for an image data set, which avoids fixed distance relationships by learning dynamic features and thereby captures the intrinsic relationship between image samples. At the same time, the similarity matrix obtained by learning can provide feedback guidance for the learning of dynamic features. In the process of learning the consistency graph matrix, not only the importance between different views can be considered, but also the consensus graph can be subjected to Laplace rank constraints, which can improve the quality of the graph matrix, thereby further improving the accuracy of image clustering.

Description

Technical Field

[0001] The present invention belongs to the fields of computer vision, image clustering, multi-view learning and machine learning, and more specifically, relates to a multi-view clustering image recognition method based on a structure graph. Background Art

[0002] With the continuous development of information technology, the way to obtain image data has become more and more complicated, and single-view learning and multi-view learning are two important dominant modes. In the past, traditional machine learning methods such as support vector machines and clustering were all learned on image data of a single view, but in this case, any other information from different views was not considered. Therefore, in order to better match the way humans think and more comprehensively describe the situation in the real world, multi-view data has emerged. For example, a picture can be described from different angles, a video consists of images and sounds, or a face can be shot from different perspectives, etc. The learning paradigm corresponding to multi-view data is multi-view learning.

[0003] Multi-view clustering is an important cornerstone in multi-view learning. Its purpose is to use the complementary information and consistency information in multi-view data to obtain reliable data partitioning results. Therefore, many graph-based multi-view spectral clustering methods have emerged to process image datasets. These methods generally first construct a similarity matrix of different views or a unified graph matrix in the original feature space, and then obtain the final clustering result through spectral clustering. However, these methods all have some disadvantages. The similarity matrix constructed based on a fixed formula is calculated based on the fixed Euclidean distance between samples in the original feature space. Therefore, the fixed relationship representation cannot capture the potential internal relationship between data, and the relationship expression based on Euclidean distance is very susceptible to anomalies such as outliers and noise. In addition, the independence of different learning stages may lead to suboptimal graph learning results, and also cause information loss, thereby obtaining unsatisfactory image clustering results.

[0004] Therefore, there is an urgent need for a multi-view clustering image recognition method based on structure graph, which avoids the fixed distance relationship between image samples by combining dynamic feature learning and multi-graph learning and captures the intrinsic relationship between data points, thereby obtaining a high-quality consistent structure graph matrix and significantly improving the clustering performance of image datasets. Summary of the invention

[0005] In view of the shortcomings of the prior art, the present invention provides a multi-view clustering image recognition method based on structure graph, which avoids the fixed distance relationship between image samples and captures the intrinsic relationship between data points by combining dynamic feature learning and multi-graph learning, thereby obtaining a high-quality consistent structure graph matrix.

[0006] A multi-view clustering image recognition method based on a structure graph comprises: a feature weight learning module, a dynamic feature matrix learning module, a multi-graph learning module, a consistent structure graph learning module and a view weight learning module.

[0007] The feature weight learning module adaptively weights the features of different dimensions of the feature matrix of multiple views of the data, and weakens the impact of noise and outliers in the feature space on the samples.

[0008] The dynamic feature matrix learning module learns a new dynamic feature representation based on a feature-weighted feature matrix, thereby fully capturing the potential relationship between image samples.

[0009] The multi-graph learning module learns the similarity matrix of different views based on dynamic feature representation, thereby capturing the topological structure of data from different views.

[0010] The view weight learning module adaptively assigns corresponding weights to different similarity matrices by considering the importance of different views.

[0011] The consistent structure graph learning module learns a unified graph matrix with Laplace rank constraints based on similarity matrices with different weights, and the matrix can fully reflect the clustering allocation results of graph sample data.

[0012] As an embodiment of the present invention, a feature weight learning module adaptively weights features of different dimensions in a feature matrix of an image data set, thereby distinguishing the importance of features of different dimensions, making important dimensional features more important, and thereby reducing the impact of noise and outliers on image sample features.

[0013] As an embodiment of the present invention, the dynamic feature matrix learning module can be used to learn new feature representations of original image sample data, thereby fully capturing the potential relationship between image samples. The obtained new feature representation can be more flexible and robust than the original feature representation.

[0014] As an embodiment of the present invention, the multi-graph learning module is obtained based on the dynamic feature matrix learning module, and can further reversely induce the learning of dynamic features through the learning of the similarity matrix, thereby promoting each other to learn a better similarity matrix.

[0015] Generally, the dynamic feature learning module and the multi-image learning module are used in combination.

[0016] As an embodiment of the present invention, the view weight learning module can assign different weights according to the importance of different similarity matrices, so that in the learning process of the unified graph, important similarity matrices occupy more importance.

[0017] As an embodiment of the present invention, a consistency structure graph learning module learns a consistency graph matrix by weighting the similarity matrices of different views. Due to the existence of Laplace constraints, the consistency graph matrix has a diagonal block structure and can directly avoid the k-means algorithm based on the learned consistency structure graph to obtain a stable image data set clustering result.

[0018] Through the above modules, the multi-view consensus graph clustering effect of the image data set can be achieved.

[0019] The beneficial effects of the present invention are:

[0020] A multi-view clustering image recognition method based on a structural graph, which learns a dynamic feature representation based on the original feature space and dynamically adjusts it through a constructed similarity matrix. Based on the dynamic feature space representation, the patent can fully explore the potential internal relationship between data and get rid of the fixed distance relationship between the original features. In addition, considering the importance of features of different dimensions, the present invention performs adaptive feature weighting on the original features to make them more discriminative and alleviate the negative effects of noise and outliers. However, in order to solve the situation that multiple learning stages in traditional spectral clustering are independent of each other and maintain the local manifold structure of multiple views, the present invention considers using multiple similarity matrices to perform adaptive weighted learning of a consistent graph matrix with Laplace rank constraints. Due to the existence of the Laplace rank constraint, the connected components in the learned unified graph can accurately match the number of clusters and maintain the ideal diagonal block structure. It should be noted that the unified graph matrix can not only reversely improve the similarity matrix of each view, but also directly obtain the final clustering result. Therefore, the present invention can allow multiple learning stages to blend with each other and achieve the effect of enhancing each other among dynamic feature learning, multi-graph learning, and consistent structural graph learning, thereby greatly improving the clustering performance of image data sets.

[0021] Furthermore, the features of different dimensions in the feature matrix of the image dataset are adaptively weighted to distinguish the importance of features of different dimensions, making important dimensional features more important, thereby reducing the impact of noise and outliers on the characteristics of image samples.

[0022] Furthermore, the multi-graph learning module is obtained based on the dynamic feature matrix learning module, and can further reversely induce the learning of dynamic features through the learning of the similarity matrix, thereby promoting each other to learn a better similarity matrix.

[0023] Furthermore, the view weight learning module can assign different weights according to the importance of different similarity matrices, so that in the learning process of the unified graph, important similarity matrices occupy more importance. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0025] Figure 1 It is an overall module flow chart of a multi-view clustering image recognition method based on a structure graph in an embodiment of the present invention;

[0026] Figure 2 A feature weight learning module diagram in a multi-view clustering image recognition method based on a structure graph in an embodiment of the present invention;

[0027] Figure 3 A diagram of a multi-dynamic feature learning and multi-graph learning module in a multi-view clustering image recognition method based on a structure graph in an embodiment of the present invention;

[0028] Figure 4 This is a diagram of the view weight learning and consistent structure graph learning modules in a multi-view clustering image recognition method based on a structure graph in an embodiment of the present invention.

[0029] Figure 5 The present invention is a flowchart of an overall method implementation in a multi-view clustering image recognition method based on a structure graph in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] The preferred embodiments of the present invention are described below in conjunction with the accompanying drawings. It should be understood that the preferred embodiments described herein are only used to illustrate and explain the present invention, and are not used to limit the present invention.

[0031] See also Figure 1 , the embodiment of the present invention provides a multi-view clustering image recognition method based on a structure graph, including: S101, a multi-view feature matrix of an image data set; S102, a feature weight learning module; S103, a dynamic feature matrix learning module; S104, a multi-graph learning module; S105, a view weight learning module; S106, a consistency structure graph learning module; S107, an image clustering result;

[0032] The working principle of the above technical solution is to learn a dynamic feature representation based on the original feature space of the image data set, and dynamically adjust it through the constructed similarity matrix. Based on the dynamic feature space representation, this patent can fully explore the potential internal relationship between the data and get rid of the fixed distance relationship between the original features. In addition, considering the importance of features of different dimensions, the present invention performs adaptive feature weighting on the original features to make them more discriminative and alleviate the negative effects of noise and outliers. However, in order to solve the situation in which multiple learning stages in traditional spectral clustering are independent of each other and maintain the local manifold structure of multiple views, the present invention considers using multiple similarity matrices to perform adaptive weighted learning of a consistency graph matrix with Laplace rank constraints. Due to the existence of the Laplace rank constraint, the connected components in the learned unified graph can accurately match the number of clusters and maintain the ideal diagonal block structure. It should be noted that the unified graph matrix can not only improve the similarity matrix of each view in reverse, but also directly obtain the final clustering result.

[0033] The beneficial effects of the above technical solution are: by combining dynamic feature learning and multi-graph learning to avoid the fixed distance relationship between image samples and thereby capture the intrinsic relationship between data points, a high-quality consistent structure graph matrix is ​​obtained, thereby greatly improving the clustering performance of the image data set.

[0034] See also Figure 2 In one embodiment, the feature weight learning module adaptively weights the features of different dimensions in the feature matrix of the image data set, thereby distinguishing the importance of features of different dimensions, making important dimensional features more important, and thereby reducing the impact of noise and outliers on image sample features.

[0035] The working principle of the above technical solution: Considering that in the image data set, the features in the original space are not discriminative, that is, the features of different dimensions are equally important. In order to make the original features in the feature space more discriminative, many effective methods have emerged, such as feature selection and dimensionality reduction. However, these methods will either lead to the loss of original information or require the selection of dimensional parameters. Obviously, if this patent has prior knowledge of the importance of different features, this patent can assign different prior weights to each feature and solve the above problems. However, this prior knowledge is difficult to obtain in practice. To this end, the present invention considers the original feature space samples. Perform adaptive weight learning:

[0036]

[0037]

[0038] Among them, the vth feature matrix of the image dataset And the corresponding v-th dynamic feature matrix W v is an adaptive feature weight matrix and satisfies the constraint condition W v =diag(w v ),

[0039] The beneficial effect of the above technical solution is that the sample features in the feature space of the image data set can become more discriminative, thereby reducing the influence of noise and outliers, and further making the model robust.

[0040] See also Figure 3 In one embodiment, the dynamic feature matrix learning module can be used to learn a new feature representation of the original image sample data, thereby fully capturing the potential relationship between the image samples. The obtained new feature representation can be more flexible and robust than the original feature representation.

[0041] See also Figure 3 In one embodiment, the multi-graph learning module is obtained based on the dynamic feature matrix learning module, and can further reversely induce the learning of dynamic features through the learning of the similarity matrix, thereby promoting each other to learn a better similarity matrix.

[0042] The working principle of the above technical solution is: In order to better capture the potential structure of the data in order to explore the local connectivity of the data, the idea of ​​KNN can be used to define the K neighbors closest to each original data sample point, and to ensure the local connectivity of the data. For the convenience of defining neighboring points, the i-th data sample point x i For all data points {x1,x2,…,x n The probability of the adjacent ij Generally speaking, the distance between two sample points The smaller the probability s between them is ij Therefore, for the sample point x i The adjacent probability solution can be solved by the following problem:

[0043]

[0044]

[0045] where s ij is a column vector s i ∈R 1×n The jth element of . The probability of two points being adjacent can also be regarded as the similarity between the two points. From model (2), we can find that the sample point x i and xj The smaller the distance, the greater the similarity between them, which is exactly the same as the expected neighbor probability. However, it should be noted that model (2) may have a trivial solution, that is, i The nearest point is assigned a probability of 1, which causes other sample points to no longer be neighbors of this point. Therefore, in order to avoid this situation, the similarity s ij The learning of regularization constraints:

[0046]

[0047] The second term is the regularization term, and λ1 is the regularization parameter. i (i=1,2,…,n), can all be solved by formula (3), so the similarity problem of all sample points can be described as follows:

[0048]

[0049]

[0050] By solving problem (4), we can easily obtain the similarity matrix S. Each vector s in the matrix S i There are only K non-zero elements, so that subsequent learning can more easily capture the local connectivity of the data. In order to process multi-view data, equation (4) is transformed into the following form:

[0051]

[0052]

[0053] Formula (5) is a classic multi-view graph learning formula, where Represents the F-norm constraint on the matrix S. However, the differences between different views are not fully considered, and the flexible local manifold structure of each view cannot be maintained. Therefore, the present invention learns different similarity graph matrices based on the feature space of different views, considering the differences between different views while maintaining the local manifold structure of the data:

[0054]

[0055]

[0056] The distance metric is Euclidean distance, which is very sensitive to noise and outliers, resulting in a low-quality graph matrix. In addition, the fixed distance relationship of the original feature space samples will cause the learned similarity matrix to be fixed. The learning of the original feature space representation matrix is ​​guided by the learned K-nearest similarity graph matrix:

[0057]

[0058]

[0059] Considering the similarity matrix S v Each column vector in There are only K non-zero elements, that is, only K away from the sample point x i There is similarity only between the nearest K sample points, and the closer the samples are in the feature space, the greater their similarity will be. ij When the larger the and The distance between will be smaller, thus being able to guide the feature representation matrix The learned matrix R v With the original feature space X v It is more flexible and has a higher degree of distinction between sample points in different clusters, and it also isolates the problem of instability based on the Euclidean distance in the original feature space. Therefore, the matrix R v It can be seen as a function of the original feature space X v A more reliable new dynamic space that can also provide feedback and optimization with the learning of the similarity matrix.

[0060] Considering the feature weight matrix of the feature weight learning module, this patent combines model (1) and model (7) to obtain the following new model:

[0061]

[0062]

[0063] Model (7) simultaneously considers the feature weight learning module, dynamic feature learning module and multi-graph learning module, which reduces the noise and outlier influence of the original features, thereby ensuring the quality of the dynamic features and constructing high-quality similarity. The obtained similarity matrix can also reversely guide the dynamic feature learning.

[0064] The beneficial effects of the above technical solution are: a more dynamic new feature representation is obtained through learning the original feature space of the image data set, and due to the addition of the multi-graph learning model, the new dynamic feature representation can fully explore the potential relationship between image samples and is more flexible. And due to the addition of feature weights, the robustness of the model is further enhanced.

[0065] See also Figure 4,The view weight learning module can assign different weights by the importance of different ,similarity matrices, so that in the learning process of the unified graph, ,the important similarity matrices occupy more importance.

[0066] See also Figure 4 The consistency structure graph learning module learns the consistency graph matrix by weighting the similarity matrices of different views. Due to the existence of Laplace constraints, the consistency graph matrix has a diagonal block structure and can directly avoid the k-means algorithm based on the learned consistency structure graph to obtain stable image dataset clustering results.

[0067] The working principle of the above technical solution is: if the similarity matrix S obtained by the multi-graph learning module is directly v Using k-means or spectral clustering algorithms will lead to unstable clustering performance and will directly ignore the direct relationship between multi-view similarity graphs. Therefore, it is necessary to consider first using compatible similarity matrices S v To learn a common graph matrix G:

[0068]

[0069] in Represents the matrix S v The F-norm constraint of the difference between and matrix G. The consistency graph matrix G can be directly compatible with different view similarity matrices S v However, considering the importance of different views, a view weight module is introduced in model (9) to measure the importance of different view matrices:

[0070]

[0071] The weight model of the vth view is:

[0072]

[0073] It can be seen from formula (11) that when the consistency graph matrix G and the similarity matrix S of the vth view v The more compatible, the v The larger the weight distribution, the greater the weight distribution. It should be noted that when obtaining the consistency graph matrix G, each similarity matrix S v They are all relatively fixed and do not achieve the effect of joint optimization with the unified graph matrix. To this end, this patent proposes the following model:

[0074]

[0075] In model (12), the learned consistency graph matrix U can also participate in the similarity matrix Sv The joint optimization of the two graphs can enhance each other. In addition, different similarity matrices can also get complementary feedback of consistency information from the unified graph matrix. Generally speaking, a good graph matrix has a good clustering structure, that is, a block diagonal block. To this end, consider the Laplace rank constraint on the consistency graph matrix, the purpose of which is to make the connected components in the unified graph matrix completely equal to the number of clusters, that is, to have an accurate clustering structure, so that each connected component can correspond to a cluster. Therefore, combined with Figure 2-4 The model between them and the derivation of the above modules can get the following model:

[0076]

[0077]

[0078]

[0079] In model (13), is the Laplace matrix. And D G The diagonal elements are ∑ j (g ij +g ji ) / 2 diagonal matrix. Due to the rank constraint rank(L G )=nc, the number of eigenvalues ​​of the Laplacian matrix that are 0 will be the same as the connected components in the consistency graph matrix. Therefore, the neighbor allocation in the unified graph will be an ideal allocation, that is, this patent divides the data points into c clusters based on the graph G. Considering that the patent learns a consistency graph matrix with a clear clustering structure, the final one can be obtained directly without the need for discretization operations such as k-means, thus having stable clustering performance.

[0080] The beneficial effects of the above technical solution are: the importance of different views in the image data set is learned through the view weight model, so that the model has diversity, and then a consistent structure graph matrix with Laplace rank constraints is learned based on the weighted similarity matrix. This matrix can repeatedly reflect the cluster division results between image samples, thereby avoiding the instability of the k-means method.

[0081] See also Figure 5 , the overall implementation process of a multi-view clustering image recognition method based on structure graph is as follows: obtaining image dataset, obtaining multi-view feature matrix of dataset, weighting feature weights of different dimensions of feature matrix, learning dynamic feature matrix of different views, multi-graph learning, different view weight learning, consistent structure graph learning, Laplace rank constraint and image dataset clustering results.

[0082] The working principle of the above technical solution is: to learn a dynamic feature representation based on the original feature space of the image data set, and to dynamically adjust it through the constructed similarity matrix. Based on the dynamic feature space representation, this patent can fully explore the potential internal relationship between the data and get rid of the fixed distance relationship between the original features. In addition, considering the importance of features of different dimensions, the present invention performs adaptive feature weighting on the original features to make them more discriminative and alleviate the negative effects of noise and outliers. However, in order to solve the situation in which multiple learning stages in traditional spectral clustering are independent of each other and maintain the local manifold structure of multiple views, the present invention considers using multiple similarity matrices to perform adaptive weighted learning of a consistent graph matrix with Laplace rank constraints. Due to the existence of the Laplace rank constraint, the connected components in the learned unified graph can accurately match the number of clusters and maintain the ideal diagonal block structure. It should be noted that the unified graph matrix can not only improve the similarity matrix of each view in reverse, but also directly obtain the final clustering result.

[0083] The beneficial effects of the above technical solution are: by jointly using dynamic feature learning and multi-graph learning to avoid fixed distance relationships between image samples and thereby capture the intrinsic relationship between data points, a high-quality consistent structure graph matrix is ​​obtained, thereby significantly improving the clustering performance of image data sets.

[0084] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A multi-view clustering image recognition method based on structure graph, characterized in that: It includes a feature weight learning module, a dynamic feature matrix learning module, a multi-graph learning module, a view weight learning module and a consistent structure graph learning module, which are connected in sequence. These modules are a gradual process: first, the image sample features are learned through the feature weight module, and then a new dynamic feature representation is learned based on the feature weighted samples in the dynamic feature matrix learning module. The multi-graph learning module also directly learns multiple similarity matrices based on the dynamic feature matrix learning module, and can reversely guide the learning process of the dynamic feature matrix; The view weight learning module assigns importance weights to multiple similarity matrices of the multi-graph learning module, thereby ensuring the diversity of similarity matrices in the consistency structure graph learning module process; The feature weight learning module considers the original feature space samples For adaptive weight learning, the module is constructed as follows: For different data points x i (i=1,2,…,n), the vth feature matrix of the image dataset d v , n represent the feature dimension and the number of samples respectively; and the corresponding vth dynamic feature matrix W v is an adaptive feature weight matrix and satisfies the constraints The multi-image learning module learns the similarity matrix of different views based on dynamic feature representation; The dynamic feature learning module and the multi-graph learning module are combined, and at the same time based on the feature weight matrix W in the feature weight learning module v : Among them, the similarity matrix S v , λ1 is the regularization parameter, for different data points x i (i=1,2,…,b),s ij is a column vector s i ∈R 1×n The jth element of the image dataset is the vth feature matrix And the corresponding v-th dynamic feature matrix W v is an adaptive feature weight matrix and satisfies the constraints The first part of the model in model (2) It is the feature weight learning module; the first part and the second part are combined and the feature weight matrix W is fixed v Model It is the dynamic feature learning module; the second and third parts are combined to fix the dynamic feature matrix R v Model It is a multi-graph learning module, where is the similarity matrix S v F-norm constraint; The consistency structure graph learning module learns a unified graph matrix with Laplace rank constraints based on similarity matrices with different weights; The view weight learning module and the consistency structure graph learning module are combined: In model (3), the weight model of the vth view learned by the view weight learning module is The consistency structure graph learning module is Where G is the consistency graph matrix to be learned, and the Laplace rank constraint rank (L G )=nc, and then the clustering result of the image data set can be directly obtained through the consistency structure graph, and c is the number of clusters into which the data points are divided.

2. According to claim 1, a multi-view clustering image recognition method based on a structure graph is characterized in that: Adaptively weight features of different dimensions in the feature matrix of the image dataset.

Images