Robust multi-view image clustering method, system and equipment based on smooth anchor diagram learning

Abstract

The invention discloses a robust multi-view image clustering method, system and device based on smooth anchor point diagram learning for large-scale label-free image data under complex noise erosion, and belongs to the technical field of information. The method comprises the following steps: 1, acquiring the characteristics of a large-scale multi-view image eroded by noise; 2, performing image filtering smooth denoising on the obtained image features; 3, generating representative anchor point sets, and adaptively learning an anchor point diagram between each view anchor point set and the smoothed image features; 4, fusing the anchor point diagrams of different views by applying a local manifold learning method to obtain a weighted anchor point diagram; 5, performing smooth denoising on the weighted anchor point diagram under a Floribenius norm to obtain a consensus anchor point diagram; and 6, obtaining consensus spectrum embedding and corresponding clustering results. When the method is used for processing a clustering task of real large-scale multi-view image data containing complex noise, the clustering efficiency and the robustness of coping with the complex noise can be remarkably improved.

Description

Technical Field

[0001] The present invention relates to the technical field of image clustering analysis in machine learning, and in particular to a robust multi-view image clustering method, system and device based on smooth anchor graph learning. Background Art

[0002] In recent years, with the rapid development of information technology and Internet technology, multi-view image data has grown explosively. In order to process these unlabeled multi-view image data, multi-view image clustering analysis technology, which can directly cluster unlabeled multi-view image data into clusters, has developed rapidly in recent years.

[0003] However, existing multi-view image clustering methods still face the following two challenges: 1) With the explosive growth of data volume, traditional multi-view image clustering methods are difficult to meet the requirements of efficient image clustering due to their high computational complexity; 2) Due to factors such as sensor errors, data transmission interference, environmental conditions or shooting equipment failure, data from the real world is often corroded by complex noise. Traditional multi-view image clustering methods are difficult to maintain high clustering accuracy when faced with complex noise erosion.

[0004] In summary, the related technologies are difficult to process multi-view image data with increasing size and eroded by complex noise. Therefore, the problems existing in the existing related technologies need to be solved urgently. Summary of the invention

[0005] In order to solve the problems existing in the above-mentioned prior art, the purpose of the present invention is to provide a robust multi-view image clustering method based on smooth anchor graph learning, which reduces the computational complexity when facing large-scale image data by learning weighted anchor graphs of all views, and performs graph filtering on image features eroded by noise using a Laplacian matrix to alleviate the complex noise erosion in the original image features. At the same time, a consensus anchor graph is obtained by smoothing and denoising the weighted anchor graph under the Frobenius norm metric to cope with the erosion of the anchor graph by complex noise, which greatly improves the efficiency and robustness when processing large-scale image data eroded by complex noise.

[0006] In order to achieve the above object, the present invention adopts the following technical solution:

[0007] On the one hand, the present invention provides a robust multi-view image clustering method based on smooth anchor graph learning, the method comprising:

[0008] Acquire large-scale multi-view image features eroded by complex noise;

[0009] Performing graph filtering on the obtained image features to smooth and denoise;

[0010] Generate a representative anchor point set and adaptively learn the anchor point map between each view anchor point set and the smoothed image data;

[0011] Apply local popular learning method to fuse anchor maps of different views to obtain weighted anchor maps;

[0012] The weighted anchor graph is smoothed and denoised under the Frobenius norm to obtain a consensus anchor graph;

[0013] The consensus spectrum embedding and the corresponding clustering results are obtained.

[0014] Optionally, the acquiring of large-scale multi-view image features corroded by complex noise includes:

[0015] A variety of salient features are extracted from visible light data, infrared data and SAR imaging data corroded by complex noise to obtain large-scale multi-view image features.

[0016] Optionally, performing graph filtering on the obtained image features to perform smoothing and denoising includes:

[0017] A Laplacian matrix is ​​constructed independently for each view, and the Laplacian matrix is ​​used to perform graph filtering on the original image features eroded by complex noise to alleviate the complex noise erosion of the original image features:

[0018]

[0019] Where X (v) represents the original image features of the vth view, L (v) represents the Laplacian matrix of the v-th view, represents the image features after smoothing by the Laplace matrix, I represents the unit matrix, and μ represents a balance parameter.

[0020] Optionally, a representative anchor point set is generated, and an anchor point map between each view anchor point set and the smoothed image data is adaptively learned, including:

[0021] For the obtained smoothed image features containing V views eroded by noise Among them, n is the number of images required to be clustered, d (v) is the dimension of the image features contained in the vth view; the anchor point set ε = {E (1) ,E (2) ,...,E (V)}, Where m is the number of anchor points; then the anchor point graph of each view is adaptively learned according to the topological relationship between the anchor point set and the smoothed image features. (1) ,B (2) ,...,B (V)},

[0022] Optionally, the step of adaptively learning the anchor point graph of each view according to the topological relationship between the anchor point set and the smoothed image features includes:

[0023]

[0024] in represents the element in the i-th row and j-th column of the anchor map of view v, Φ(·) represents the distance metric function, and k represents that the anchor map only records the features of the i-th original image. The relationship between the k nearest anchor points; Represents the feature of the i-th original image The jth anchor point closest to it.

[0025] Optionally, a local popular learning method is applied to fuse anchor graphs of different views to obtain a weighted anchor graph, including:

[0026] The local popular learning method is used to assign different weights to the anchor maps of different views, and the anchor maps of different views are fused to obtain the weighted anchor map:

[0027]

[0028] where α (v) and B (v) Represent the weight of the v-th anchor graph and the v-th anchor graph respectively. By weighted summing the anchor graphs from different views, we get the weighted anchor graph B A .

[0029] Optionally, the weighted anchor graph is smoothed and denoised under the Frobenius norm to obtain a consensus anchor graph, including:

[0030] Under the Frobenius norm metric, the influence of noise and outliers in the weighted anchor graph is eliminated to obtain a robust consensus anchor graph:

[0031]

[0032] in represents the learned consensus anchor graph, Represents the weighted anchor map obtained by weighted summation of anchor maps from different views.

[0033] Optionally, the consensus spectrum embedding and corresponding clustering results are obtained, including:

[0034] Learn the consensus spectral embedding based on the consensus anchor graph, and learn the final cluster indicator matrix based on the consensus spectral embedding:

[0035]

[0036] where Tr[·] represents the trace of the matrix, F is the consensus spectral embedding representation of the learned consensus anchor graph, I represents the identity matrix, and F and Z represent the transpose of F and Z respectively; finally, the clustering result is obtained by performing K-means clustering on the consensus spectral embedding representation.

[0037] Another aspect of the present invention further provides a system for implementing the robust multi-view image clustering method based on smooth anchor graph learning, comprising:

[0038] Feature extraction module: obtains multi-view image features with complex noise erosion for clustering tasks;

[0039] Laplacian filter module: constructs the Laplacian matrix to obtain image features, and uses the Laplacian matrix to perform graph filtering on the original image features to alleviate the complex noise erosion in the original image features;

[0040] Anchor map generation module: First, an anchor set is generated for each view. Then, based on the topological relationship between the anchor set and the smoothed image features, an anchor map corresponding to each view is adaptively generated.

[0041] Anchor graph weighting module: It uses local popular learning to assign different weights to the anchor graphs of all views, and fuses the anchor graphs of different views to obtain a weighted anchor graph.

[0042] Anchor graph smoothing module: It performs smoothing learning on the weighted anchor graph under the Frobenius norm metric to reduce the impact of noise in the weighted anchor graph and obtain a robust consensus anchor graph.

[0043] Clustering result acquisition module: The consensus spectrum embedding is obtained according to the consensus anchor point graph, and the final clustering result is obtained through the K-means clustering method.

[0044] In another aspect of the present invention, a multi-view image clustering device is provided, which includes a memory and a processor, wherein the memory is used to store instructions and data, and the processor is used to execute instructions stored in the memory; the instructions are used to implement the robust multi-view image clustering method based on smooth anchor graph learning. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] In order to more clearly illustrate the technical implementation scheme in the embodiments of the present application, the present application provides relevant drawings. It should be noted that the drawings described in this section are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on the present invention and the drawings without creative work.

[0046] Figure 1 is a flow chart of the algorithm of the present invention;

[0047] Figure 2 It is the optimization iteration of the objective function;

[0048] Figure 3 This is a comparison of clustering robustness. DETAILED DESCRIPTION

[0049] In order to describe the implementation process of the present invention in detail, the implementation of the present invention will be described in more detail below in conjunction with the relevant drawings. It should be pointed out that the implementation examples described here are only used to better describe the present invention, and the present invention can be implemented in various forms and is not limited to the examples described herein.

[0050] The specific implementation steps of the present invention are as follows Figure 1 As shown, the steps are as follows:

[0051] Step S01: Acquire large-scale multi-view image features under complex noise erosion. It should be pointed out that in some scenarios, the available data only contains imageable data, and different salient feature extraction methods are required for the imageable data to obtain multi-view image features. For example, when the object only contains visible light data, a variety of salient feature extraction methods can be used to extract features to obtain multi-view image features. Noise includes salt and pepper noise, Gaussian noise, Poisson noise, speckle noise, etc., which are caused by sensor errors, data transmission interference, changes in environmental conditions, or shooting equipment failures and other factors.

[0052] Step S02: For the obtained multi-view image feature χ containing V views, (1) ,X (2) ,...,X (V)}, Generate the Laplacian matrix L = {L (1) ,L (2) ,...,L (V)}, Use the Laplacian matrix to perform graph filtering on the noise-corroded original image features to alleviate the complex noise erosion in the original image features:

[0053]

[0054] Where X (v) represents the original image features of the vth view, L (v) represents the Laplacian matrix of the v-th view, represents the image features after smoothing by the Laplace matrix, I represents the unit matrix, and μ represents a balance parameter.

[0055] Step S03: Generate a representative anchor point set, and then use adaptive learning to generate an anchor point map between the anchor point set and the smoothed filtered image features for each view. Among them, n is the number of images required to be clustered, d (v) is the dimension of the image features contained in the vth view; the anchor point set ε = {E (1) ,E (2) ,...,E (V)}, Where m is the number of anchor points. Then, the anchor point graph of each view is adaptively learned based on the topological relationship between the anchor point set and the image features after smoothing filtering. (1) ,B (2) ,...,B (V)},

[0056] For the process of generating anchor point sets for all views, methods including random sampling, K-means clustering, hierarchical clustering, etc. may be used. In some implementations, using K-means clustering to generate anchor point sets as an example, for all views v∈V, the anchor point set of view v may be generated in the following manner:

[0057]

[0058] Where m represents the number of anchor points. represents the jth anchor point, represents the i-th original image feature, represents the region formed by the j-th anchor point as a cluster, It should be noted that for the present invention, the number of anchor points between different views should be unified to ensure that the dimensions of the anchor maps of different views are consistent.

[0059] In the process of generating anchor graphs for all views, various anchor graph forms such as fully connected anchor graphs and k-nearest neighbor anchor graphs may be generated. In some embodiments, a k-nearest neighbor anchor graph is generated. For all views v∈V, an anchor graph of view v may be generated in the following manner:

[0060]

[0061] in represents the element in the i-th row and j-th column of the anchor map of view v, Φ(·) represents the distance metric function, and k represents that the anchor map only records the features of the i-th original image. The relationship between the k nearest anchor points, Represents the feature of the i-th original image The jth anchor point closest to it.

[0062] Step S04: Apply the local popular learning method to fuse the anchor graphs of different views. In some embodiments, different weights are assigned to the anchor graphs of different views, and the fused anchor graph is regarded as the weighted sum of the anchor graphs of different views to obtain a weighted anchor graph:

[0063]

[0064] where α (v) and B (v) Represent the weight of the v-th anchor graph and the v-th anchor graph respectively. By weighted summing the anchor graphs from different views, the weighted anchor graph B can be obtained. A .

[0065] Step S05: Smoothing and denoising the weighted anchor graph under the Frobenius norm metric to obtain a consensus anchor graph, so as to filter out the complex noise contained in the real world.

[0066] In some embodiments, the similarity in the anchor feature space is measured under the Frobenius norm:

[0067]

[0068] in represents the learned consensus anchor graph, Represents the weighted anchor map obtained by weighted summation of anchor maps from different views.

[0069] Step S06: Learn the consensus spectral embedding through the consensus anchor graph, and finally perform K-means clustering on the consensus spectral embedding to obtain the final clustering result.

[0070] In some embodiments, a method for learning spectral embedding using the normalized cut (Ncut) theory is described as follows:

[0071]

[0072] Where Tr[·] represents the trace of the matrix, I represents the identity matrix, F is the consensus spectral embedding of the learned consensus anchor graph, and F and Z represent the transpose of F and Z, respectively. Performing ZZ on the consensus anchor graph is equivalent to obtaining the full sample graph between all samples, and performing I-ZZ is equivalent to obtaining the relaxed Laplacian matrix of the full sample graph.

[0073] The elements in the consensus spectrum embedding F are distributed in the entire real number space, and it is difficult to directly obtain the clustering result. In some embodiments, the K-means clustering method can be used to obtain the categories corresponding to the samples for the obtained consensus spectrum embedding F to obtain the final clustering result. The iterative convergence of the method of the present invention is as follows Figure 2As shown in the figure, it can be seen that this method can converge within 15 iterations and has a good convergence speed.

[0074] The embodiments of the present invention can effectively process large-scale unlabeled multi-view image data under complex noise erosion, and significantly improve the efficiency and accuracy of multi-view clustering tasks. Compared with the multi-view clustering method in the prior art, the clustering accuracy of the present invention is higher, as shown in Table 1.

[0075] Table 1 Comparison of clustering results between the proposed method and advanced multi-view clustering methods on real data sets

[0076]

[0077]

[0078] Table 1 shows the clustering accuracy comparison between the examples of the present invention and other advanced multi-view clustering methods on four image data sets. NUSW is a large-scale data set with more than 10,000 samples. From the results, it can be seen that among different clustering indicators, the clustering effect of the embodiments of the present invention has the highest or second highest clustering accuracy. Compared with the multi-view clustering method in the prior art, the algorithm of the present invention is more robust when facing complex noise erosion, such as Figure 3 shown. Figure 3 The clustering robustness comparison of the example of the present invention and other advanced clustering methods on the image dataset AWA is shown. From the results, it can be seen that under different degrees of salt and pepper noise erosion, the embodiment of the present invention has stable clustering accuracy and shows excellent robustness.

[0079] On the other hand, the present invention also provides a robust multi-view image clustering system based on smooth anchor graph learning, comprising:

[0080] Feature extraction module: obtains multi-view image features with complex noise erosion for clustering tasks;

[0081] Laplacian filter module: constructs the Laplacian matrix to obtain image features, and uses the Laplacian matrix to perform graph filtering on the original image features to alleviate the complex noise erosion in the original image features;

[0082] Anchor map generation module: First, an anchor set is generated for each view. Then, based on the topological relationship between the anchor set and the smoothed image features, an anchor map corresponding to each view is adaptively generated.

[0083] Anchor graph weighting module: It uses local popular learning to assign different weights to the anchor graphs of all views, and fuses the anchor graphs of different views to obtain a weighted anchor graph.

[0084] Anchor graph smoothing module: It performs smoothing learning on the weighted anchor graph under the Frobenius norm metric to reduce the impact of noise in the weighted anchor graph and obtain a robust consensus anchor graph.

[0085] Clustering result acquisition module: The consensus spectrum embedding is obtained according to the consensus anchor point graph, and the final clustering result is obtained through the K-means clustering method.

[0086] On the other hand, the present invention also provides a multi-view image clustering device, which includes a memory and a processor. The memory is used to store instructions and data, and the processor is used to execute instructions stored in the memory; the instructions are used to implement the robust multi-view image clustering method based on smooth anchor graph learning.

[0087] It should be pointed out that although the present invention has given a detailed implementation example to help those skilled in the art understand the overall framework and technical details of the present invention, the present invention is not limited to the described embodiment. Those skilled in the art can make equivalent modifications or substitutions under the premise of following the ideas and spirit of the present invention, and these equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A robust multi-view image clustering method based on smooth anchor graph learning, characterized in that: include: Acquire large-scale multi-view image features eroded by complex noise; Performing graph filtering on the obtained image features to smooth and denoise; Generate a representative anchor point set and adaptively learn the anchor point map between each view anchor point set and the smoothed image data; Apply local popular learning method to fuse anchor maps of different views to obtain weighted anchor maps; The weighted anchor graph is smoothed and denoised under the Frobenius norm to obtain a consensus anchor graph; The consensus spectrum embedding and the corresponding clustering results are obtained.

2. The robust multi-view image clustering method based on smooth anchor graph learning according to claim 1, characterized in that: The method of obtaining large-scale multi-view image features corroded by complex noise includes: A variety of salient features are extracted from visible light data, infrared data and SAR imaging data corroded by complex noise to obtain large-scale multi-view image features.

3. The robust multi-view image clustering method based on smooth anchor graph learning according to claim 1, characterized in that: The performing graph filtering on the obtained image features to perform smoothing and denoising includes: A Laplacian matrix is ​​constructed independently for each view, and the Laplacian matrix is ​​used to perform graph filtering on the original image features eroded by complex noise to alleviate the complex noise erosion of the original image features: Where X (v) represents the original image features of the vth view, L (v) represents the Laplacian matrix of the v-th view, represents the image features after smoothing by the Laplace matrix, I represents the unit matrix, and μ represents a balance parameter.

4. The robust multi-view image clustering method based on smooth anchor graph learning according to claim 1, characterized in that: Generate a representative anchor point set and adaptively learn the anchor point map between each view anchor point set and the smoothed image data, including: For the obtained smoothed image features containing V views eroded by noise Among them, n is the number of images required to be clustered, d (v) is the dimension of the image features contained in the vth view; the anchor point set ε = {E (1) ,E (2) ,...,E (V) }, Where m is the number of anchor points; then the anchor point graph of each view is adaptively learned according to the topological relationship between the anchor point set and the smoothed image features. (1) ,B (2) ,...,B (V) }, 5. The robust multi-view image clustering method based on smooth anchor graph learning according to claim 4, characterized in that: The steps of adaptively learning the anchor point graph of each view according to the topological relationship between the anchor point set and the smoothed image features include: in represents the element in the i-th row and j-th column of the anchor map of view v, Φ(·) represents the distance metric function, and k represents that the anchor map only records the features of the i-th original image. The relationship between the k nearest anchor points; Represents the original image feature The jth anchor point closest to it.

6. The robust multi-view image clustering method based on smooth anchor graph learning according to claim 1, characterized in that: The local popular learning method is used to fuse the anchor graphs of different views to obtain a weighted anchor graph, including: The local popular learning method is used to assign different weights to the anchor maps of different views, and the anchor maps of different views are fused to obtain the weighted anchor map: where α (v) and B (v) Represent the weight of the v-th anchor graph and the v-th anchor graph respectively. By weighted summing the anchor graphs from different views, we get the weighted anchor graph B A .

7. The robust multi-view image clustering method based on smooth anchor graph learning according to claim 1, characterized in that: The weighted anchor graph is smoothed and denoised under the Frobenius norm to obtain a consensus anchor graph, including: Under the Frobenius norm metric, the influence of noise and outliers in the weighted anchor graph is eliminated to obtain a robust consensus anchor graph: in represents the learned consensus anchor graph, Represents the weighted anchor map obtained by weighted summation of anchor maps from different views.

8. The robust multi-view image clustering method based on smooth anchor graph learning according to claim 1, characterized in that: Obtain consensus spectrum embedding and corresponding clustering results, including: Learn the consensus spectral embedding based on the consensus anchor graph, and learn the final cluster indicator matrix based on the consensus spectral embedding: where Tr[·] represents the trace of the matrix, F is the consensus spectral embedding representation of the learned consensus anchor graph, I represents the identity matrix, and F and Z represent the transpose of F and Z respectively; finally, the clustering result is obtained by performing K-means clustering on the consensus spectral embedding representation.

9. A system for implementing the robust multi-view image clustering method based on smooth anchor graph learning according to any one of claims 1 to 8, characterized in that: include: Feature extraction module: obtains multi-view image features with complex noise erosion for clustering tasks; Laplacian filter module: constructs the Laplacian matrix to obtain image features, and uses the Laplacian matrix to perform graph filtering on the original image features to alleviate the complex noise erosion in the original image features; Anchor map generation module: First, an anchor set is generated for each view. Then, according to the topological relationship between the anchor set and the smoothed large-scale multi-view image features, an anchor map corresponding to each view is adaptively generated. Anchor graph weighting module: It uses local popular learning to assign different weights to the anchor graphs of all views, and fuses the anchor graphs of different views to obtain a weighted anchor graph. Anchor graph smoothing module: It performs smoothing learning on the weighted anchor graph under the Frobenius norm metric to reduce the impact of noise in the weighted anchor graph and obtain a robust consensus anchor graph. Clustering result acquisition module: The consensus spectrum embedding is obtained according to the consensus anchor point graph, and the final clustering result is obtained through the K-means clustering method.

10. A multi-view image clustering device, characterized in that: The multi-view image clustering device includes a memory and a processor, wherein the memory is used to store instructions and data, and the processor is used to execute instructions stored in the memory; wherein the instructions are used to implement the robust multi-view image clustering method based on smooth anchor graph learning as described in any one of claims 1-8.

Images