Graph convolution clustering method and device learned from original images

Abstract

A graph convolution clustering method and apparatus for learning from raw images has low requirements on the quality of pre-provided graphs, greater flexibility in model learning, and better robustness for various complex clustering applications. The method includes: (1) using a deep autoencoder (DAE) to learn a highly discriminative data representation in the raw image and explore the potential attribute information of the data; (2) learning an adaptive graph with low rank and sparse structure from the raw graph; introducing the Laplace constraint of the graph to simultaneously learn the optimal graph relationship and the discriminative representation of the data; (3) performing self-supervised clustering and self-supervising the learning of node representations, using soft-assigned distributions as cluster labels in clustering.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a graph convolution clustering method for learning from original images, and a graph convolution clustering device for learning from original images. Background Art

[0002] Clustering is a fundamental and important task for exploring inter-sample correlations. Its goal is to group samples into disjoint clusters based on the intrinsic similarity of their data representations. Due to the strong learning capabilities of discriminative representations, many deep clustering methods based on autoencoders and graph convolutional networks have been widely studied and have achieved promising results.

[0003] (1) Deep clustering based on autoencoders

[0004] Due to the strong representation learning ability of convolutional networks, deep clustering methods have attracted widespread attention and achieved remarkable performance. The core of deep clustering methods is to learn a discriminative representation of the original data. Autoencoder (AE) is a basic and commonly used clustering learning framework for unsupervised clustering tasks. For example, based on the AE framework, Deep Embedded Clustering (DEC) and Improved DEC (IDEC) designed a clustering objective KL divergence loss to guide data representation learning and improve the separability of the representation. Then, some DEC / IDEC-based methods were proposed to address the shortcomings of current methods. In addition, in order to handle multi-view data, many AE-based multi-view deep clustering methods were designed to make full use of complementary multi-view information to improve clustering performance. Unfortunately, these methods only focus on learning effective representations from data attribute features, while ignoring the inherent topological map information.

[0005] (2) Clustering based on graph convolutional networks

[0006] Recently, several deep clustering methods based on graph convolutional networks (GCNs) have been proposed to learn representations that can encode graph structural information based on graph convolution operations. Specifically, the graph autoencoder (GAE) embeds graph convolution operations into the AE framework, aiming to learn graph structural representations for unsupervised tasks. Based on the variational AE, the variational graph autoencoder (VGAE) was naturally proposed. In addition, the adversarial regularized GAE (ARGAE) was proposed, which designed an adversarial constraint in GAE to enhance the robustness of the representation. Furthermore, the structured deep clustering network (SDCN) was proposed to merge attribute information and structural information by unifying the GCN and DEC frameworks. Moreover, SDCN has achieved the most significant clustering performance to date. However, the graph convolution operation treats the different neighbors of each node equally, ignoring the differences between nodes.

[0007] To better distinguish the importance of neighboring nodes to a target node, the Graph Attention Convolutional Network (GAT) was proposed. Based on this, researchers have proposed several GAT-based clustering methods and achieved superior performance. For example, the Deep Graph Attention Embedding Clustering (DAEGC) method integrates a graph attention mechanism with an autoencoder and introduces a self-supervised module to directly obtain cluster labels. Then, by considering the distribution consistency between the learned node representation and the original data representation, the SGCPD model was designed to improve the graph convolutional clustering method.

[0008] Graph-based clustering learns representations of the underlying data by leveraging topological graph structures. In recent years, clustering methods based on graph convolutional networks (GCNs) have gained widespread attention and achieved promising performance. However, their performance heavily depends on the quality of the pre-provided graph, which is typically determined by predefined models such as k-nearest neighbors. Consequently, the graph may be inaccurate due to noise and a fixed graph limits the flexibility of model learning. Summary of the Invention

[0009] In order to overcome the defects of the prior art, the technical problem to be solved by the present invention is to provide a graph convolution clustering method for learning from original images, which has low requirements on the quality of the pre-provided graphs, greater flexibility in model learning, and better robustness to various complex clustering applications.

[0010] The technical solution of the present invention is: this graph convolution clustering method learned from the original image includes the following steps:

[0011] (1) Deep autoencoder (DAE) is used to learn discriminative data representations in original images and explore the potential attribute information of the data;

[0012] (2) Learning an adaptive graph with low rank and sparse structure from the original graph; introducing the Laplace constraint of the graph and learning the optimal graph relationship and discriminative representation of the data at the same time;

[0013] (3) Perform self-supervised clustering, self-supervised node representation learning, and use soft assignment distribution Q in clustering Z as cluster labels.

[0014] During the optimization process, the present invention learns an adaptive graph with low rank and sparse structure, which can better encode the structural information of the data than a fixed graph. Then, in order to explore the local connectivity of the data, the Laplace constraint of the graph is introduced, so that the optimal graph relationship and the discriminant representation of the data can be learned simultaneously, thereby improving the flexibility of the model. By designing a self-supervised clustering module, the learning of node representations can be self-supervised to explore better clustering structures. Therefore, the present method does not have high requirements on the quality of the pre-provided graph, the model learning is more flexible, and it is more robust to various complex clustering applications.

[0015] A graph convolution clustering device for learning from original images is also provided, the device comprising:

[0016] Deep Autoencoder module DAE, which is configured to learn discriminative data representations and explore the latent attribute information of the data;

[0017] Dynamic Graph Embedding (AGE) module, which is configured to learn an adaptive graph with low rank and sparse structure from the original graph; it introduces Laplace constraints on the graph and simultaneously learns the optimal graph relations and discriminative representations of the data;

[0018] A self-supervised clustering module, which is configured to perform self-supervised clustering, self-supervised learning of node representations, and use a soft assignment distribution Q in clustering Z as cluster labels. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A flowchart of a graph convolution clustering method for learning from original images according to the present invention is shown.

[0020] Figure 2 A framework diagram of a graph convolution clustering device for learning from original images according to the present invention is shown. DETAILED DESCRIPTION

[0021] like Figure 1 As shown in Figure 2, this graph convolution clustering method for learning from original images includes the following steps:

[0022] (1) Deep autoencoder (DAE) is used to learn discriminative data representations in original images and explore the potential attribute information of the data;

[0023] (2) Learning an adaptive graph with low rank and sparse structure from the original graph; introducing the Laplace constraint of the graph and learning the optimal graph relationship and discriminative representation of the data at the same time;

[0024] (3) Perform self-supervised clustering, self-supervised node representation learning, and use soft assignment distribution Q in clustering Z as cluster labels.

[0025] During the optimization process, the present invention learns an adaptive graph with low rank and sparse structure, which can better encode the structural information of the data than a fixed graph. Then, in order to explore the local connectivity of the data, the Laplace constraint of the graph is introduced, so that the optimal graph relationship and the discriminant representation of the data can be learned simultaneously, thereby improving the flexibility of the model. By designing a self-supervised clustering module, the learning of node representations can be self-supervised to explore better clustering structures. Therefore, the present method does not have high requirements on the quality of the pre-provided graph, the model learning is more flexible, and it is more robust to various complex clustering applications.

[0026] Preferably, in step (1), it is assumed that the DAE includes encoder layers, and it is assumed that the (l-1)-th layer representation The l-th layer encoder learns a new representation through the following nonlinear operation:

[0027]

[0028] in is the transformation matrix, is the corresponding deviation, φ is the nonlinear activation function; H (0) represents the original feature X;

[0029] After the L encoder learning process, latent and discriminative representations are learned The decoder of DAE The original data is symmetrically reconstructed, and the decoder operation is defined as:

[0030]

[0031] in and are the transformation matrix and the deviation in the decoder learning process,

[0032] The natural definition of the objective function is as follows:

[0033]

[0034] in The reconstructed data features.

[0035] Preferably, in step (2), an adjacency graph matrix S is adaptively learned, and its specific structure and properties are close to the original graph structure A:

[0036]

[0037] stS=S T , S∈[0,1] N×N (5)

[0038] Among them, λ>0 is a balance parameter, the first term Ensure that the new adjacency connection matrix S does not deviate from the original graph matrix A.

[0039] Preferably, in step (2), the objective function of the unified adaptive graph learning model is expressed as:

[0040]

[0041] stS=S T, S∈[0,1] N×N (6)

[0042] Among them, α and β are hyperparameters that balance the contributions of sparse terms and low-rank terms. It means learning an adjacency graph matrix S whose structure and properties are close to the original graph structure A. α||S||1 means pursuing a sparse structure by minimizing the l1 norm of the matrix S. β||S|| * It means minimizing the nuclear norm of the matrix S to pursue a low-rank structure.

[0043] Preferably, in step (2), the objective function of the unified adaptive graph learning model is expressed as:

[0044]

[0045] stS=S T , S∈[0,1] N×N (10)

[0046] Where A is the original graph structure, S is the learned adjacency matrix, N is the number of nodes, and X is the node attribute feature. α, β, and λ are hyperparameters that balance the contributions of sparse terms, low-rank terms, and smooth terms. S =DS is the graph Laplacian matrix of S. Further, we use the normalized Laplacian matrix Replace L S .

[0047] Preferably, in step (2), when learning the adjacency matrix S, a dynamic graph embedding module is defined:

[0048]

[0049] Among them, S is the learned adjacency matrix, I is the N×N identity matrix, is the node degree matrix, where Moreover, W (l) is a learnable parameter. The (l-1)-th layer graph convolution learns representation As the input of the lth layer, the new representation Z is obtained (l) .

[0050] Preferably, in step (3), the overall objective function is:

[0051]

[0052] stS=S T , S∈[0,1] N×N (19)

[0053] Where S is the learned adjacency matrix, N is the number of nodes, and γ is a hyperparameter to balance the contribution of dynamic graph learning and graph convolutional embedding learning representation. and

[0054] Preferably, in step (3), the target cluster distribution P H With the original cluster representation distribution Q H Minimize the KL divergence between:

[0055]

[0056] Among them, P H =[P H|ij ] represents the target cluster distribution, Q H =[Q H|ij ] represents the cluster distribution of H.

[0057] Minimize the distribution P H and Q z KL divergence between

[0058]

[0059] Among them, P H =[P H|ij ] represents the target cluster distribution, Q Z =[Q Z|ij ] represents the cluster distribution of Z, Q Z|ij The calculation method is

[0060]

[0061] Among them, {μ1,μ2,…,μ C} means Z=[z1,z2,…,z N ], and initialize Z by performing K-means. Z =[Q Z|ij ] represents the cluster distribution of Z.

[0062] Preferably, in step (3), a soft assignment distribution Q is used in clustering Z As the cluster label, the label of sample i is obtained by the following formula:

[0063]

[0064] where Q Z =[Q Z|ij ] represents the cluster distribution of Z. i is the cluster label of sample i.

[0065] Those skilled in the art will appreciate that all or part of the steps in the above-described embodiment method can be implemented by instructing the relevant hardware through a program. The program can be stored in a computer-readable storage medium. When the program is executed, it includes the steps of the above-described embodiment method. The storage medium can be: ROM / RAM, a magnetic disk, an optical disk, a memory card, etc. Therefore, corresponding to the method of the present invention, the present invention also includes a graph convolution clustering device for learning from original images. The device is generally represented in the form of functional modules corresponding to the steps of the method. The device includes:

[0066] Deep Autoencoder module DAE, which is configured to learn discriminative data representations and explore the latent attribute information of the data;

[0067] Dynamic Graph Embedding (AGE) module, which is configured to learn an adaptive graph with low rank and sparse structure from the original graph; it introduces Laplace constraints on the graph and simultaneously learns the optimal graph relations and discriminative representations of the data;

[0068] A self-supervised clustering module, which is configured to perform self-supervised clustering, self-supervised learning of node representations, and use a soft assignment distribution Q in clustering Z as cluster labels.

[0069] The technical solution of the present invention is described in more detail below.

[0070] Although existing GCN-based deep clustering methods have achieved satisfactory performance, they are sensitive to the quality of pre-provided graph relations. The adjacency graph matrix constructed by predefined models (such as low-rank self-representation, k-nearest neighbor) may be noisy and cannot accurately represent the relationship between nodes. Moreover, during the training process, the graph is always fixed, which limits the flexibility of model learning. Therefore, in order to explore structural information more accurately, a robust graph convolutional clustering (RGCC) method is proposed, which combines dynamic graph learning and graph embedding representation learning into a unified model. Figure 2 As shown in Figure 1, RGCC consists of three modules: the Deep Autoencoder (DAE) module, the Dynamic Graph Embedding (AGE) module, and the Self-Supervised Clustering module. Next, we give the corresponding details of each module and obtain the final objective function of the RGCC model.

[0071] 1. Deep Autoencoder Module (DAE)

[0072] The original features are usually high-dimensional, redundant, and contain noise and irrelevant information. The autoencoder (AF) can nonlinearly project the original data into a low-dimensional discriminative feature space. Therefore, the deep autoencoder (DAF) is used to learn a data representation with strong discriminative power, which can explore the potential attribute information of the data. Here, it is assumed that the DAE contains encoder layers. Assuming that the (l-1)-th layer representation is given The l-th layer encoder learns a new representation through the following nonlinear operation:

[0073]

[0074] in is the transformation matrix, is the corresponding bias, and φ is the nonlinear activation function. Specifically, H (0) represents the original feature X.

[0075] After the L encoder learning process, latent and discriminative representations are learned In order to improve the discriminability and reconstruction ability of the representation, the decoder of DAE The original data of is symmetrically reconstructed. Similarly, the decoder operation is defined as:

[0076]

[0077] in and are the transformation matrix and the deviation during the decoder learning process respectively. Specifically,

[0078] The representation reconstructed by the decoder should be similar to the original data, so the objective function is naturally defined as follows:

[0079]

[0080] in The reconstructed data features.

[0081] 2. Dynamic Image Embedding Module (AGE)

[0082] In order to learn structural representation, GCN explores high-order structural correlations through multi-layer graph convolution. In GCN, graph convolution learns embedding representation based on graph structure A and node attribute features X, where the lth layer of the graph convolution encoder is

[0083]

[0084] Where W (l) is the learned parameter, and φ is also a nonlinear activation function. Usually, the data feature X is set to Z (0) .

[0085] However, Z (l) The discriminative ability of GCN heavily depends on the quality of the graph structure A. Moreover, GCN is sensitive to the input graph matrix, which may degrade the performance of downstream tasks due to the interference of noise on the graph matrix. In addition, the graph is always fixed during the model learning process, which limits the flexibility of the model. Therefore, to address this issue, a natural strategy is to learn a robust and clean graph structure that accurately represents the connection relationship between nodes. Here, a dynamic graph embedding (AGE) module is proposed, which dynamically learns the graph structure during the node representation learning process.

[0086] Here, a unified dynamic graph learning model is designed. In order to learn robustness and optimal graph structure from the original graph, an adjacency graph matrix S is adaptively learned, whose specific structure and properties are close to the original graph structure A. The objective function of the unified adaptive graph learning model is expressed as:

[0087]

[0088] stS=S T , S∈[0,1] N×N (5)

[0089] Among them, λ>0 is a balance parameter. The first term It ensures that the new adjacency matrix S does not deviate from the original graph matrix A. Moreover, similar to A, the new adjacency matrix S should be symmetric, that is, S = S T Specifically, the general function f(S) expresses structural constraints on S, which improves the quality of the learned adjacency graph matrix.

[0090] Inspired by some work, the ideal adjacency graph matrix is ​​usually low-rank, which explores its intrinsic structure from a global perspective. On the other hand, the adjacency matrix reflects the local connectivity of the nodes, that is, assigning a small number of neighbors. Therefore, the learned adjacency matrix S should be sparse. It is well known that low rank and sparse structure can be pursued by minimizing the nuclear norm and l1 norm of the matrix S. Therefore, the objective function can be reformulated as:

[0091]

[0092] stS=S T , S∈[0,1] N×N (6)

[0093] Among them, α and β are hyperparameters that balance the contribution of sparse terms and low-rank terms.

[0094] In fact, the clean adjacency matrix S reflects the similarity between nodes. Moreover, when the distance between data samples is small, they should have a greater probability of being clustered into the same group. Specifically, similar sample pairs with small distances are divided into neighborhoods, that is, the distance metric The smaller the neighborhood probability S ij The larger the value, the better. This goal can be achieved by optimizing the following model:

[0095]

[0096] Where S is the learned adjacency matrix, where the element S ij represents the probability of neighborhood assignment. Therefore, the learnable adjacency matrix S promotes neighborhood assignment, which is a dynamic process. The optimal neighborhood can be assigned to each target node during the model training process. In addition, the appropriate neighbors of the target node can be learned to perform clustering tasks. The above formula corresponds to the following

[0097]

[0098] Among them L S =DS is the graph Laplacian matrix of S. Further, using the normalized Laplacian matrix Replace L S ,Right now,

[0099]

[0100] where d i is the node v in the graph i Therefore, the objective function of the adaptive graph learning model is

[0101]

[0102] stS=S T , S∈[0,1] N×N (10)

[0103] When learning the adjacency matrix S, a dynamic graph embedding module can be defined:

[0104]

[0105] in and

[0106] In the dynamic graph embedding module, only the output of the (l-1)-th layer is used as the input of the l-th layer. However, there are two representations with different properties, namely the structure representation Z encoded with graph information (l-1) , and the latent representation H encoded with basic attribute information (l-1)Therefore, in the convolution process of the graph, the information of two attributes can be utilized. Specifically, the representation H (l-1) and represents Z (l-1) Fusion is performed as the input of the lth graph convolution layer, that is,

[0107]

[0108] Among them, ∈ is the weight coefficient. Then use As the input of the lth layer in GCN, the new representation Z is obtained (l) :

[0109]

[0110] Therefore, the learned Z (l) It is a comprehensive representation that contains both structured information and latent attribute information.

[0111] 3. Self-supervised clustering module

[0112] For unsupervised clustering tasks, label information is unknown, which cannot provide useful guidance for representation learning. Therefore, the representations learned by the DAE module and the AGE module are not separable. Here, a self-supervised clustering module is used to help the model learn highly discriminative representations. The purpose of the self-supervised clustering module is to improve the confidence of the data clustering distribution. The self-supervised clustering module combines the DAE module and the AGE module into a unified framework, realizing end-to-end training under self-supervised constraints. Let's assume that the outputs of the last layer of DAE and AGE are H and Z respectively. The student's t-distribution is used to measure the similarity between the representation H and its cluster center, that is,

[0113]

[0114] Among them, {μ1,μ2,…,μ C} means H=[h1,h2,…,h N ], and initialized by performing K-means on H. Using Q H =[Q H|ij ] represents the cluster distribution of H. In order to improve the cluster credibility, it provides higher credibility and makes the sample closer to the cluster center. Specifically, Q H Square it and normalize it, that is,

[0115]

[0116] where f j =∑ i Q H|ij . Due to P Hhas an obvious cluster distribution, so it is taken as the target cluster distribution. Therefore, in order to supervise the representation learning, the target cluster distribution P H With the original cluster representation distribution Q H Minimize the KL divergence between

[0117]

[0118] Here, we can also use the target distribution P H To guide the learning of Z on the AGE module, thereby improving the discrimination ability of Z. Similarly, minimizing the distribution P H and Q Z The KL divergence between , that is,

[0119]

[0120] Where Q Z =[Q Z|ij ] represents the cluster distribution of Z. Z|ij The calculation method is

[0121]

[0122] Self-supervised clustering objective function and It not only improves the discrimination ability, but also ensures the consistency of the cluster distribution representing H and Z. In summary, the overall objective function is:

[0123]

[0124] stS=S T , S∈[0,1] N×N (19)

[0125] in Under the above unified framework, a better structured dynamic graph will improve the discriminative ability of the representation. Conversely, the information from the self-supervised clustering module can also guide the learning process of the dynamic graph to fit the clustering task. After optimizing the above objective function, the discriminative representation Z is obtained. (L+1) . And, the representation Z learned by the AGE module (L+1) The two types of information are combined to provide a comprehensive representation of the original data. Therefore, the soft assignment distribution Q is used in clustering. Z As cluster labels. Specifically, the label of sample i can be obtained as follows:

[0126]

[0127] The above description is merely a preferred embodiment of the present invention and does not constitute any form of limitation to the present invention. Any simple modifications, equivalent changes and modifications made to the above embodiments based on the technical essence of the present invention are still within the scope of protection of the technical solution of the present invention.

Claims

1. A graph convolution clustering method learned from raw images, characterized by: The method comprises the following steps: (1) Deep autoencoder (DAE) is used to learn discriminative data representations in original images and explore the potential attribute information of the data; (2) Learning an adaptive graph with low rank and sparse structure from the original graph; introducing the Laplace constraint of the graph and learning the optimal graph relationship and discriminative representation of the data at the same time; (3) Perform self-supervised clustering, self-supervised node representation learning, and use soft assignment distribution Q in clustering Z as cluster labels; In step (1), it is assumed that the DAE contains encoder layers, and given the (l-1)-th layer representation The l-th layer encoder learns a new representation through the following nonlinear operation: in is the transformation matrix, is the corresponding deviation, φ is the nonlinear activation function; H (0) represents the original feature X; After the L encoder learning process, latent and discriminative representations are learned The decoder of DAE The original data is symmetrically reconstructed, and the decoder operation is defined as: in and are the transformation matrix and the deviation in the decoder learning process, The natural definition of the objective function is as follows: in is the data feature of the reconstruction; In step (2), an adjacency graph matrix s is adaptively learned, whose specific structure and properties are close to the original graph structure A: s.t.S=S T ,S∈[0,1] N×N (5) Among them, λ>0 is a balance parameter, the first term Ensure that the new adjacency connection matrix S does not deviate from the original graph matrix A; In step (2), the objective function of the unified adaptive graph learning model is expressed as: s.t.S=S T ,S∈[0,1] N×N (6) Among them, α and β are hyperparameters, balancing the contribution of sparse terms and low-rank terms. Represents learning an adjacency graph matrix S whose structure and properties are close to the original graph structure A, α||S||1 represents pursuing a sparse structure by minimizing the l1 norm of the matrix S, and β||S|| * It means minimizing the nuclear norm of the matrix S to pursue a low-rank structure; In step (2), the objective function of the unified adaptive graph learning model is expressed as: s.t.S=S T ,S∈[0,1] N×N (10) Among them, A is the original graph structure, S is the learned adjacency matrix, N is the number of nodes, X is the node attribute feature, α, β and λ are hyperparameters that balance the contribution of sparse terms, low-rank terms and smooth terms, L S =DS is the graph Laplacian matrix of S, using the normalized Laplacian matrix Replace L S ; In step (2), when learning the adjacency matrix S, a dynamic graph embedding module is defined: Among them, S is the learned adjacency matrix, I is the N×N identity matrix, is the node degree matrix, where Moreover, W (l) is a learnable parameter, and the (l-1)-th layer of graph convolution learns to represent As the input of the lth layer, the new representation Z is obtained (l) ; In step (3), the overall objective function is: s.t.S=S T ,S∈[0,1] N×N (19) Where s is the learned adjacency matrix, N is the number of nodes, and γ is a hyperparameter to balance the contribution of dynamic graph learning and graph convolutional embedding learning representation. and In step (3), the target cluster distribution P H With the original cluster representation distribution Q H Minimize the KL divergence between: Among them, P H =[P H|ij ] represents the target cluster distribution, Q H =[Q H|ij ] represents the cluster distribution of H, minimizing the distribution P H and Q Z KL divergence between Among them, P H =[P H|ij ] represents the target cluster distribution, Q Z =[Q Z|ij ] represents the cluster distribution of Z, Q Z|ij The calculation method is Among them, {μ1,μ2,…,μ C } means Z=[z1,z2,…,z N ] cluster center, perform K-means on Z to initialize, Q Z =[Q Z|ij ] represents the cluster distribution of Z; In step (3), the soft assignment distribution Q is used in clustering Z As the cluster label, the label of sample i is obtained by the following formula: where Q Z =[Q Z|ij ] represents the cluster distribution of Z, r i is the cluster label of sample i.

2. A graph convolution clustering device for learning from raw images, configured to perform the method according to claim 1, characterized in that: The device includes: Deep Autoencoder module DAE, which is configured to learn discriminative data representations and explore the latent attribute information of the data; Dynamic Graph Embedding (AGE) module, which is configured to learn an adaptive graph with low rank and sparse structure from the original graph; it introduces Laplace constraints on the graph and simultaneously learns the optimal graph relations and discriminative representations of the data; A self-supervised clustering module, which is configured to perform self-supervised clustering, self-supervised learning of node representations, and use a soft assignment distribution Q in clustering Z as cluster labels.

Images