A local information preserving deep contrastive clustering method for graph structured data clustering

Abstract

The present invention discloses a local information preserving deep comparative clustering method applied to graph structure data clustering, and belongs to the field of artificial intelligence technology. It includes the following modules: a learnable data enhancement module trained based on reconstruction method, a high confidence sampling module, and a graph comparison clustering module. Compared with the prior art, the present invention designs a graph data enhancement method that preserves local information for graph structure data, generates embeddings of different semantic information through two graph autoencoders with non-shared parameters, and then uses them for edge sampling to generate two views for passing into the graph comparison model. At the same time, the present invention designs a screening mechanism for the collection of positive and negative samples to ensure that the embeddings of samples of the same type have higher semantic similarity; compared with traditional clustering algorithms, it can fully explore the structural information and feature information of the graph, and has higher clustering accuracy.

Description

Technical Field

[0001] The present invention belongs to the fields of data mining, graph clustering and artificial intelligence technology, and more specifically, is a local information preserving deep contrast clustering method applied to graph structure data clustering. Background Art

[0002] Graphs are a type of high-dimensional, irregular data (graph structured data, hereinafter referred to as graph data), which contains rich attribute information and structural information. They are widely used in the fields of biomolecule prediction, social network analysis, and recommendation systems. How to efficiently extract graph information and classify it has become an important part of graph representation learning. Clustering, as an unsupervised method, can divide data based on the similarity of its features without the guidance of real labels. However, traditional clustering methods, such as K-means, spectral clustering, DeepWalk, and DBSCAN, can often only discover graph attribute information or structural information, and cannot effectively extract both. At the same time, their efficiency and performance are relatively low when facing high-dimensional sparse data.

[0003] A more effective approach is to combine deep neural networks with traditional clustering methods. The former can be used to extract features and reduce the dimension of data, and then used for subsequent clustering tasks. At present, the main deep graph clustering algorithms can be summarized into two types. One type is a deep graph clustering algorithm based on a generative model. For example, Pan et al. proposed an adversarial way to train a graph autoencoder, so that the distribution of sample embeddings obeys a prior distribution (ARGA algorithm). Wang et al. used a graph attention network to construct a graph autoencoder, and constructed soft labels through Student T distribution to optimize the clustering process (DAEGC algorithm). The other type is a deep graph clustering method based on a contrast model. For example, Zhao et al. used the embedding matrix of the graph contrast module to cluster, and used pseudo labels to filter out false negative samples in negative samples (GDCL algorithm). Liu et al. proposed a sampling strategy for using difficult positive samples in the graph contrast model, and achieved good clustering results (HSAN algorithm). The former type of algorithm can effectively extract the features of graph data, but the defects of the graph convolution layer itself cannot effectively shorten the distance between samples of the same type. As the number of graph convolution layers increases, the discriminability of features between categories decreases. The latter type of algorithm generates two inputs through artificially set data enhancement methods, and trains based on the collection of positive and negative samples to shorten the distance between positive samples and increase the distance between negative samples. This method relies heavily on the setting method and cannot be effectively promoted between different data set processing tasks.

[0004] In comparison, the present invention proposes a local information preserving deep contrastive clustering method for graph structure data clustering, which uses deep learning graph contrastive learning as the basic framework, and uses a learnable data enhancement module trained based on reconstruction to adaptively generate data enhancement views of the graph according to different data sets, while retaining the discriminative information of the two views and increasing the amount of key information they share. At the same time, a high-confidence sampling module is used to improve the reliability of the positive and negative samples used. Summary of the invention

[0005] The purpose of the present invention is to solve the problem that the existing deep graph clustering algorithm cannot efficiently extract discriminative features, resulting in unsatisfactory clustering effect. To this end, the present invention provides a local information preserving deep contrast clustering method for graph structure data clustering.

[0006] In order to solve the above problems, the model training steps adopted by the present invention are as follows: A. In the pre-training stage, two graph autoencoders with non-shared parameters are used to extract features from the graph data to obtain different semantic information, and the preliminary data enhancement views obtained by their reconstruction are edge sampled to obtain two target data enhancement views. At the same time, the embeddings of the two model branches are feature fused for training the reconstruction loss. The two obtained data enhancement views are input into the graph comparison learning module and trained by the conventional InfoNCE loss function. B. In the formal training stage, the parameters of the data enhancement learning module in the pre-training stage are fixed. At the same time, the Euclidean distance between the sample and the center of each cluster is used to select samples with higher confidence. Samples with pseudo labels that are identified as being in the same cluster and having higher confidence are used as positive samples, and the rest are used as negative samples. Afterwards, the positive and negative samples are further screened through the first-order field of the adjacency matrix. The method includes a learnable data enhancement module trained based on a reconstruction method, a high-confidence sampling module, and a graph comparison clustering module;

[0007] The reconstruction-based trained learnable data augmentation module uses a neural network to dynamically generate two views based on different data sets, which share sufficient mutual information for downstream clustering tasks;

[0008] The graph comparison clustering module is responsible for extracting features from the two branches of the data-enhanced sample input model, shortening the distance between the features of the positive samples and increasing the distance between the features of the negative samples, and then performing clustering.

[0009] The high-confidence sampling module is responsible for screening out some positive and negative samples with high confidence and sufficient structural information to improve the ability of the graph contrast clustering module to extract discriminative features, thereby obtaining a better graph clustering result; further, the learnable data enhancement module trained based on the reconstruction method includes two graph autoencoders 100 with non-shared parameters, and an edge sampling module 200. Among them, the graph structure data set 000 is input into the graph convolution layer 110 and the graph convolution layer 111 for feature extraction to obtain embedded features 120 and embedded features 121, and then the features are input into the decoder layer 130 and the decoder layer 131 to obtain a preliminary data enhancement view. Subsequently, the two views are subjected to the edge sampling module 200 to obtain data enhancement views 210 and data enhancement views 211. At the same time, the embedded features 120 and the embedded features 121 are fused to obtain a feature fusion module 140, which is input into the decoder layer 150 to obtain the reconstructed original graph data.

[0010] Further, the graph comparison clustering module includes a graph filtering module 300 and a graph comparison module 400. The data enhancement view 210 and the data enhancement view 211 are feature aggregated by the graph filtering module 300 to obtain a filtered feature matrix 310 and a filtered feature matrix 311, and then feature extraction is performed by the double-layer DNN layer 410 and the double-layer DNN layer 411 of the graph comparison module 400 to obtain embedded features 420 and embedded features 421.

[0011] Furthermore, the high-confidence sampling module 500 includes two parts: reliable sample screening and topology screening. The embedded features 420 and embedded features 421 of the graph comparison module are feature fused to obtain 430, and a pseudo-label is obtained through K-means. Then, the credibility of the pseudo-label is determined according to the Euclidean distance between the sample corresponding to the pseudo-label and the K-means cluster center, and the samples in the same cluster with a credibility greater than a threshold are selected as positive and negative samples. The first-order neighborhood of the target node in the adjacency matrix is ​​further screened to obtain the target positive and negative samples.

[0012] Furthermore, the graph convolution layer 110 and the graph convolution layer 111 use double-layer GCNs, and each layer uses ReLU as an activation function. The decoder layer 130, the decoder layer 131, and the decoder layer 150 obtain the reconstructed graph structure data by performing the inner product of the embedding matrix. The feature fusion module 140 uses the method of taking the average of two embedded features.

[0013] Furthermore, the edges of the edge sampling method are composed of the edges in the original adjacency matrix and the first K edges with the largest inner product similarity, wherein the value of K is between 1 and 5. The views obtained by the decoder layer 130 and the decoder layer 131 are subjected to Bernoulli sampling according to the size of the inner product similarity to obtain the data enhancement view 210 and the data enhancement view 211.

[0014] After using the above modules, the deep clustering task of graph datasets can be completed.

[0015] Beneficial effects of the present invention:

[0016] 1. The present invention has a significant effect on the problem of large differences in the topological information of the attributes of different graph structure data. The proposed learnable data enhancement module based on reconstruction training adaptively learns and generates corresponding graph data enhancement subgraphs according to the type of graph structure data (such as citation network or social network), avoiding a large number of manual parameter modulation operations and potential damage to the data caused by manual operations. At the same time, training in the direction of reconstruction retains the local information shared between the two views, improving the discriminability of the features extracted by the graph comparison model.

[0017] 2. The present invention uses a graph comparison clustering module to extract features of graph structure data. While reducing the higher dimensions of graph structure data, it can also effectively obtain discriminative feature embedding, and obtain better clustering effect by fusing the features of the two branches.

[0018] 3. Compared with the traditional contrastive learning method applied to graph structured data, the high-confidence sampling module proposed in the present invention can supplement more positive samples with high confidence (with higher topological and attribute similarities in social networks or citation networks), while reducing the adverse effects of false negative samples in negative samples, avoiding the situation where some clusters are embedded and alienated, thereby improving the accuracy of model clustering. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is a schematic diagram of the overall structure of a local information preserving deep contrast clustering method applied to graph structure data clustering on a graph data set of the present invention.

[0020] Figure 2 It is a schematic diagram of the clustering process of a local information preserving deep contrast clustering method applied to graph structure data clustering on a graph data set in the present invention. DETAILED DESCRIPTION

[0021] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments, wherein the specific embodiments described are intended to explain the present invention, but are not intended to limit the present invention.

[0022] A local information preserving deep contrast clustering method for graph structure data clustering includes a learnable data enhancement module trained based on reconstruction, a high confidence sampling module, and a graph contrast clustering module which are sequentially connected; the learnable data enhancement module trained based on reconstruction uses a neural network to dynamically generate two views according to different types of graph structure data (such as citation networks, social networks), and they share sufficient mutual information for downstream clustering tasks; the graph contrast clustering module is responsible for inputting data-enhanced citation network node samples into two branches of the model for feature extraction, shortening the distance between positive sample features and increasing the distance between negative sample features, and then performing clustering to divide citation network nodes of different categories; the high confidence sampling module is responsible for screening out some positive and negative samples with high confidence and sufficient structural information to improve the ability of the graph contrast clustering module to extract discriminative features, thereby obtaining a better graph clustering result;

[0023] The following is combined with Figure 1 to Figure 2 A local information preserving deep contrast clustering method for graph structure data clustering provided by the present invention is described in detail.

[0024] The above-mentioned learnable data enhancement module trained based on reconstruction includes two graph autoencoders 100 with non-shared parameters and an edge sampling module 200. The original graph structure data set 000 of the citation network is respectively input into the graph convolution layer 110 and the graph convolution layer 111 for feature extraction to obtain embedded features 120 and embedded features 121, and then the features are input into the decoder layer 130 and the decoder layer 131 to obtain preliminary data enhancement views. Subsequently, the two views are subjected to the edge sampling module 200 to obtain data enhancement views 210 and data enhancement views 211. At the same time, the embedded features 120 and the embedded features 121 are fused by features to obtain a feature fusion module 140, which is input into the decoder layer 150 to obtain reconstructed graph structure data.

[0025] The two non-shared parameter graph autoencoders 100 are composed of a graph convolutional neural network, where each layer of the graph convolutional neural network is in the form of l represents the number of model layers, W (l) represents the weight matrix of the l-th layer graph convolutional neural network, H (l) is the output of the l-th layer network, and σ represents the activation function. represents the adjacency matrix with self-loops added, Denotes its corresponding degree matrix. In the present invention, l=2, σ is the ReLU activation function. The decoder layers 130, 131, 150 are composed of the inner product of the embedding matrix: in represents the reconstructed matrix, represents the inner product similarity between the i-th element and the j-th element, z(u) represents the embedding matrix, and u represents which decoder generates the embedding. The fused feature 140 is obtained by taking the average of the embedded feature 120 and the embedded feature 121. By comparing the output of decoder 150 The difference between the adjacency matrix of the original graph structure data is used as the loss function to optimize the learnable data enhancement module trained based on the reconstruction method.

[0026] The edge set obtained by the edge sampling method consists of the edges in the original adjacency matrix and the first K edges with the largest inner product similarity, which represent the topological association of citations and the association of similar attributes between the nodes of the citation network. The value of K is between 1 and 5. The view obtained by the decoder layer 130 and the decoder layer 131 and View Bernoulli sampling is performed based on the similar size of the inner product to obtain the data augmentation view 210 and the data augmentation view 211, which are respectively represented as and

[0027] The above-mentioned graph comparison clustering module includes a graph filtering module 300 and a graph comparison module 400. The data enhancement view 210 and the data enhancement view 211 are feature aggregated by the graph filtering module 300 to obtain a filtered feature matrix 310 and a filtered feature matrix 311, and then feature extraction is performed through two DNN layers 410 and 411 of the graph comparison module 400 to obtain an embedding matrix 420 and an embedding matrix 421.

[0028] The graph filtering module 300 is composed of the product of t layers of Laplace smoothing filters of the general form: Represents the number of feature matrices after filtering, i=2 in the present invention. Represents the i-th data augmentation view Added self-loop, express The corresponding degree matrix. X represents the vector of the original features of the graph structure data. The vector representing the i-th feature after graph filtering. The value of t ranges from 2 to 5. The number of units in the first hidden layer of the double-layer DNN layer is in the set {128, 256, 512}, and the number of units in the embedding layer is half of the former, performing nonlinear transformation operations on the input features.

[0029] The above-mentioned high-confidence sampling module 500 includes two parts: reliable sample screening and topology screening. The embedded features 420 and embedded features 421 of the graph comparison module are feature fused to obtain 430, and the pseudo-label is obtained through K-means. Then, the credibility of the pseudo-label is determined according to the Euclidean distance between the sample corresponding to the pseudo-label and the K-means cluster center, and the samples in the same cluster with a credibility greater than the threshold are selected as positive and negative samples. The first-order neighborhood of the target node in the adjacency matrix is ​​further screened to obtain the target positive and negative samples.

[0030] The method of selecting samples in the same cluster is P represents the pseudo label, Q ij =1 means that the pseudo labels of the i-th sample and the j-th sample are the same; Q ij = 0 means that the pseudo labels of the clusters of the i-th sample and the j-th sample are different. The collection method of reliable samples is as follows:

[0031]

[0032] sample reliable ={i|d i <thre}

[0033] Center k represents the embedding of the cluster center of the Kth cluster, Z i represents the embedding of the i-th sample output. ik It represents the Euclidean distance between the i-th sample and the K-th cluster center after softmax. i Indicates the distance of the i-th sample to the cluster to which it belongs. i The sample pairs whose similarity is less than the threshold thre are selected as reliable samples sample reliable Then, positive samples are selected through reliable sample constraints, cluster constraints, and first-order neighborhood constraints: sample pos ={(i, j)|i∈sample reliable , j∈sample reliable , Q ij =1, A ij =1}. And select the reliable sample pairs that are not in the same cluster as negative sample pos ={(i, j)|i∈sample reliable , j∈sample reliable , Q ij =0}. In the present invention, the threshold thre is set to 0.3.

[0034] Table 1: Experimental results on two public graph structure datasets (CORA, ACM) and compared with existing algorithms

[0035]

[0036] From the clustering results in Table 1, it can be seen from the values ​​of the three common metrics of graph clustering algorithms, accuracy (ACC), normalized mutual information (NMI), and adjusted Rand index (ARI), that the performance of the method proposed in the present invention is higher than that of several excellent deep graph clustering algorithms in recent years (DAEGC, ARGA, SDCN, GDCL, HSAN, these comparison methods are introduced in the background technology).

[0037] The above are only preferred embodiments of the present invention and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent substitution, improvement, etc. made within the spirit and principle of the present invention should be included in the scope of the claims of the present invention.

Claims

1. A local information preserving deep contrast clustering method for graph structure data clustering, characterized in that: It includes a sequentially connected learnable data enhancement module trained based on reconstruction, a graph comparison clustering module, and a high-confidence sampling module; The reconstruction-based trained learnable data augmentation module uses a neural network to dynamically generate two views based on different data sets, which share sufficient mutual information for downstream clustering tasks; The graph comparison clustering module is responsible for extracting features from the two branches of the data-enhanced sample input model, shortening the distance between the features of the positive samples and increasing the distance between the features of the negative samples, and then performing clustering. The high-confidence sampling module is responsible for screening out some positive and negative samples with high confidence and sufficient structural information to improve the ability of the graph comparison clustering module to extract discriminative features, thereby obtaining a graph clustering result; The reconstruction-based training learnable data enhancement module comprises two graph autoencoders (100) with non-shared parameters and an edge sampling module (200), wherein a graph structure data set (000) is respectively input into a first graph convolution layer (110) and a second graph convolution layer (111) for feature extraction to obtain a first embedded feature (120) and a second embedded feature (121), and then the first embedded feature (120) and the second embedded feature (121) are input into a first decoder layer (130) and a second decoder layer (131) to obtain a preliminary data enhancement view; then the two views are input into the edge sampling module (200) to obtain a first data enhancement view (210) and a second data enhancement view (211); at the same time, the first embedded feature (120) and the second embedded feature (121) are input into a feature fusion module to obtain a first fused feature (140), and the first fused feature (140) is input into a third decoder layer (150) to obtain reconstructed graph structure data; The graph comparison clustering module comprises a graph filtering module (300) and a graph comparison module (400); the first data enhancement view (210) and the second data enhancement view (211) are subjected to feature aggregation through the graph filtering module (300) to obtain a filtered first feature matrix (310) and a filtered second feature matrix (311); and then feature extraction is performed through a first double-layer DNN layer (410) and a second double-layer DNN layer (411) of the graph comparison module (400) to obtain a first embedding matrix (420) and a second embedding matrix (421); The high-confidence sampling module (500) includes two parts: reliable sample screening and topology screening; the first embedding matrix (420) and the second embedding matrix (421) of the graph comparison module are subjected to feature fusion to obtain a second fused feature (430), and the second fused feature (430) is subjected to a standard K-means algorithm to obtain a pseudo label; then, the credibility of the pseudo label is determined according to the Euclidean distance between the sample corresponding to the pseudo label and the cluster center of the K-means algorithm, and samples in the same cluster with a credibility greater than a threshold are selected as positive and negative samples, and the first-order neighborhood of the target node in the adjacency matrix is ​​used for further screening to obtain target positive and negative samples; The first graph convolution layer (110) and the second graph convolution layer (111) both use double-layer GCNs, each layer uses ReLU as an activation function, the first decoder layer (130), the second decoder layer (131), and the third decoder layer (150) all obtain reconstructed graph structure data by performing an inner product of an embedding matrix, and the first fusion feature (140) uses a method of taking the average of two embedding matrices; The edges of the edge sampling module are composed of the edges in the original adjacency matrix and the first K edges with the largest inner product similarity, where the value of K is between 1 and 5. The views obtained by the first decoder layer (130) and the second decoder layer (131) are subjected to Bernoulli sampling according to the size of the inner product similarity to obtain the first data enhancement view (210) and the second data enhancement view (211).

Images