Graph comparison learning method based on community prototype

By generating enhanced view and graph attention network based on attention mechanism, combining label propagation algorithm and comparison loss, the community structure information in the graph is captured, and the problem of supervised learning relies on high-cost labels and ignoring community structure in the existing technology is solved. Achieve more robust node representation and higher downstream task performance.

CN119006871APending Publication Date: 2024-11-22NANJING UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202310581071.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2023-05-22
Publication Date
2024-11-22

AI Technical Summary

Technical Problem

The existing graph representation learning methods mainly focus on supervised learning, require high-cost manual labels, and have poor generalization in the case of scarce training data and overfitting, ignoring the community structure information in the graph.

Method used

Enhanced views are generated through random feature masks and random edge deletion, and a graph attention network (GAT) encoding nodes are used based on attention mechanism, and community detection is performed in combination with tag propagation algorithm, the mean value of node embedding is calculated as a community prototype, and node- Optimize the loss compared with nodes and nodes.

Benefits of technology

The node-level and community-level information on the graph is captured, the robustness and generalization capabilities of node representation are improved, the performance of downstream tasks is improved, and the experimental results are optimal on multiple standard data sets.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119006871A_ABST
    Figure CN119006871A_ABST
Patent Text Reader

Abstract

The invention relates to a graph comparison learning method based on a community prototype. Compared with an existing graph comparison learning method, the method has the advantages that more attention is paid to community information, and community detection, community prototypes and node embedding are mutually associated, so that the quality of graph representation learning is improved. According to the method, node embedding with community structure information is learned by combining a graph attention network (GAT) and a traditional community detection algorithm (label propagation algorithm, LPA). A large number of experiments show that the performance of the method is superior to that of the existing method. The method can be applied to graph representation learning tasks of label-free data, alleviates the limitations of severe dependence on labels, poor generalization performance, weak robustness and the like of supervised learning, and has a wide application prospect.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence in computer technology, especially the field of data mining and graph representation learning in artificial intelligence. Graph representation learning aims to learn the intricate structural information and feature information in the network, help people better model and analyze various complex data, and provide support for decision-making. Background Art

[0002] Graphs are widely used to represent network structures of connected data. Graph data can be widely obtained in application fields such as social systems, ecosystems, biological networks, knowledge graphs, and information systems. With the development of artificial intelligence technology, graph representation learning has attracted much attention. Graph representation learning can learn the complex structural information and feature information in the network, thereby improving downstream tasks such as node classification and link prediction.

[0003] Existing graph representation learning methods focus on supervised learning scenarios, which require the use of good-quality manual labels to train models. This brings several disadvantages: first, the cost of collecting and annotating manual labels is too high, especially for research fields with large-scale datasets (such as citations and social networks) or high requirements for domain knowledge (such as chemistry and medicine); second, due to overfitting, pure supervised learning schemes usually suffer from poor generalization problems, especially when training data is scarce. To address the above problems, the contrastive learning paradigm provides a feasible solution, which learns discriminative representations by generating enhanced views and comparing positive and negative samples. In the case of only non-semantic labels, unsupervised graph contrastive learning can provide general node representations for various downstream tasks, including node classification, link prediction, and node clustering.

[0004] However, most existing methods only focus on node-level similarity, while ignoring the inherent community structure in the graph. Community structure is ubiquitous in real-world graphs. For example, in citation networks, papers are classified according to their topics, and papers within the same topic community are often closely connected through citation edges, while papers on different topics are loosely connected. Therefore, we believe that nodes within the same community are semantically similar, and community information should be considered to identify node pairs in the same community. Summary of the invention

[0005] This paper aims to capture both node-level information and community-level information on the graph to learn more robust and information-rich node representations, thereby improving the performance of various downstream tasks. The technical solution is:

[0006] 1) Generate two enhanced views by random feature masking and random edge deletion.

[0007] 2) Encode nodes through the graph attention network GAT based on the attention mechanism.

[0008] 3) The edge attention of GAT in 2) is used as the edge weight, and the label propagation algorithm is used to perform community detection on the enhanced views respectively.

[0009] 4) Calculate the mean of node embeddings in the same community as the community prototype.

[0010] 5) Use node-node and node-community contrast losses and optimize with back-propagation algorithm.

[0011] The beneficial effects of the present invention are as follows: a general graph contrastive learning framework is proposed, which enhances the learned node representation by capturing community structure information, making it more generalizable and robust, thereby improving the performance of downstream tasks. We conducted node classification and node clustering experiments on five commonly used standard datasets, namely Cora, Pubmed, Wiki-CS, AmazonCoBuyPhoto, and AmazonCoBuyComputers. The experimental results show that our method has reached the current optimal level. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Figure 1 This is the graph comparison learning framework based on community prototype in this invention DETAILED DESCRIPTION

[0013] In order to better understand the technical content of the present invention, a specific embodiment is given and described as follows with reference to the accompanying drawings:

[0014] 1. Data Augmentation

[0015] Data augmentation is an important part of contrastive learning. It is mainly responsible for constructing semantically similar samples and affects the quality of the learned representation. We use two graph data augmentation strategies to enhance the attributes and topological information in the graph: randomly select some node feature dimensions and mask them with 0; randomly delete a certain proportion of edges from the original graph. The two enhanced views are denoted as

[0016] 2. Graph Encoder

[0017] In order to make the graph encoder and community detector promote each other, we choose the graph attention network (GAT) based on the attention mechanism as the basic encoder, and use the attention coefficient in the encoder as the edge weight to combine it with community detection. Similar to previous contrastive learning, we also apply a simple projection head (2-layer MLP) to map the representation to the space of calculating community prototypes and contrastive loss.

[0018] 3. Community testing

[0019] The label propagation algorithm (LPA) is used for community detection. LPA assumes that the labels between connected nodes tend to be the same. Like graph neural networks, it is an information transfer model and has a nearly linear time complexity. We use the attention coefficient in the above GAT as the edge weight, and then use LPA to divide the communities of the two enhanced views separately, and calculate the mean value of the node embedding in the same community as the community prototype.

[0020] Note that LPA requires obtaining the initial community labels of nodes. Different from the random initialization of the original algorithm, we use K-Means to cluster the node embeddings as the initial community labels to reduce random errors.

[0021] 4. Contrastive loss

[0022] We use the node-community contrastive loss (ProtoNCE) and node-node contrastive loss (InfoNCE) to optimize the model:

[0023] a) Node-community contrast loss. For view and the community prototypes it contains The node v i Prototype of the community is regarded as a positive sample, and other community prototypes are regarded as negative samples. We define:

[0024]

[0025] Among them, s refers to the cosine similarity and τ is the temperature coefficient. Since there are two views, the node-community comparison loss is:

[0026]

[0027] b) Node-node contrast loss. To learn node-level information, we consider different view representations of the same node as positive samples and different nodes as negative samples:

[0028]

[0029] Symmetrically, the node-node contrastive loss is

[0030] c) Overall loss. We first warm up the model using only the node-node contrast loss to obtain better initial community labels while preventing overfitting of community structure information, and then use two losses to jointly optimize:

[0031]

[0032] 5. Downstream tasks

[0033] After the training is completed, the model parameters can be frozen and the trained model can be used to generate node representations for various downstream tasks such as node classification, node clustering, etc.

Claims

1. A graph contrast learning method based on community prototype. Features The present invention pays more attention to community information, and correlates community detection, community prototypes, and node embedding to improve the quality of graph representation learning. The present invention learns node embedding with community structure information by combining the graph attention network (GAT) and the traditional community detection algorithm (label propagation algorithm, LPA). After experiments, the results show that its performance is better than the existing methods. This method can be applied to the graph representation learning task of unlabeled data, alleviating the limitations of supervised learning such as heavy reliance on labels, poor generalization performance, and weak robustness, and has broad application prospects.

2. According to claim 1, a graph comparison learning method based on community prototype is characterized in that Two graph data enhancement strategies are used to enhance the attributes and topological information of the graph: 1) randomly select some node feature dimensions and mask them with 0; 2) randomly delete a certain proportion of edges from the original graph. Then, the nodes are encoded through the graph attention network GAT based on the attention mechanism.

3. The feature of right 3 is that the edge attention of GAT in right 2 is used as the edge weight, and the label propagation algorithm (LPA) is used to perform community detection on the enhanced views respectively.

4. Claim 4 is characterized in that the community detection result of 3 is used to calculate the mean of node embeddings in the same community and use it as the community prototype.

5. Claim 5 proposes an optimization method, characterized in that The optimization is done using the back-propagation algorithm using node-to-node and node-to-community comparative losses.

6. Claim 6 proposes an application direction of the method, characterized in that After the training is completed based on the above steps, the model parameters are frozen, and the trained model is used to generate node representations, which can then be used for various downstream tasks such as node classification, node clustering, etc.