Graph comparison learning method based on community prototype
By generating enhanced view and graph attention network based on attention mechanism, combining label propagation algorithm and comparison loss, the community structure information in the graph is captured, and the problem of supervised learning relies on high-cost labels and ignoring community structure in the existing technology is solved. Achieve more robust node representation and higher downstream task performance.
Patent Information
- Application Number
- CN202310581071.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-22
- Publication Date
- 2024-11-22
AI Technical Summary
The existing graph representation learning methods mainly focus on supervised learning, require high-cost manual labels, and have poor generalization in the case of scarce training data and overfitting, ignoring the community structure information in the graph.
Enhanced views are generated through random feature masks and random edge deletion, and a graph attention network (GAT) encoding nodes are used based on attention mechanism, and community detection is performed in combination with tag propagation algorithm, the mean value of node embedding is calculated as a community prototype, and node- Optimize the loss compared with nodes and nodes.
The node-level and community-level information on the graph is captured, the robustness and generalization capabilities of node representation are improved, the performance of downstream tasks is improved, and the experimental results are optimal on multiple standard data sets.
Smart Images

Figure CN119006871A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of artificial intelligence in computer technology, especially the field of data mining and graph representation learning in artificial intelligence. Graph representation learning aims to learn the intricate structural information and feature information in the network, help people better model and analyze various complex data, and provide support for decision-making. Background Art
[0002] Graphs are widely used to represent network structures of connected data. Graph data can be widely obtained in application fields such as social systems, ecosystems, biological networks, knowledge graphs, and information systems. With the development of artificial intelligence technology, graph representation learning has attracted much attention. Graph representation learning can learn the complex structural information and feature information in the network, thereby improving downstream tasks such as node classification and link prediction.
[0003] Existing graph representation learning methods focus on supervised learning scenarios, which require the use of good-quality manual labels to train models. This brings several disadvantages: first, the cost of collecting and annotating manual labels is too high, especially for research fields with large-scale datasets (such as citations and social networks) or high requirements for domain knowledge (such as chemistry and medicine); second, due to overfitting, pure supervised learning schemes usually suffer from poor generalization problems, especially when training data is scarce. To address the above problems, the contrastive learning paradigm provides a feasible solution, which learns discriminative representations by generating enhanced views and comparing positive and negative samples. In the case of only non-semantic labels, unsupervised graph contrastive learning can provide general node representations for various downstream tasks, including node classification, link prediction, and node clustering.
[0004] However, most existing methods only focus on node-level similarity, while ignoring the inherent community structure in the graph. Community structure is ubiquitous in real-world graphs. For example, in citation networks, papers are classified according to their topics, and papers within the same topic community are often closely connected through citation edges, while papers on different topics are loosely connected. Therefore, we believe that nodes within the same community are semantically similar, and community information should be considered to identify node pairs in the same community. Summary of the invention
[0005] This paper aims to capture both node-level information and community-level information on the graph to learn more robust and information-rich node representations, thereby improving the performance of various downstream tasks. The technical solution is:
[0006] 1) Generate two enhanced views by random feature masking and random edge deletion.
[0007] 2) Encode nodes through the graph attention network GAT based on the attention mechanism.
[0008] 3) The edge attention of GAT in 2) is used as the edge weight, and the label propagation algorithm is used to perform community detection on the enhanced views respectively.
[0009] 4) Calculate the mean of node embeddings in the same community as the community prototype.
[0010] 5) Use node-node and node-community contrast losses and optimize with back-propagation algorithm.
[0011] The beneficial effects of the present invention are as follows: a general graph contrastive learning framework is proposed, which enhances the learned node representation by capturing community structure information, making it more generalizable and robust, thereby improving the performance of downstream tasks. We conducted node classification and node clustering experiments on five commonly used standard datasets, namely Cora, Pubmed, Wiki-CS, AmazonCoBuyPhoto, and AmazonCoBuyComputers. The experimental results show that our method has reached the current optimal level. BRIEF DESCRIPTION OF THE DRAWINGS
[0012] Figure 1 This is the graph comparison learning framework based on community prototype in this invention DETAILED DESCRIPTION
[0013] In order to better understand the technical content of the present invention, a specific embodiment is given and described as follows with reference to the accompanying drawings:
[0014] 1. Data Augmentation
[0015] Data augmentation is an important part of contrastive learning. It is mainly responsible for constructing semantically similar samples and affects the quality of the learned representation. We use two graph data augmentation strategies to enhance the attributes and topological information in the graph: randomly select some node feature dimensions and mask them with 0; randomly delete a certain proportion of edges from the original graph. The two enhanced views are denoted as
[0016] 2. Graph Encoder
[0017] In order to make the graph encoder and community detector promote each other, we choose the graph attention network (GAT) based on the attention mechanism as the basic encoder, and use the attention coefficient in the encoder as the edge weight to combine it with community detection. Similar to previous contrastive learning, we also apply a simple projection head (2-layer MLP) to map the representation to the space of calculating community prototypes and contrastive loss.
[0018] 3. Community testing
[0019] The label propagation algorithm (LPA) is used for community detection. LPA assumes that the labels between connected nodes tend to be the same. Like graph neural networks, it is an information transfer model and has a nearly linear time complexity. We use the attention coefficient in the above GAT as the edge weight, and then use LPA to divide the communities of the two enhanced views separately, and calculate the mean value of the node embedding in the same community as the community prototype.
[0020] Note that LPA requires obtaining the initial community labels of nodes. Different from the random initialization of the original algorithm, we use K-Means to cluster the node embeddings as the initial community labels to reduce random errors.
[0021] 4. Contrastive loss
[0022] We use the node-community contrastive loss (ProtoNCE) and node-node contrastive loss (InfoNCE) to optimize the model:
[0023] a) Node-community contrast loss. For view and the community prototypes it contains The node v i Prototype of the community is regarded as a positive sample, and other community prototypes are regarded as negative samples. We define:
[0024]
[0025] Among them, s refers to the cosine similarity and τ is the temperature coefficient. Since there are two views, the node-community comparison loss is:
[0026]
[0027] b) Node-node contrast loss. To learn node-level information, we consider different view representations of the same node as positive samples and different nodes as negative samples:
[0028]
[0029] Symmetrically, the node-node contrastive loss is
[0030] c) Overall loss. We first warm up the model using only the node-node contrast loss to obtain better initial community labels while preventing overfitting of community structure information, and then use two losses to jointly optimize:
[0031]
[0032] 5. Downstream tasks
[0033] After the training is completed, the model parameters can be frozen and the trained model can be used to generate node representations for various downstream tasks such as node classification, node clustering, etc.
Claims
1. A graph contrast learning method based on community prototype. Features The present invention pays more attention to community information, and correlates community detection, community prototypes, and node embedding to improve the quality of graph representation learning. The present invention learns node embedding with community structure information by combining the graph attention network (GAT) and the traditional community detection algorithm (label propagation algorithm, LPA). After experiments, the results show that its performance is better than the existing methods. This method can be applied to the graph representation learning task of unlabeled data, alleviating the limitations of supervised learning such as heavy reliance on labels, poor generalization performance, and weak robustness, and has broad application prospects.
2. According to claim 1, a graph comparison learning method based on community prototype is characterized in that Two graph data enhancement strategies are used to enhance the attributes and topological information of the graph: 1) randomly select some node feature dimensions and mask them with 0; 2) randomly delete a certain proportion of edges from the original graph. Then, the nodes are encoded through the graph attention network GAT based on the attention mechanism.
3. The feature of right 3 is that the edge attention of GAT in right 2 is used as the edge weight, and the label propagation algorithm (LPA) is used to perform community detection on the enhanced views respectively.
4. Claim 4 is characterized in that the community detection result of 3 is used to calculate the mean of node embeddings in the same community and use it as the community prototype.
5. Claim 5 proposes an optimization method, characterized in that The optimization is done using the back-propagation algorithm using node-to-node and node-to-community comparative losses.
6. Claim 6 proposes an application direction of the method, characterized in that After the training is completed based on the above steps, the model parameters are frozen, and the trained model is used to generate node representations, which can then be used for various downstream tasks such as node classification, node clustering, etc.
