A graph classification training method based on supervised contrast learning and structure inference

Abstract

The application discloses a kind of graph classification training methods based on supervised contrast learning and structure inference, first, the model is fully mined by structure inference Topological information of graph data itself as originally graph data Enhanced graph.Then through label random sampling constructs sample positive-negative example pair, and using hierarchical graph neural network respectively to positive-negative example sample learning to extract whole graph embedding.Finally, through ordinary classification loss and supervised contrast learning loss jointly guide the learning process of hierarchical graph neural network, improve the classification performance of embedding.The application fuses structure inference and label information, the data enhancement mode based on structure inference does not need prior knowledge, expands the scope of model use, accelerates model learning speed.Self-supervised contrast learning on graph data is extended to contrast learning under label supervision, enhances its contrast learning ability.The application improves graph classification performance, and has good generalizability on generalized graph classification data.

Description

Technical Field

[0001] The present invention relates to a topological information extraction and contrastive learning method for graph data structures. Aiming at the problem that graph data structures are difficult to classify using traditional methods, a graph classification training model based on supervised contrastive learning and structural inference is designed. Background Art

[0002] Graph data structure (graph) is a general term for a class of relational data structures, which are usually used to represent systems with complex interrelationships in the real world, such as social networks, protein networks, and transportation networks. Specifically, each graph consists of nodes and edges, where each node corresponds to an entity that is either concrete or abstract, the features on the node are abstracted from the specific features of the entity, and each edge represents the relationship between a pair of entities. This relationship between nodes and edges constitutes the topological information structure of the graph. The graph classification task is to summarize the topological structure and features of each graph and classify different graphs into different categories. Today, graph classification tasks have been applied to many fields, such as chemical molecule property prediction, brain disease classification, point cloud image classification, and so on.

[0003] Currently popular graph classification methods can be roughly divided into two categories: graph kernel (GK) methods and graph neural networks (GNNs). Common GNN methods include random walk kernels, shortest path kernels, and Weiss-Lehman kernels. GNNs require manually defined kernel functions to summarize the graph's topological structure. This lack of generalizability and learnability in more general datasets limits their applicability. Graph neural networks (GNNs) adapt currently popular artificial neural network algorithms to the topological properties of graph data, offering excellent scalability and learnability. Because GNNs can have deep structures, stacking depth can be used to extract hidden topological information from graphs. However, blindly increasing the number of GNN layers can lead to oversmoothing of node features, reducing feature discernibility. To address this issue, subgraph neural networks and GNNs have been proposed, forming a hierarchical feature extraction model. However, targeted designs for graph classification remain lacking, hindering the full potential of GNNs.

[0004] In general classification, contrastive learning can enhance a model's classification capabilities. However, there are few attempts in graph classification, particularly in supervised contrastive learning. Furthermore, structural inference can be used to improve the quality of graphs, thereby enhancing their structure and improving their classification. Structural inference is a technique that infers the structure of graphs through sequential sampling. It can intuitively represent the deep topological information of graphs and improve the quality of graph data. Summary of the Invention

[0005] In response to the problems of lack of label information and reliance on prior knowledge in current graph classification tasks using contrastive learning for data augmentation, this paper proposes a graph classification model based on supervised contrastive learning with structure inference into graph neural networks (SupCosine). First, the model fully exploits the topological information of the graph data itself as an enhanced graph of the original graph data through structural inference. Then, positive and negative pairs of samples are constructed by random sampling through labels, and a hierarchical graph neural network is used to learn the positive and negative samples separately to extract the embedding of the entire graph. Finally, the learning process of the hierarchical graph neural network is jointly guided by the ordinary classification loss and the supervised contrastive learning loss to improve the classification performance of the embedding.

[0006] The main idea of ​​implementing the present invention is to score each edge of the graph in the process of information flow according to the simulated time series, and to mine the nodes and edges with important information hubs to extract the potential deep topological features of the graph. These topological features combined with the original graph can directly express the information that originally required multi-layer GNNs to extract in the change of topological structure, thereby improving its information extraction ability when the number of GNN layers is certain. In terms of training guidance for the entire model, a combination of supervised contrastive learning and traditional classification learning is adopted. Introducing supervised contrastive learning in graph classification tasks can improve the uniqueness of embeddings. At the same time, introducing label information in self-supervised contrastive learning can help contrastive learning loss clarify the learning direction and increase the distinctiveness between embeddings of different categories, thereby improving the overall classification performance of the model.

[0007] A graph classification method based on supervised contrastive learning and structural inference includes the following steps:

[0008] Step 1: Data acquisition: Based on datasets such as MUTAG, PTC, PROTEINS, and IMDBBINARY, obtain the basic data required for graph classification, namely the adjacency matrix A and node features X.

[0009] Step 2: Generate simulated time series: Based on the basic data obtained in step 1, generate a corresponding simulated time series set C for the adjacency matrix A of each graph.

[0010] Step 3, structural inference: According to the time series set C of each graph, the adjacency probability of each edge is calculated through the posterior, and applied to, and the threshold k is applied to generate the enhanced graph

[0011] Step 4: Construct positive and negative samples: and X are divided into positive and negative samples G according to the graph labels p and G n .

[0012] Step 5: Generate subgraph embedding: for positive and negative samples G p and G n , breadth-first search is used on each node to generate the corresponding subgraph partition, and subgraph neural network is applied on the corresponding subgraph to generate subgraph embedding.

[0013] Step 6: Generate graph embedding: Score and sample the subgraph embedding to construct a one-dimensional embedding that can represent the current graph data, and use this embedding to generate the corresponding graph classification label P.

[0014] Step 7: Supervised contrastive learning: The image embeddings of positive and negative samples and the corresponding supervised labels are used to calculate the loss through the contrastive learning function. The image classification label P and the true label are used to calculate the general classification loss, and the two are combined to form the final classification loss.

[0015] In step eight, the model is iteratively updated based on the classification loss, and after convergence, the final graph classification label P' is used as the final graph classification model output.

[0016] Compared with the prior art, the present invention has the following obvious advantages and beneficial effects:

[0017] (1) The integration of structural inference and label information enhances the ability of contrastive learning in graph classification tasks.

[0018] (2) The data enhancement method based on structural inference does not require prior knowledge, which expands the scope of model application and accelerates the learning speed of the model.

[0019] (3) The self-supervised contrastive learning on graph data is extended to contrastive learning under label supervision, which enhances its contrastive learning ability.

[0020] (4) Experimental results on PTC show that the present invention can improve graph classification performance and has good generalizability on generalized graph classification data. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Figure 1 : Flowchart of the model involved in this method.

[0022] Figure 2 : Ablation experiment.

[0023] Figure 3 : Comparison of model accuracy during training process. DETAILED DESCRIPTION

[0024] The specific implementation and detailed steps of the present invention are described below. The specific implementation process of the present invention is as follows: Figure 1As shown, specifically including:

[0025] (Step 1) Data acquisition: In order to verify the effectiveness of the model proposed in the present invention, experiments were conducted on the MUTAG, PTC, IMDBBINARY and PROTEINS datasets to evaluate the classification performance of the model. Among them, the MUTAG and PTC datasets are compound molecule datasets, and each graph represents a compound molecule. IMDBBINARY is the Internet Movie Database, each node represents an actor, and an edge represents two actors appearing in the same movie. PROTEINS is a protein molecule dataset, and its topological structure represents the topological shape of the corresponding protein molecule in space. For each graph in the above datasets, it can be expressed as G = (V, E), where V represents a node set and E represents an edge set. For an attribute graph, it can be further expressed as G = (X, A) | X∈R n×d ,A∈R n×n , where X is the set of attributes of each node, A is the adjacency matrix that represents the topological structure of the graph, n is the number of nodes in each time series, and d is the maximum value of the node feature dimension.

[0026] (Step 2) Generate Simulated Time Series: This paper uses a simulated infection program that incorporates node degrees to generate a simulated data sequence for each graph in the dataset. First, a node in the graph is selected. Then, based on the difference between its degree and the degrees of its adjacent nodes, the infection probability of each neighbor is generated. Based on this infection probability, the corresponding infection time and infected nodes are calculated and labeled accordingly. Repeating this process within a time limit yields a time infection sequence for a node. For each graph, we generate 1,000 time infection sequences for structural inference.

[0027] (Step 3) Structural inference: Calculate the connection probability of each edge based on the time series of each graph, and apply the threshold ξ to generate the enhanced graph A′. For each graph time series set C: (c 1 ,...,c q )∈R q×n , where q is the number of time series, n is the number of nodes in each time series, and c 1 ,...,c q For each element of the time series, the corresponding adjacency matrix M is calculated based on the set C to maximize the probability function φ(C;M). In the time window [0, T], the infection process starts from the node v i To node v j The transfer equation f(t j |t i ;M i，j )for:

[0028]

[0029] where t i For node v i The infection time, t j For node v j The infection time, t i , t j ≤T, e is the natural base, M i，j is the (i, j)th item of the adjacency matrix M. Then the node v i To node v j The transition probability equation φ() can be defined as:

[0030]

[0031] where t k is time t i to t j All moments between, all satisfy v i (t i <t j ) conditions, we can get j The probability distribution of the adjacency matrix M that satisfies the infection sequence at the current time:

[0032]

[0033] For all moments t of each time series, multiplying the corresponding probability distributions can obtain the posterior probability distribution of the adjacency matrix M at any moment T under the condition of the time series c:

[0034]

[0035] According to the probability distribution and conditional independence assumption, the probability density of the possibility adjacency matrix M at time T and the corresponding time series c are as follows:

[0036]

[0037] Since (5) only expresses the probability distribution of a single time series c, for the set C, it is necessary to solve f(C; M) = ∏ c∈C f(c;M), the final solution for M can be obtained as follows:

[0038] max M≥0 ∑ c∈C logf(c;M) (6)

[0039] After filtering the noise using the threshold ξ, M is obtained and The enhanced graph A′ is generated as the enhanced adjacency matrix of the data.

[0040] (Step 4) Construct positive and negative examples: For each sample of each label, use the sample itself and the corresponding label as the positive example, and use a sample of samples from other labels as the negative example. For each sample in the dataset, divide it into samples belonging to the current label and samples not belonging to the current label according to the label. Randomly select a sample from the set not belonging to the current label as the negative example of the current sample, and together with the original sample, form a positive and negative example pair.

[0041] (Step 5) Generate subgraph embedding: Use breadth-first search to generate corresponding subgraph partitions for each node, and apply subgraph neural network to generate subgraph embedding on the corresponding subgraph. i Applying BFS (Breath-First-Search) and searching the upper threshold β generates the corresponding subgraph g i and the corresponding node set S BFS (v i ). The process of applying the subgraph neural network on this set to generate initialized node features is as follows:

[0042]

[0043]

[0044] Among them, x is the initial feature of the node, h (0) is node v i The initialization features used for graph neural networks, AGGEREGATE is an aggregation function used to aggregate neighbor features, and COMBINE is a function that calculates neighbor features and current node features. Through formulas (7) and (8), we can calculate the initial features H of the entire graph (0) .

[0045] (Step 6) Generate graph embedding: Score and sample the subgraph embedding to construct a one-dimensional embedding that can represent the current graph data. (0) Based on this, multiple neighborhood aggregation layers are used to form a deep graph neural network, where the feature update function of each layer is as follows:

[0046]

[0047] Where l≥1 represents the layer number of the current aggregation, Represents node v i The set of neighborhood nodes of , γ is the corresponding weight parameter MLP is a multi-layer perceptron used to learn aggregation methods. After several layers of learning, the node feature representation of the entire graph can be used Indicates that d nis the node feature dimension of layer L. In order to fuse the feature representations of several nodes in the graph into a single embedding representing the entire graph, we first apply the learnable TOP k The function selects k important nodes:

[0048]

[0049] Among them, idx represents the selected node number, Represents a learnable parameter vector. Applying the following formula to the selected nodes can calculate the final graph embedding r:

[0050]

[0051]

[0052] in, is the weight matrix, d s is the number of dimensions of the supernode, is the embedding expression of the entire graph, and P is the graph label prediction for this learning.

[0053] (Step 7) Supervised Contrastive Learning: The positive and negative sample graph embeddings and corresponding supervision labels are calculated through the contrastive learning function and combined with the general classification loss to form the final classification loss. We use cross entropy loss as the general classification loss:

[0054]

[0055] Where Y is the true label information of each image. The contrastive learning loss after introducing label information can be expressed as follows:

[0056]

[0057] Where Ω = {1, ..., 2m} represents the subscripts of all samples in the current batch (including positive and negative examples), Φ(.) represents the subscripts of all positive samples in the current batch, for each positive sample i there is a corresponding negative sample p, τ is the corresponding temperature control hyperparameter, and Γ(i) is the set of all samples except sample i. By constructing positive and negative sample pairs based on labels in advance, the label information is implicit in formula (14), thereby improving the guidance quality of contrastive learning in graph classification tasks. The final loss is as follows:

[0058]

[0059] Where λ is a hyperparameter that controls the proportion of supervised contrastive learning.

[0060] (Step 8) The model is iteratively updated based on the classification loss, and after convergence, the final graph classification label P′ is used as the final graph classification model output.

[0061] To fully demonstrate the superiority of this method, we compared it with many existing graph classification methods on the MUTAG, PTC, PROTEINS, and IMDBBINARY datasets. These methods can be roughly divided into two categories: graph neural network methods and graph contrastive learning methods. Graph neural network methods include GraphSAGE, GIN, DAGCN, PPGN, CapsGNN, SGN, etc., while graph contrastive learning methods include GraphCL, InfoGraph, M-GCL, GXN, sGIN, SUGAR, etc. Classification accuracy (acc) was used as the comparison metric, and the results are shown in Table 1 as mean (%) ± standard deviation:

[0062] Table 1 Comparison of various methods on MUTAG, PTC, PROTEINS, and IMDBBINARY datasets

[0063] method MUTAG PTC PROTEINS IMDBBINARY GraphSAGE 79.8±13.9 - 65.9±2.7 72.4±3.6 GIN 89.4±5.6 64.6±7.0 76.2±2.8 75.1±5.1 DAGCN 87.2±2.05 62.9±9.6 76.3±4.3 - PPGN 90.6±8.7 66.2±6.5 77.2±4.7 73.0±5.8 CapsGNN 86.7±6.9 - 72.0±1.1 72.2±0.9 SGN 89.5±7.4 64.1±3.7 76.3±4.1 76.5±5.7 GraphCL 86.8±1.3 - 74.4±0.5 71.1±0.4 InfoGraph 89.0±1.1 61.7±1.4 - 73.0±0.9 M-GCL 89.7±1.1 62.5±1.7 - 74.2±0.7 GXN 86.1±8.3 63.5±5.8 79.9±4.1 78.6±2.3 sGIN 94.1±2.7 73.6±4.3 79.0±3.2 77.9±4.3 SUGAR 96.7±4.1 77.5±2.8 81.3±0.9 73.0±3.5 SupCosine(ours) 98.3±2.5 87.8±10.4 80.0±3.6 83.0±3.2

[0064] As shown in the table above, our model achieves higher mean accuracy than several other methods, achieving top performance on all four datasets. On the MUTAG, PTC, and IBDBBINARY datasets, our method achieves 1.0%, 10%, and 4.4% performance improvements over the next-best method, respectively, and ties for first place on the PROTEINS dataset. This significant improvement over other graph contrastive learning methods demonstrates the effectiveness of our strategy.

[0065] To further illustrate the effectiveness of each module of the proposed method, we conducted corresponding ablation experiments on four datasets. The base method (Base) used the GIN model without adding the structure inference and supervised contrastive learning parts. On this basis, the structure inference module (+StruInf) and the supervised contrastive learning part (SupGCon) were added, and the final model (SupCosine) that integrated all modules was used. The data results are shown in Figure 2 This was demonstrated in

[15] . In most cases, the structure inference module outperformed models using only the supervised contrastive learning module. Compared to Base, both modules improved graph classification performance, demonstrating their effectiveness. Furthermore, the SupCosine model, combining both modules, achieved significant performance improvements. This demonstrates that the structure inference and supervised contrastive learning modules can be well integrated with graph neural networks, complementing each other to enable graph neural networks to effectively classify graphs.

[0066] To further illustrate the effectiveness of this method, we Figure 3 The training accuracy curves of GIN and our method are shown in Figure 2. As can be seen from the figure, our method has relatively high accuracy from the outset and is quite stable during training. In contrast, the GIN method fluctuates significantly during training, and its training accuracy is close to 100% after approximately 100 epochs, indicating significant overfitting. As a result, its performance on the test set is actually inferior to our method. The relatively stable training process of our method is mainly due to the structural inference module's strengthening of latent links in the graph, highlighting the characteristics of different graphs. The supervised contrastive learning module helps our model learn the identifiability between various graphs, avoiding overfitting during training and improving the model's generalizability.

[0067] The above experiments show that the SupCosine model proposed in this invention has superior performance compared with other graph classification methods based on graph neural networks, has better adaptability to general graph classification problems, and has good scalability and application prospects.

Claims

1. A graph classification training method based on supervised contrastive learning and structural inference, characterized by: The following steps are involved: Step 1: Data acquisition: Based on the MUTAG and PTC datasets, obtain the basic data required for graph classification, namely the adjacency matrix A and node features X; Step 2: Generate simulated time series: Based on the basic data obtained in step 1, generate the corresponding simulated time series set C for the adjacency matrix A of each graph; Step 3: Structural inference: Calculate the adjacency probability of each edge based on the time series set C of each graph through posterior calculation, and apply the threshold to generate the enhanced graph. Step 4: Construct positive and negative samples: and X are divided into positive and negative samples G according to the graph labels p and G n ; Step 5: Generate subgraph embedding: for positive and negative samples G p and G n , breadth-first search is used on each node to generate the corresponding subgraph partition, and subgraph neural network is applied on the corresponding subgraph to generate subgraph embedding; Step 6: Generate graph embedding: Score and sample the subgraph embedding to construct a one-dimensional embedding that can represent the current graph data, and use this embedding to generate the corresponding graph classification label P; Step 7: Supervised contrastive learning: The image embeddings of positive and negative samples and the corresponding supervised labels are used to calculate the loss through the contrastive learning function. The image classification label P and the true label are used to calculate the general classification loss, and the two are combined to form the final classification loss. Step 8: The model is iteratively updated based on the classification loss. After convergence, the final image classification label P' is used as the final image classification model output. In step 1, the MUTAG and PTC datasets are compound molecule datasets, and each graph represents a compound molecule. For each graph in the above dataset, it is expressed as G = (V, E), where V represents the node set and E represents the edge set. For an attribute graph, it is expressed as G = (X, A) | X∈R n×d ,A∈R n×n , where X is the set of attributes of each node, A is the adjacency matrix that represents the topological structure of the graph, n is the number of nodes in each time series, and d is the maximum value of the node feature dimension.

2. A graph classification training method based on supervised contrastive learning and structure inference according to claim 1, characterized in that: In step 2, a simulation infection program that integrates node degrees is used to generate a data simulation sequence for each graph in the dataset. First, a node in the graph is selected, and then the infection probability of each neighbor is generated based on the difference between its degree and the degree of the adjacent nodes. Based on the infection probability, the corresponding infection time and infected nodes are calculated and marked accordingly. The process is repeated within a limited time to obtain a time infection sequence for a node. For each graph, 1,000 time infection sequences are generated for structural inference.

3. The graph classification training method based on supervised contrastive learning and structure inference according to claim 1, characterized in that: In step 3, the connection probability of each edge is calculated based on the time series of each graph, and the threshold ξ is applied to generate the enhanced graph A'; for each graph time series set C: (c 1 ,…,c q )∈R q×n , where q is the number of time series, n is the number of nodes in each time series, and c 1 ,…,c q For each element of the time series; calculating the corresponding adjacency matrix M according to the set C is to maximize the probability function φ(C; M); in the time window [0, T], the infection process starts from node v i To node v j The transfer equation f(t j ∣t i ;M i,j )for: where t i For node v i The infection time, t j For node v j Infection time, t i ,t j ≤T, e is the natural base, M i,j is the (i, j)th item of the adjacency matrix M; then the node v i To node v j The transition probability equation φ() can be defined as: where t k is time t i to t j All moments between, all satisfy v i (t i <t j ) conditions, we can get j The probability distribution of the adjacency matrix M that satisfies the infection sequence at the current time: For all moments t of each time series, multiplying the corresponding probability distributions can obtain the posterior probability distribution of the adjacency matrix M at any moment T under the condition of the time series c: According to the probability distribution and conditional independence assumption, the probability density of the possibility adjacency matrix M at time T and the corresponding time series c are as follows: Since (5) only expresses the probability distribution of a single time series c, for the set C, it is necessary to solve f(C; M) = ∏ c∈C f(c;M), the final solution for M is as follows: max M≥0 ∑ c∈C logf(c;M) (6) After filtering the noise using the threshold ξ, the enhanced graph M is obtained, and the enhanced graph A' is generated by A'=A⊕M as the enhanced adjacency matrix of the data.

4. The graph classification training method based on supervised contrastive learning and structure inference according to claim 1, characterized in that: In step 4, for each sample of each label, the sample itself and the corresponding label are used as positive examples, and the samples of other labels are used as negative examples; for each sample in the data set, it is divided into samples belonging to the current label and samples not belonging to the current label according to the label; a sample is randomly selected from the set not belonging to the current label as the negative example of the current sample, and together with the original sample, it constitutes a positive-negative pair.

5. The graph classification training method based on supervised contrastive learning and structure inference according to claim 1, characterized in that: Step 5) Generate subgraph embedding: Use breadth-first search to generate corresponding subgraph partitions for each node, and apply subgraph neural network to generate subgraph embedding on the corresponding subgraph; for each node v on the graph i Apply breadth-first search BFS and search upper threshold β to generate the corresponding subgraph g i and the corresponding node set S BFS (v i ); The process of applying the subgraph neural network to this set to generate initialized node features is as follows: Among them, x is the initial feature of the node, h (0) is the initialization feature of node v for graph neural network, AGGEREGATE is the aggregation function used to aggregate neighbor features, and COMBINE is the function for calculating neighbor features and current node features. Through formula (7) and formula (8), the initial feature H of the entire graph is calculated. (0) .

6. The graph classification training method based on supervised contrastive learning and structure inference according to claim 1, characterized in that: In step 6, the subgraph embedding is scored and sampled to construct a one-dimensional embedding that can represent the current graph data; (0) Based on this, multiple neighborhood aggregation layers are used to form a deep graph neural network, where the feature update function of each layer is as follows: Where l≥1 represents the layer number of the current aggregation, Represents node v i The set of neighborhood nodes, γ is the corresponding weighting parameter MLP is a multi-layer perceptron used to learn aggregation methods; after several layers of learning, the node feature representation of the entire graph can be used Indicates that d n is the node feature dimension of the Lth layer; in order to fuse the feature representations of several nodes in the graph into a single embedding representing the entire graph, we first apply the learnable TOP k The function selects k important nodes: Among them, idx represents the selected node number, Represents a learnable parameter vector; for the selected nodes, the following formula is applied to calculate the final graph embedding r: in, is the weight matrix, d s is the number of dimensions of the supernode, is the embedding expression of the entire graph, and P is the graph label prediction for this learning.

7. The graph classification training method based on supervised contrastive learning and structure inference according to claim 1, characterized in that: In step 7, the graph embeddings of positive and negative samples and the corresponding supervision labels are calculated through the contrastive learning function and combined with the general classification loss to form the final classification loss; cross entropy loss is used as the general classification loss: Where Y is the true label information of each graph; the contrastive learning loss after introducing label information is expressed as follows: Where Ω = {1,…,2m} represents the subscripts of all samples in the current batch data, Φ(.) represents the subscripts of all positive samples in the current batch data, for each positive sample i there is a corresponding negative sample p, τ is the corresponding temperature control hyperparameter, and Γ(i) is the set of all samples except sample i. By constructing positive and negative sample pairs based on labels in advance, the label information is implicit in formula (14), thereby improving the guidance quality of contrastive learning in graph classification tasks. The final loss is as follows: Where λ is a hyperparameter that controls the proportion of supervised contrastive learning.

Images