A graph representation learning method and system based on topological structure preservation

Abstract

The present invention discloses a graph representation learning method and system based on topology preservation, which can integrate topology and semantic features to jointly improve the quality of graph representation learning under the premise of maintaining the topology invariance of graph data; the method includes: designing a self-supervised task of topology preservation according to the characteristics of graph data; inputting graph data, using a graph convolutional neural network to perform feature encoding on it, so as to learn the initial vector representation of the node; inputting the learned node initial vector representation into a TGSSL (Topology-preserving Graph Self-Supervised Learning) model for graph self-supervised learning, and finally obtaining a high-quality node vector representation on the basis of structure preservation. The present invention can effectively solve the problem that the existing graph representation learning method cannot effectively integrate structural information when learning node vectors.

Description

Technical Field

[0001] The present invention relates to the field of graph data and representation learning, and in particular to a graph representation learning method and system based on topological structure preservation. Background Art

[0002] When mapping graph data to a low-dimensional vector space, how to make the vector preserve the semantic and structural information of the nodes on the graph as much as possible is the key to graph representation learning research. Graph data generally has structural characteristics such as community and hierarchy, and the structure of graph data is crucial to graph reasoning. However, existing graph representation learning methods such as graph neural networks pay more attention to retaining the semantic features of nodes in the graph and the information of low-order neighbor nodes, while losing the higher-order proximity between nodes in the graph and other types of graph structural characteristics. Therefore, when performing graph representation learning on graph data, it is necessary to integrate the topological structure and node features to further improve the quality of representation learning while maintaining its topological invariance. Summary of the invention

[0003] The main purpose of the present invention is to overcome the problem that the existing graph data representation learning methods cannot effectively integrate structural information when learning node vectors, and propose a graph representation learning method and system based on topology preservation, which adopts a topology-preserving graph self-supervised learning model (Topology-preserving Graph Self-Supervised Learning, TGSSL) to learn node vectors that effectively integrate topological structures and semantic features.

[0004] The present invention adopts the following technical solution:

[0005] On the one hand, a graph representation learning method based on topology preservation includes the following steps:

[0006] Step 1: Design a self-supervisory task that preserves the topology structure based on the characteristics of graph data.

[0007] Step 2: Input graph data and use graph convolutional neural network to encode its features, so as to learn the initial vector representation of the node;

[0008] Step 3: Input the learned node initial vector representation into the TGSSL model for graph self-supervised learning, and finally obtain high-quality node vector representation on the basis of structure preservation.

[0009] Preferably, the step 1 specifically includes:

[0010] Step 1.1: Design the graph topology partitioning self-supervision task based on the structure of graph data. Specifically, use the connection density of the edges in the graph data to partition the graph, and then predict the partition index to which the node belongs.

[0011] Step 1.2: Design the mask node self-supervision task based on the structure of graph data. Specifically, randomly mask the features of some nodes and then reconstruct the masked nodes according to the features of their neighbor nodes.

[0012] Preferably, the step 1.1 specifically includes:

[0013] Step 1.1.1: Use the graph partitioning algorithm METIS to partition the graph data into K∈{1,2,…,|V|} communities and output a node set {V p_1 ,…,V p_k ,…,V p_K |V p_k ∈V,k=1,…,K}; where |V| is the number of nodes in the graph data,

[0014] Step 1.1.2: Use the node partition index as the pseudo label y for the graph topology partitioning task TP For TGSSL model to learn; where the partition index of the nth node is k, which can be formally expressed as: y TP_n =k,if n ∈V p_k ,n=1,…,|V|, …,K;

[0015] Step 1.1.3: Based on pseudo label y TP Define the graph topology partitioning task loss function L TP is the multi-classification cross entropy:

[0016]

[0017] Among them, N is the total number of node samples divided by the graph topology, is the predicted value of the nth node for the kth community.

[0018] Preferably, the step 1.2 specifically includes:

[0019] Step 1.2.1: In the graph data, randomly mask|M a | node features x; where,

[0020] Step 1.2.2: Use the symptom node feature vector x before the mask operation in step 1.2.1 as the pseudo label y of the mask node task MN For TGSSL model to learn;

[0021] Step 1.2.3, based on pseudo label y MNDefine the mask node task loss function L MN is the mean absolute error:

[0022]

[0023] in, For node v i Node vector representation learned by the TGSSL model.

[0024] Preferably, the step 2 specifically includes:

[0025] Input graph data node initial vector matrix X and normalized adjacency matrix Use a two-layer graph convolutional neural network to learn graph representations, thereby learning the initial vector representation H of the graph data nodes;

[0026]

[0027] in, For regularized is the adjacency matrix with self-loops added, I is the identity matrix, yes The degree matrix, W 1 is the training parameter matrix of the first convolution.

[0028] Preferably, the step 3 specifically includes:

[0029] Step 3.1: Calculate the loss L of the graph topology partitioning task in the TGSSL model TP ;

[0030] Step 3.2: Calculate the loss L of the mask node task in the TGSSL model MN ;

[0031] Step 3.3: Calculate the multi-classification cross entropy loss L for downstream tasks in the TGSSL model main :

[0032]

[0033] Among them, I and C are the total number of samples and the number of node categories in the downstream task of node classification, respectively. is the predicted value of the i-th sample being of the c-th class.

[0034] Step 3.4: Update network parameters W according to the total loss L 1 Until the maximum number of iterations T is reached, a high-quality node vector representation Z and a trained TGSSL model based on structure preservation can be obtained, as follows:

[0035] L=λ1Lmain +λ2L TP +λ3L MN (5)

[0036] Among them, λ1, λ2, They are the weights of downstream tasks, graph topology partitioning tasks, and mask node tasks respectively.

[0037] On the other hand, a graph representation learning system based on topology preservation includes:

[0038] The self-supervised task design module is used to design self-supervised tasks that preserve the topology structure based on the characteristics of graph data;

[0039] The node initial vector representation learning module is used to input graph data and use graph convolutional neural network to encode its features, so as to learn the node initial vector representation;

[0040] The node vector representation acquisition module is used to input the learned node initial vector representation into the TGSSL model for graph self-supervised learning, and finally obtain high-quality node vector representation on the basis of structure preservation.

[0041] It can be seen from the above description of the present invention that, compared with the prior art, the present invention has the following beneficial effects:

[0042] According to the structural characteristics of graph data, the self-supervision tasks of graph topology partitioning and mask nodes designed in this paper can maintain the topological structure of graph data, and integrate these structural information into the message passing mechanism of graph convolutional neural network, so as to improve the quality of graph representation learning. On the one hand, it can provide a solution for graph representation learning with topological structure preservation, and on the other hand, the learned high-quality node vectors can serve graph reasoning tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 A flowchart of a graph representation learning method based on topology structure preservation of the present invention;

[0044] Figure 2 A research framework diagram for graph representation learning that preserves the topological structure of graph data;

[0045] Figure 3 This is a pseudo code diagram of the TGSSL algorithm;

[0046] Figure 4 It is a structural block diagram of the graph representation learning system based on topology structure preservation of the present invention. DETAILED DESCRIPTION

[0047] The present invention will be further described below in conjunction with specific embodiments. It should be understood that these embodiments are only used to illustrate the present invention and are not intended to limit the scope of the present invention. In addition, it should be understood that after reading the content taught by the present invention, those skilled in the art can make various changes or modifications to the present invention, and these equivalent forms fall within the scope limited by the appended claims of the application equally.

[0048] See also Figures 1 to 3 As shown, the graph representation learning method based on topological structure preservation of the present invention comprises the following steps:

[0049] Step 1: Design a self-supervisory task that preserves the topology structure based on the characteristics of graph data.

[0050] Step 2: Input graph data and use graph convolutional neural network to encode its features, so as to learn the initial vector representation of the node;

[0051] Step 3: Input the learned node initial vector representation into the TGSSL model for graph self-supervised learning, and finally obtain high-quality node vector representation on the basis of structure preservation.

[0052] The node classification task on the public citation network PubMed dataset is used as an example. The dataset is a graph consisting of medical papers and their citation relationships. The nodes of the graph are papers, and the edges in the graph represent the citation relationships between papers. The feature dimension of the node is 500, and the label of the node is the category to which the paper belongs. The statistical information of the dataset is shown in Table 1.

[0053] Table 1 Citation network graph data statistics

[0054]

[0055] The step 1 specifically includes:

[0056] Step 1.1: Design the graph topology partitioning self-supervision task based on the structure of graph data. Specifically, use the connection density of the edges in the graph data to partition the graph, and then predict the partition index to which the node belongs.

[0057] Step 1.2: Design a mask node self-supervision task based on the structure of graph data. Specifically, the mask node is reconstructed based on the features of its neighbor nodes by randomly masking the features of some nodes.

[0058] Furthermore, the step 1.1 specifically includes:

[0059] Step 1.1.1: Use the graph partitioning algorithm METIS to partition the graph data into k∈{1,2,…,|V|} communities and output a node set {V p_1 ,…,V p_k,…,V p_K |V p_k ∈V,k=1,…,K}. Here, the number of nodes in the graph data |V|=19717; the number of partitions K=16;

[0060] Step 1.1.2: Use the node partition index as the pseudo label y for the graph topology partitioning task TP For TGSSL model to learn. Among them, the partition index of the nth node is k, which can be formally expressed as: y TP_n =k,if v n ∈V p_k ,n=1,…,|V|, …,K;

[0061] Step 1.1.3: Based on pseudo label y TP Define the graph topology partitioning task loss function L TP is the multi-classification cross entropy:

[0062]

[0063] Among them, N is the total number of node samples divided by the graph topology, is the predicted value of the nth node for the kth community.

[0064] Furthermore, the step 1.2 specifically includes:

[0065] Step 1.2.1: In the graph data, randomly The feature x of each node;

[0066] Step 1.2.2: Use the symptom node feature vector x before the mask operation in step 1.2.1 as the pseudo label y of the mask node task MN For the TGSSL model to learn; in the experiment, since the initial vector dimension of the node is large, the singular value decomposition method is used to reduce the dimension of the initial vector to 28, and then the reduced node vector is used as the pseudo label of the self-supervised task;

[0067] Step 1.2.3, based on pseudo label y MN Define the mask node task loss function L MN is the mean absolute error:

[0068]

[0069] in, For node v i Node vector representation learned by the TGSSL model.

[0070] The step 2 specifically includes:

[0071] Step 2: Input the node initial vector matrix X and normalized adjacency matrix of the graph data Use a two-layer graph convolutional neural network to learn graph representations, thereby learning the initial vector representation H of the graph data nodes;

[0072]

[0073] in, For regularized is the adjacency matrix with self-loops added, I is the identity matrix, yes The degree matrix, W 1 is the training parameter matrix of the first convolution.

[0074] The step 3 specifically includes:

[0075] Step 3.1: Calculate the loss L of the graph topology partitioning task in the TGSSL model TP (Formula 1);

[0076] Step 3.2: Calculate the loss L of the mask node task in the TGSSL model MN (Formula 2);

[0077] Step 3.3: Calculate the multi-classification cross entropy loss L for downstream tasks in the TGSSL model main :

[0078]

[0079] Among them, I and C are the total number of samples and the number of node categories in the downstream task of node classification, respectively. is the predicted value of the i-th sample being of the c-th class.

[0080] Step 3.4: Update the network parameters W according to the total loss L (Formula 5) 1 Until the maximum number of iterations T=10000 is reached, a high-quality node vector representation Z and a trained TGSSL model based on structure preservation can be finally obtained.

[0081] L=λ1L main +λ2L TP +λ3L MN (5)

[0082] Among them, λ1, λ2, are the weights of downstream tasks, graph topology partitioning tasks, and mask node tasks, respectively. The specific settings of the TGSSL model in the experiment are: the learning rate is set to 0.01, the dropout is set to 0.5, the number of hidden layer neurons is set to 16, and the L2 regularization weight is set to 5×10 -4 .

[0083] Step 3.5: Based on the trained TGSSL model and the learned high-quality node vector representation Z, the prediction results of the downstream tasks can be output.

[0084] In the experiment, the node classification test set is sent to the trained TGSSL model, and the label prediction results of the node classification are output. Accuracy is used as the evaluation index of node classification. In addition, in order to verify the effectiveness of the TGSSL model, the two types of models in Table 2 are used as baselines for comparison. The first type is the existing related models, and the second type is the TGSSL ablation experimental model Graph Convolutional Network (GCN), the graph self-supervised learning model TP_GCN based only on graph topology partitioning, and the graph self-supervised model MN_GCN based only on mask nodes to verify the effectiveness of the TGSSL model. The experimental results are shown in Table 3.

[0085] Table 2 Baseline model settings

[0086]

[0087]

[0088] Table 3 Node classification accuracy of the model on the PubMed citation network dataset (%)

[0089]

[0090] At this point, a graph representation learning method based on topological structure preservation is completely completed. It is not difficult to find that the present invention designs the self-supervision tasks of graph topology partitioning and mask nodes according to the structural characteristics of graph data, and integrates the structural information into the message passing mechanism of the graph convolutional neural network under the premise of maintaining the topological structure, thereby improving the quality of graph representation learning and effectively serving graph data reasoning tasks such as node classification.

[0091] See also Figure 4 As shown, a graph representation learning system based on topological structure preservation includes:

[0092] A self-supervised task design module 401, used to design a self-supervised task that preserves the topology structure according to the characteristics of the graph data;

[0093] The node initial vector representation learning module 402 is used to input graph data and perform feature encoding on the graph data using a graph convolutional neural network, thereby learning the node initial vector representation;

[0094] The node vector representation acquisition module 403 is used to input the learned node initial vector representation into the TGSSL model for graph self-supervised learning, and finally obtain high-quality node vector representation on the basis of structure preservation.

[0095] A specific implementation of a graph representation learning system based on topological structure preservation. The present invention will not repeat the same graph representation learning method based on topological structure preservation.

[0096] The above is only a specific implementation of the present invention, but the design concept of the present invention is not limited thereto. Any non-substantial changes to the present invention using this concept shall be deemed as an infringement of the protection scope of the present invention.

Claims

1. A graph representation learning method based on topological structure preservation, characterized in that: The steps include: Step 1: Design a self-supervisory task that preserves the topology structure based on the characteristics of graph data. Step 2: Input graph data and use graph convolutional neural network to encode its features, so as to learn the initial vector representation of the node; Step 3: Input the learned node initial vector representation into the TGSSL model for graph self-supervised learning, and finally obtain high-quality node vector representation on the basis of structure preservation; The step 1 specifically includes: Step 1.1: Design the graph topology partitioning self-supervision task based on the structure of graph data. Specifically, use the connection density of the edges in the graph data to partition the graph, and then predict the partition index to which the node belongs. Step 1.2: Design the mask node self-supervision task based on the structure of graph data. Specifically, randomly mask the features of some nodes and then reconstruct the masked nodes according to the features of their neighbor nodes.

2. A graph representation learning method based on topological structure preservation according to claim 1, characterized in that: The step 1.1 specifically includes: Step 1.1.1: Use the graph partitioning algorithm METIS to partition the graph data into K∈{1,2,…,|V|} communities and output a node set {V p_1 ,…,V p_k ,…,V p_K |V p_k ∈V,k=1,…,K}; where |V| is the number of nodes in the graph data, Step 1.1.2: Use the node partition index as the pseudo label y for the graph topology partitioning task TP For TGSSL model to learn; where the partition index of the nth node is k, which can be formally expressed as: Step 1.1.3: Based on pseudo label y TP Define the graph topology partitioning task loss function L TP is the multi-classification cross entropy: Among them, N is the total number of node samples divided by the graph topology, is the predicted value of the nth node for the kth community.

3. A graph representation learning method based on topological structure preservation according to claim 2, characterized in that: The step 1.2 specifically includes: Step 1.2.1: In the graph data, randomly mask|M a | node features x; where, Step 1.2.2: Use the symptom node feature vector x before the mask operation in step 1.2.1 as the pseudo label y of the mask node task MN For TGSSL model to learn; Step 1.2.3, based on pseudo label y MN Define the mask node task loss function L MN is the mean absolute error: in, For node v i Node vector representation learned by the TGSSL model.

4. A graph representation learning method based on topological structure preservation according to claim 3, characterized in that: The step 2 specifically includes: Input graph data node initial vector matrix X and normalized adjacency matrix Use a two-layer graph convolutional neural network to learn graph representations, thereby learning the initial vector representation H of the graph data nodes; in, For regularized is the adjacency matrix with self-loops added, I is the identity matrix, yes The degree matrix, W 1 is the training parameter matrix of the first convolution.

5. A graph representation learning method based on topological structure preservation according to claim 4, characterized in that: The step 3 specifically includes: Step 3.1: Calculate the loss L of the graph topology partitioning task in the TGSSL model TP ; Step 3.2: Calculate the loss L of the mask node task in the TGSSL model MN ; Step 3.3: Calculate the multi-classification cross entropy loss L for downstream tasks in the TGSSL model main : Among them, I and C are the total number of samples and the number of node categories in the downstream task of node classification, respectively. is the predicted value of the i-th sample being the c-th class; Step 3.4: Update network parameters W according to the total loss L 1 Until the maximum number of iterations T is reached, a high-quality node vector representation Z and a trained TGSSL model based on structure preservation can be obtained, as follows: L=λ1L main +λ2L TP +λ3L MN (5) in, They are the weights of downstream tasks, graph topology partitioning tasks, and mask node tasks respectively.

6. A graph representation learning system based on topological structure preservation, characterized in that: include: The self-supervised task design module is used to design self-supervised tasks that preserve the topology structure based on the characteristics of graph data; The node initial vector representation learning module is used to input graph data and use graph convolutional neural network to encode its features, so as to learn the node initial vector representation; The node vector representation acquisition module is used to input the learned node initial vector representation into the TGSSL model for graph self-supervised learning, and finally obtain high-quality node vector representation on the basis of structure preservation; The self-supervisory task design module is used to design a self-supervisory task that maintains the topology structure according to the characteristics of the graph data, and specifically includes: Step 1.1: Design the graph topology partitioning self-supervision task based on the structure of graph data. Specifically, use the connection density of the edges in the graph data to partition the graph, and then predict the partition index to which the node belongs. Step 1.2: Design the mask node self-supervision task based on the structure of graph data. Specifically, randomly mask the features of some nodes and then reconstruct the masked nodes according to the features of their neighbor nodes.

Images