Node classification graph neural network model based on neighborhood label distribution and global label relation

Abstract

The invention discloses a node classification graph neural network model based on neighborhood label distribution and global label relation, and belongs to the technical field of complex network analysis. The method comprises the steps of 1, calculating pseudo labels; and generating a pseudo label for a node with an unknown category label in the graph by using a simulation classifier. 2, calculating initial node representation; neighborhood label distribution vectors of nodes are calculated through pseudo labels, and adaptive fusion is carried out on the neighborhood label distribution vectors and node features to obtain initial node representation. 3, recovering a global label relation matrix; and recovering the global label relation matrix by utilizing the low-rank characteristic of the global label relation matrix and combining the attention score between the initial node representations. And step 4, dividing channels for the recovered global label relation matrix, and carrying out multi-channel message transmission and aggregation to obtain final node representation for node classification. According to the method, the problem of disparity and the problem of long-distance dependence in node classification are solved based on neighborhood label distribution and the global label relation, information of node neighborhoods in a graph is effectively utilized, and the performance of a model in node classification is effectively improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of complex network analysis, and in particular relates to a node classification graph neural network model based on neighborhood label distribution and global node relations. Background Art

[0002] Graph neural networks (GNNs) have been widely used in many graph-related research fields, such as node classification, link prediction, and graph classification, due to their superior performance in processing unstructured data. Graph neural networks based on neighborhood message passing, such as graph convolutional neural networks (GCNs), have better performance than graph structure-independent models such as multi-layer perceptrons (MLPs) by iteratively aggregating the features of neighboring nodes. However, GNN models based on neighborhood message passing usually rely on the implicit assortativity assumption that nodes with the same label tend to be connected to each other, and the resulting node representations show better performance in downstream tasks. Despite this, many graphs in the real world often exhibit heteroassortativity, that is, nodes with different labels are often connected together. For example, in dating networks, most people tend to communicate with the opposite sex; in online trading networks, fraudsters tend to establish connections with ordinary customers rather than other fraudsters; in protein networks, different types of amino acids are more likely to be connected to each other. In these heteroassortative graphs, most of the nodes in a node's neighborhood are non-same nodes. After message passing and aggregation by the GNN model, the node representation may be confused by messages from nodes of different categories in the neighborhood, resulting in the inability to accurately classify the nodes on the heterogeneous graph.

[0003] One problem that affects the performance of node classification in heterogeneous graphs is long-distance dependency. Specifically, in heterogeneous graphs, nodes that belong to the same category as the target node are usually not near the node, but may be far away from the target node. In this case, the expressive power of traditional GNN models based on neighborhood message passing may be greatly limited because they cannot capture important features from nodes that are far away but rich in information. Therefore, how to determine the relationship between nodes in a larger range is also the key to improving the performance of GNN models on heterogeneous graphs. In addition, the topological structure of heterogeneous graphs also determines the node representation to a certain extent. However, existing related methods believe that the original topological structure of heterogeneous graphs will affect the performance of node classification, and they often aggregate more messages from similar nodes by changing the neighborhood structure of the nodes. However, this approach not only aggregates more messages from similar nodes, but also loses some of the information contained in the original topological structure. Summary of the invention

[0004] In order to effectively solve the above problems, the present invention first analyzes the reasons why the traditional graph isomorphism index cannot correctly reflect the node classification performance of the GNN model, and verifies that the node neighborhood label distribution discrimination is the key factor affecting the node classification performance. On this basis, this study proposes a new node classification graph neural network model NLDGNN based on neighborhood label distribution and global label relationship, which first obtains the initial representation of the node by fusing node features and neighborhood label distribution. At the same time, in order to effectively solve the long-distance dependency problem of heterogeneous graphs, this study uses the low-rank characteristics of the global label relationship matrix to approximate the problem as a low-rank matrix completion problem. Specifically, NLDGNN uses a low-rank matrix completion method, combined with the initial node representation for completion, and realizes the long-distance transmission of node messages based on the matrix, thereby obtaining a node representation with stronger expressive power.

[0005] In order to achieve the above object, the technical solution adopted by the present invention is:

[0006] A node classification graph neural network model based on neighborhood label distribution and global node relationship includes the following steps:

[0007] Step 1: Calculate pseudo labels; use the simulated classifier to generate pseudo labels for nodes in the graph whose class labels are unknown, that is:

[0008]

[0009] where f NN is a neural network model for classification, such as MLP or GCN. O is the mask of the training set, and ⊙ is the Hadamard product.

[0010] Step 2: Calculate the initial node representation; calculate the neighborhood label distribution vector of the node through pseudo-labels, and adaptively fuse it with the node features to obtain the initial node representation.

[0011] Step 3: Restore the global label relationship matrix; utilize the low-rank property of the global label relationship matrix and combine the attention scores between the initial node representations to restore the global label relationship matrix.

[0012] Step 4: Divide the restored global label relationship matrix into channels, perform multi-channel message passing and aggregation, and obtain the final node representation for node classification.

[0013] In step 2, the process of calculating the initial node representation includes:

[0014] Step 2.1: Calculate the neighborhood distribution vector of all nodes using the pseudo labels of the nodes:

[0015]

[0016] Step 2.2: After obtaining the neighborhood label distribution vector of the node, combine the node features and the adjacency matrix to calculate the initial node representation:

[0017] H=(1-μ)MLP X (X)+μMLP A (A)+wMLP N (NLD)

[0018] In step 3, the process of restoring the global label relationship matrix in combination with the attention scores represented by the nodes includes:

[0019] Step 3.1: First, use the node pseudo labels to calculate the initial global label relationship matrix, that is:

[0020]

[0021] in is the pseudo label of node i, Represents the dot product of the pseudo labels of node i and node j. Since the pseudo labels are represented by one-hot vectors, if the two nodes belong to the same category, the dot product is 1, otherwise it is 0. 0<δ<1 is a parameter that controls the ratio of weights.

[0022] Step 3.2: Use an improved low-rank matrix completion method to restore the global label relationship matrix using the initial label relationship matrix calculated from the pseudo-labels and the initial node representation. Specifically, this method is based on the maximum margin matrix recovery method and combines the attention scores calculated from the initial node representation to further fit the matrix. Finally, the matrix Z is approximately restored by solving the following low-rank matrix decomposition problem. Its objective function is as follows:

[0023]

[0024] In order to more effectively determine the relationship between nodes, this formula introduces a fitting term based on node representation attention in the low-rank matrix completion process, where a represents the attention vector. Since the node representation combines the node features and the neighborhood label distribution vector, fitting with this node representation not only helps to identify nodes with similar features at a long distance, but also can determine nodes with similar neighborhood label distributions.

[0025] Step 3.3: For this objective function, since Contains the projection function, which cannot be solved directly. To solve this problem, a proxy function is constructed here. (k) and V (k) Get a new U (k+1) .

[0026]

[0027] If (i,j)∈Ω,M i,j =1, otherwise M i,j = 0. For this objective function, the following minimization problem can be solved:

[0028] Finally get U (k+1) The optimization formula is as follows:

[0029]

[0030] Similarly, using U (k+1) and V (k) You can get V (k+1) Through alternating optimization, the global label relationship matrix is ​​finally restored as follows:

[0031] Z=UV T

[0032] In step 4, the restored global label relationship matrix is ​​divided into channels according to the relationship between nodes, that is:

[0033] Z pos =Z(Where Z ij >0)

[0034] Z neg = Z(where Z ij <0)

[0035] Since the global label relationship matrix is ​​used to guide the aggregation process, the scope of aggregation has been expanded to the global scope, so there is no need to expand the scope of aggregation by stacking multiple layers. Finally, the final node representation is obtained by the following formula:

[0036] H out =(αI+βZ pos +ηZ neg )H

[0037] where α, β and η are learnable parameters, H out The node representation vector for the final output. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] Figure 1 This is the generation of the neighborhood label distribution vector and the calculation process of the initial node representation of the present invention.

[0039] Figure 2 This is the process of low-rank matrix completion and partition channel aggregation combining attention scores in the present invention.

[0040] Figure 3 This is an example of the discrimination of neighborhood label distribution in the present invention. DETAILED DESCRIPTION

[0041] The present invention extracts the information of the node neighborhood through the neighborhood label distribution, and solves the long-distance dependency problem existing in the heterogeneous graph by completing the node relationship matrix, so as to realize the effective processing of the heterogeneous graph. In the real world, there are often cases of heterogeneous graphs, such as in dating networks, most people tend to communicate with the opposite sex; in online trading networks, fraudsters tend to establish connections with ordinary customers rather than other fraudsters; in protein networks, different types of amino acids are more easily connected to each other. Different categories of nodes usually have different neighborhood label distributions in the network, and a better neighborhood label distribution can help us better distinguish different categories of nodes. Therefore, solving the node classification problem in the heterogeneous graph can help the model better process the graph in the real world.

[0042] In the implementation of the present invention, the specific implementation process of the technical solution is clearly described by means of diagrams. The diagrams include the process of generating node pseudo labels, calculating node neighborhood label distribution vectors, fusing node features to generate initial node representations, completing low-rank matrices, and dividing channel aggregation. The diagrams can help readers better understand the technical implementation details of the present invention and obtain clear operational guidance from them. It should be clear that the technical solutions described in the embodiments of the present invention only represent some embodiments, not all embodiments. All other embodiments that can be obtained by ordinary technicians in this field without performing creative work are within the scope of protection of the present invention.

[0043] Therefore, the present invention, through detailed description and illustration, shows the technical personnel a node classification graph neural network model based on neighborhood label distribution and global node relationship. This method uses the neighborhood label distribution vector of the node, and completes the node label relationship through a low-rank matrix completion method combined with attention score, and finally obtains a more expressive node representation, achieving a more accurate node classification performance.

[0044] The present invention will be further described in detail below in conjunction with the accompanying drawings.

[0045] A node classification graph neural network model based on neighborhood label distribution and global node relationship includes the following steps:

[0046] Step 1: Calculate pseudo labels; use the simulated classifier to generate pseudo labels for nodes in the graph whose class labels are unknown, that is:

[0047]

[0048] where f NNis a neural network model for classification, such as MLP or GCN. O is the mask of the training set, and ⊙ is the Hadamard product.

[0049] Step 2: Calculate the initial node representation; calculate the neighborhood label distribution vector of the node through pseudo-labels, and adaptively fuse it with the node features to obtain the initial node representation.

[0050] Step 2.1: Calculate the neighborhood distribution vector of all nodes using the pseudo labels of the nodes:

[0051]

[0052] Step 2.2: After obtaining the neighborhood label distribution vector of the node, combine the node features and the adjacency matrix to calculate the initial node representation:

[0053] H=(1-μ)MLP X (X)+μMLP A (A)+wMLP N (NLD)

[0054] Step 3: Restore the global label relationship matrix; use the low-rank property of the global label relationship matrix and the attention scores between the initial node representations to restore the global label relationship matrix. The process of restoring the global label relationship matrix includes:

[0055] Step 3.1: First, use the node pseudo labels to calculate the initial global label relationship matrix, that is:

[0056]

[0057] in is the pseudo label of node i, Represents the dot product of the pseudo labels of two nodes. Since the pseudo labels are represented by one-hot vectors, if the two nodes belong to the same category, the dot product is 1, otherwise it is 0. 0<δ<1 is a parameter that controls the weight ratio.

[0058] Step 3.2: Use an improved low-rank matrix completion method to recover the global label relationship matrix using the calculated matrix and the initial node representation. Specifically, this method is based on the maximum margin matrix recovery method and combines the attention scores calculated from the initial node representation to further fit the matrix. Finally, the matrix Z is approximately recovered by solving the following low-rank matrix decomposition problem. Its objective function is as follows:

[0059]

[0060] In order to more effectively determine the relationship between nodes, this formula introduces a fitting term based on node representation attention in the low-rank matrix completion process, where a represents the attention vector. Since the node representation combines the node features and the neighborhood label distribution vector, fitting with this node representation not only helps to identify nodes that are far away but have similar features, but also can determine nodes with similar neighborhood label distributions.

[0061] Step 3.3: For this objective function, since contains a projection function, which cannot be solved directly. To solve this problem, a proxy function is constructed here. (k) and V (k) Get a new U (k+1) .

[0062]

[0063] If (i,j)∈Ω,M i,j =1, otherwise M i,j = 0. For this objective function, the following minimization problem can be solved:

[0064] Finally get U (k+1) The optimization formula is as follows:

[0065]

[0066] Similarly, using U (k+1) and V (k) You can get V (k+1) Through alternating optimization, the global label relationship matrix is ​​finally restored as follows:

[0067] Z=UV T

[0068] Step 4: Divide the restored global label relationship matrix into channels, perform multi-channel message transmission and aggregation, obtain the final node representation and use it for node classification. According to the relationship between nodes, divide the restored global label relationship matrix into channels, that is:

[0069] Z pos = Z(where Z ij >0)

[0070] Z neg = Z(where Z ij <0)

[0071] Since the global label relationship matrix is ​​used for aggregation, the aggregation scope has been expanded to the global scope, so there is no need to expand the aggregation scope by stacking multiple layers. Finally, the final node representation is obtained by the following formula:

[0072] H out =(αI+βZ pos +ηZ neg )H

[0073] where α, β and η are learnable parameters, H out The node representation vector for the final output.

Claims

1. A node classification graph neural network model based on neighborhood label distribution and global label relationship, characterized in that: The steps include: Step 1: Calculate pseudo labels; use the simulated classifier to generate pseudo labels for nodes in the graph whose class labels are unknown, that is: Among them, f NN is a neural network model for classification, such as a multi-layer perceptron (MLP) or a graph convolutional neural network (GCN). O is the mask of the training set, and ⊙ is the Hadamard product. Step 2: Calculate the initial node representation; calculate the neighborhood label distribution vector of the node through pseudo-labels, and adaptively fuse it with the node features to obtain the initial node representation. Step 3: Restore the global label relationship matrix; utilize the low-rank property of the global label relationship matrix and combine the attention scores between the initial node representations to restore the global label relationship matrix. Step 4: Divide the restored global label relationship matrix into channels, perform multi-channel message passing and aggregation, and obtain the final node representation for node classification. In step 2, the process of calculating the initial node representation includes: Step 2.1: Calculate the neighborhood distribution vector of all nodes using the pseudo labels of the nodes: Step 2.2: After obtaining the neighborhood label distribution vector of the node, combine the node features and the adjacency matrix to calculate the initial node representation: H=(1-μ)MLP X (X)+μMLP A (A)+inMLP N (NLD) In step 3, the process of restoring the global label relationship matrix in combination with the attention scores represented by the nodes includes: Step 3.1: First, use the node pseudo labels to calculate the initial global label relationship matrix, that is: in is the pseudo label of node i, Represents the dot product of the pseudo labels of two nodes. Since the pseudo labels are represented by one-hot vectors, if the two nodes belong to the same category, the dot product is 1, otherwise it is 0. 0<δ<1 is a parameter that controls the weight ratio. Step 3.2: Use an improved low-rank matrix completion method to restore the global label relationship matrix using the calculated matrix and the initial node representation. Specifically, this method is based on MMMF and combines the attention scores calculated by the initial node representation to further fit the matrix. Finally, the matrix Z is approximately recovered by solving the following low-rank matrix decomposition problem. Its objective function is as follows: In order to more effectively determine the relationship between nodes, the formula introduces a fitting term based on node representation attention in the low-rank matrix completion process, where a represents the attention vector. Since the node representation combines the node features and the neighborhood label distribution vector, fitting with node representation not only helps to identify nodes with similar features but also can determine nodes with similar neighborhood label distribution. Step 3.3: Since the objective function contains a projection function, it cannot be solved directly. To solve this problem, a proxy function is constructed here. (k) and V (k) Get a new U (k +1) . where if (i,j)∈Ω,M i,j =1, otherwise M i,j = 0. For this objective function, the following minimization problem can be solved: Finally get U (k+1) The optimization formula is as follows: Similarly, using U (k+1) and V (k) You can get V (k+1) Through alternating optimization, the global label relationship matrix is ​​finally restored as follows: Z=UV T In step 4, the restored global label relationship matrix is ​​divided into channels according to the relationship between nodes, that is: Z pos =Z(where Z ij >0) Z neg =Z(where Z ij <0) Since the global label relationship matrix is ​​used for aggregation, the aggregation scope has been expanded to the global scope, so there is no need to expand the aggregation scope by stacking multiple layers. Finally, the final node representation is obtained by the following formula: H out =(αI+βZ pos +ηZ neg )H where α, β and η are learnable parameters, H out The node representation vector for the final output.

Images