Undirected graph semi-supervised node classification method based on hierarchical conditional random field
By adopting a hierarchical conditional random field method in graph data processing, nodes with similar structures are aggregated into supernodes, and improved conditional random field is added to each layer operation, it is solved that it is difficult to utilize graph structure and capture in the prior art. The problem of global information significantly improves the accuracy and model stability of semi-supervised node classification.
Patent Information
- Application Number
- CN202410933693.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-12
- Publication Date
- 2024-11-01
AI Technical Summary
Existing graph data processing methods are difficult to simultaneously utilize the structural properties of graphs and capture global information in higher dimensions and deeper layers. In addition, the hidden layer representation of graph neural networks lacks constraints during iterative propagation, resulting in node characteristics becoming consistent and the node's cannot be maintained. Special information.
Using the undirected graph semi-supervised node classification method based on hierarchical conditional random fields, through multi-layer graph coarse and refinement operations, nodes with similar structures are aggregated into supernodes, and improved conditional random fields are added to each layer operation. , retain the similarity relationship and location information of nodes, and use shortcut connections to restore the original graph topology.
It significantly improves the accuracy of semi-supervised node classification, can better retain the unique information and structural properties of the node, and improves the stability and performance of the model.
Smart Images

Figure CN118885652A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of graph data processing, and in particular relates to an undirected graph semi-supervised node classification method based on hierarchical conditional random fields. Background Art
[0002] Graphs can express objects and their relationships with nodes and edges. Real-world data such as social networks, molecular structures, citation networks, and urban traffic can all be abstractly represented in the form of graphs. Graph neural networks and probabilistic graph models are two different powerful tools for learning graph representations, and both have been successfully developed to solve practical problems in graph structures. Graph neural networks can process graph data well, but they cannot fully utilize the properties of graphs and have difficulty expressing similar relationships between nodes. Probabilistic graph models are mathematically and statistically interpretable models that can express conditional dependency structures between random variables. These two methods have their own unique advantages, and the similarities allow the two models to be merged together to obtain more efficient and effective algorithms. Graph neural networks achieve structured representation and prediction through conditional random fields. This is because most graph neural networks hold a strong assumption that node representations and labels are independent when observing node features and edges, while probabilistic graph models always try to model the relationship between input data and output representations. Conditional random fields allow graph neural networks to learn from the joint dependencies of node hidden representations to preserve spatial structural information in learning so that hidden features can explicitly preserve similar relationships in local neighborhoods. However, previous work has found it difficult to simultaneously exploit the structural properties of graphs and capture higher-dimensional and deeper global information.
[0003] The methods for processing graph data are essentially based on neighbor aggregation mechanisms and can be divided into two categories: spatial domain methods and frequency domain methods. Spatial domain methods generate the embedding of a node by aggregating the neighbor information of a vertex. MoNet uses a range patch operation to integrate local signals. GraphSAGE samples a fixed number of neighbors and then uses multiple aggregation mechanisms. et al. proposed a self-attention mechanism in graph data and measured the importance of different neighbors. Gao et al. pooled the graph to expand the node receptive field and obtain deeper graph topological structure features. However, these methods lack the use of local structural information and the constraints of hidden layer representation during the iterative propagation of neural networks. The frequency domain method is based on graph spectrum theory, finds the "convolution kernel", and performs convolution in the Fourier domain. Kipf et al. proposed a convolutional graph neural network (GCN), which can extract the node features and structure in the graph by using message passing and aggregation mechanisms. However, the message passing mechanism has a disadvantage: after multiple layers of iteration, the node features of the entire graph will tend to be consistent, that is, the unique information of the node cannot be maintained, and over-smoothing occurs. In response to the above problems, the present invention proposes an undirected graph node classification method based on hierarchical conditional random field (Hierarchical Conditional Random Field, H-CRF) to improve the accuracy of semi-supervised node classification tasks. Summary of the invention
[0004] The purpose of the present invention is to provide a semi-supervised node classification method for undirected graphs based on hierarchical conditional random fields, aiming to solve the problems raised in the above background technology.
[0005] To achieve the above object, the present invention provides the following technical solutions:
[0006] A semi-supervised node classification method for undirected graphs based on hierarchical conditional random fields comprises the following steps:
[0007] Step 1: Establish a model structure of a multi-layer graph coarsening layer, coarsen nodes with similar structures into a super node, and then continue to coarsen the coarsening operation on the coarsened graph to finally obtain a coarsened graph; after each layer of coarsening operation, add an improved conditional random field to strengthen the similarity relationship and position information of the nodes;
[0008] Step 2: Use the same number of graph refinement layers as the graph coarsening layers to restore the original graph topology, and add shortcut connections between corresponding levels to transfer the structural information in the graph.
[0009] Furthermore, each of the graph coarsening layers includes two steps, namely, graph coarsening operation and CRF normalization; specifically, as follows:
[0010] Graph coarsening operation: Nodes with similar structures are coarsened into a supernode. If two nodes u and v share nodes in the same set as neighbors, that is, {N(u)} = {N(v)}, then the two nodes u and v are said to have structural equivalence, where N(u) represents the neighbor nodes of u, {·} represents the set, and u and N(u) constitute a subgraph S. u , N(v) represents the neighbor nodes of v, v and N(v) constitute the subgraph Sv , then S u and S v There is at least one overlapping subgraph;
[0011] First, all nodes with structural equivalence in the graph are marked; second, for unmarked node pairs <v i ,v j >, calculate the structural similarity s between node pairs:
[0012] Formula 1:
[0013] where a k,ij Indicates that in the kth layer, <v i ,v j > is the weight of the edge, D(·) is the degree of the node;
[0014] Each unlabeled node v i Calculate the node similarity with its neighbor nodes and select the node with v i The node with the largest node similarity is coarsened, and the two nodes are marked at the same time; for unmarked nodes, the coarsening operation is performed in descending order of the node degree; if the neighbors of an unmarked node are all marked, the node is marked as a supernode; after marking all nodes, a coarsened graph is obtained;
[0015] Supernode v i With v i Neighbor node v k The weight of the edge between them is v k The sum of the edge weights with all neighboring nodes, including node v i ;
[0016] In the matrix The relationship between the nodes corresponding to the super nodes is retained; in the kth layer, each element t ij Calculate according to the following formula:
[0017]
[0018] The node hidden layer is represented as:
[0019] Formula 3: B k+1 =T k B k ;
[0020] Among them B k+1 represents the hidden layer representation of the node in the k+1th layer, T k represents the transformation matrix in the kth layer, B k represents the hidden layer representation of the node in the kth layer;
[0021] The adjacency matrix in each layer of the coarsened graph is calculated as follows:
[0022] Formula 4: A k+1 =T k A k T k ;
[0023] Among them A k+1 represents the adjacency matrix of the k+1th layer coarsened graph, A k represents the adjacency matrix of the k-th level coarsened graph, T k represents the transformation matrix in the kth layer;
[0024] CRF Normalization: In any graph, define a function Used to represent v in the figure i and v j The spatial relationship of Yes i and v j The feature space of the edges in the maximum overlapping subgraph is formed; adding the position encoding formed by the edges between nodes The calculation is as follows:
[0025] Formula 5:
[0026] Among them C src ,C trg are the embedding vectors composed of the edge starting point and target point, respectively, k,ij Indicates that in the kth layer, <v i ,v j >is the weight of the edge.
[0027] Furthermore, the graph refinement layer includes two steps, namely, graph refinement operation and saving node feature representation, as follows:
[0028] Graph refinement operation: Using T L-k For M obtained after the coarsening layer k To restore, the specific formula is as follows:
[0029] Formula 6: H k =T L-k M k +M l-k ;
[0030] Where k = 1, 2...L represents the current layer number, H k represents the hidden layer representation of the node after the CRF process, T L-k Represents the stored transformation matrix, M k represents the coarsened graph obtained at the kth layer, M l-k represents the kth coarsened graph from the end;
[0031] Save node feature representation: Add shortcut connections between corresponding coarsening-refining layers to transfer graph structure information between corresponding levels.
[0032] Compared with the prior art, the present invention has the following beneficial effects:
[0033] The present invention adopts a multi-layer coarsening structure to aggregate nodes with similar structures into super nodes. Each super node represents a local structure of the original graph, which makes it easier to obtain the structural information of the entire graph. The present invention adds an improved conditional random field to standardize the hidden layer features, retaining the node similarity relationship while increasing the node location information and structural attributes. The topological structure of the original graph is restored through refinement operations, and shortcut connections are added to ensure the stability of the hierarchical model. The results of the experiments conducted by the present invention on three public citation data sets are significantly improved compared with previous methods, and can better complete semi-supervised node classification, proving the effectiveness of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 It is a flow chart of the present invention.
[0035] Figure 2 It is a structural diagram of the H-CRF model in the present invention. DETAILED DESCRIPTION
[0036] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.
[0037] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.
[0038] An embodiment of the present invention provides a semi-supervised node classification method for undirected graphs based on hierarchical conditional random fields, and proposes an H-CRF model ( Figure 2 ), the method comprises the following steps: Figure 1 ):
[0039] Step 1: Establish a model structure of multi-layer graph coarsening layers, coarsen nodes with similar structures into a supernode, and then continue to coarsen the coarsening operation on the coarsened graph to finally obtain a smaller coarsening graph, which is more convenient for obtaining the global structural information of the graph. The topological structure of the graph and the similarity relationship between nodes cannot be guaranteed, so after each layer of coarsening operation, an improved conditional random field is added to strengthen the similarity relationship and position information of the nodes;
[0040] Each of the graph coarsening layers includes two steps, namely, graph coarsening operation and CRF normalization; specifically, as follows:
[0041] Graph coarsening operation: Nodes with similar structures are coarsened into a supernode. If two nodes u and v share nodes in the same set as neighbors, that is, {N(u)} = {N(v)}, then the two nodes u and v are said to have structural equivalence, where N(u) represents the neighbor nodes of u, {·} represents the set, and u and N(u) constitute a subgraph S. u , N(v) represents the neighbor nodes of v, v and N(v) constitute the subgraph S v , then S u and S v There is at least one overlapping subgraph;
[0042] First, all nodes with structural equivalence in the graph are marked; second, for unmarked node pairs <v i ,v j >, calculate the structural similarity s between node pairs:
[0043] Formula 1:
[0044] where a k,ij Indicates that in the kth layer, <v i ,v j > is the weight of the edge, D(·) is the degree of the node;
[0045] Each unlabeled node v i Calculate the node similarity with its neighbor nodes and select the node with v i The node with the largest node similarity is coarsened, and both nodes are marked at the same time; in particular, if the neighbors of an unmarked node are all marked, then the node itself will be marked as a supernode. The marking order of supernodes will result in different coarsened graphs generated in the end. It should be noted that the later the unmarked node is coarsened, the fewer its unmarked neighbors are, that is, nodes with smaller degrees have a greater probability of directly marking themselves as supernodes. Therefore, the present invention performs coarsening operations on unmarked nodes in descending order of degree.
[0046] After marking all nodes, a coarsened graph is obtained. i With v i Neighbor node v k The weight of the edge between them is v k The sum of the edge weights between all neighboring nodes, including node v i ; The degrees of nodes and edges in the coarsened graph will continue to be coarsened in the next layer using Formula 5.
[0047] In order to restore the coarsened graph to the initial graph scale, the present invention uses the matrix The relationship between the nodes corresponding to the supernode is preserved in the kth layer; each element t ijCalculate according to the following formula:
[0048] Formula 2:
[0049] The node hidden layer is represented as:
[0050] Formula 3: B k+1 =T k B k ;
[0051] Among them B k+1 represents the hidden layer representation of the node in the k+1th layer, T k represents the transformation matrix in the kth layer, B k represents the hidden layer representation of the node in the kth layer;
[0052] The adjacency matrix in each layer of the coarsened graph is calculated as follows:
[0053] Formula 4: A k+1 =T k A k T k ;
[0054] Among them A k+1 represents the adjacency matrix of the k+1th layer coarsened graph, A k represents the adjacency matrix of the k-th level coarsened graph, T k represents the transformation matrix in the kth layer;
[0055] CRF normalization: Conditional random fields can guarantee the similarity between hidden layer features to a certain extent, but lack explicit location information and structural dependence. Therefore, this paper adds the structural properties of the graph to improve the performance of the network. In any graph, define a function Used to represent v in the figure i and v j The spatial relationship of Yes i and v j The feature space of the edges in the maximum overlapping subgraph is formed; adding the position encoding formed by the edges between nodes The calculation is as follows:
[0056] Formula 5:
[0057] Among them C src ,C trg are the embedding vectors composed of the edge starting point and target point, respectively, k,ij Indicates that in the kth layer, <v i ,v j >is the weight of the edge. The larger the value, the more edges there are between the overlapping subgraphs of the two nodes, and the higher their structural similarity. The improved CRF module is improved by using It can adaptively calculate the structural information between nodes in the graph, so that the similarity relationship between nodes can be better preserved and the structural features are more obvious.
[0058] Step 2: In order to facilitate downstream tasks, preserve the topological structure of the graph and improve the performance of node classification tasks, the present invention uses a graph refinement layer with the same number of layers as the graph coarsening layer to restore the original graph topological structure, and at the same time adds shortcut connections between corresponding levels to transmit the structural information in the graph and ensure the stability of the model;
[0059] The graph refinement layer includes two steps, namely, graph refinement operation and saving node feature representation, as follows:
[0060] Graph refinement operation: The model of the present invention no longer performs CRF normalization in the graph refinement layer. Since the coarsened graph and simplified matrix are retained during the graph coarsening process, the T L-k For M obtained after the coarsening layer k To restore, the specific formula is as follows:
[0061] Formula 6: H k =T L-k M k +M l-k ;
[0062] Where k = 1, 2...L represents the current layer number, H k represents the hidden layer representation of the node after the CRF process, T L-k Represents the stored transformation matrix, M k represents the coarsened graph obtained at the kth layer, M l-k Represents the kth coarsening graph from the last. Save node feature representation: Add shortcut connections between corresponding coarsening-refining layers to transfer graph structure information between corresponding levels.
[0063] Example 1: The H-CRF model proposed in the present invention is compared with baseline methods such as MoNet, GAT, GCN, and Chebyshev to illustrate the effectiveness of the model proposed in the present invention.
[0064] 1.1 Dataset and Experimental Settings;
[0065] In order to verify the performance of the method of the present invention, the present invention compares H-CRF with a variety of methods that can achieve better performance. The present invention uses exactly the same parameter settings as GCN: 2 graph convolution layers, the hidden layer dimension is 16, and the dropout of all network layers is 0.5. The neuron weights are initialized using the Glorot method. The number of layers for coarsening and refining operations is 2. In addition, the model weight decay parameter is 0.0005, and the Adam optimizer with a learning rate of 0.01 is used to optimize H-CRF.
[0066] The datasets used in the experiments of this paper are the same as those in GCN: Cora, Citeseer and Pubmed. The nodes in these citation networks are documents from different topics, and the references between documents constitute the edges of the graph. The bag-of-words of the corresponding document is used to represent the node features. The following is a detailed description of the three citation networks:
[0067] (1) Citeseer is a research paper citation network. There are 3327 papers on 6 topics. The total number of citations is 4732. The dimension of node features is 3703.
[0068] (2) Cora is also a citation network. It has 2708 documents on 7 topics. Each document has 1433 features. The number of citations between different documents is 5429.
[0069] (3) Pubmed is another citation network. It consists of 19,717 documents in 3 categories. Each document has 500 features. The statistics of the dataset are summarized in Table 1.
[0070] Table 1 Standard dataset
[0071]
[0072] The present invention designs a semi-supervised node classification task to evaluate the performance of the proposed model: consistent with the setting of GCN, only 20 nodes are used for training for each class of labels; 500 nodes are used as the validation set to verify the model; the test set contains 1000 nodes. The comparison results are shown in Table 2.
[0073] Table 2 Accuracy of semi-supervised node classification
[0074]
[0075] Table 2 shows the accuracy of the proposed method in semi-supervised tasks. As can be seen from Table 2, H-CRF has a significant improvement on the performance of the GCN network.
[0076] In summary, the experimental results of the present invention on three public citation datasets are significantly improved compared with previous methods, and can better complete semi-supervised node classification, which proves the effectiveness of the present invention.
[0077] The above are only preferred embodiments of the present invention. It should be pointed out that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention. These should also be regarded as the protection scope of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent.
Claims
1. A semi-supervised node classification method for undirected graphs based on hierarchical conditional random fields, It is characterized in that The following steps are involved: Step 1: Establish a model structure of a multi-layer graph coarsening layer, coarsen nodes with similar structures into a super node, and then continue to coarsen the coarsening operation on the coarsened graph to finally obtain a coarsened graph; after each layer of coarsening operation, add an improved conditional random field to strengthen the similarity relationship and position information of the nodes; Step 2: Use the same number of graph refinement layers as the graph coarsening layers to restore the original graph topology, and add shortcut connections between corresponding levels to transfer the structural information in the graph.
2. The undirected graph semi-supervised node classification method based on hierarchical conditional random fields according to claim 1, It is characterized in that Each of the graph coarsening layers includes two steps, namely, graph coarsening operation and CRF normalization; specifically, as follows: Graph coarsening operation: Nodes with similar structures are coarsened into a supernode. If two nodes u and v share nodes in the same set as neighbors, that is, {N(u)} = {N(v)}, then the two nodes u and v are said to have structural equivalence, where N(u) represents the neighbor nodes of u, {·} represents the set, and u and N(u) constitute a subgraph S. u , N(v) represents the neighbor nodes of v, v and N(v) constitute the subgraph S v , then S u and S v There is at least one overlapping subgraph; First, all nodes with structural equivalence in the graph are marked; second, for unmarked node pairs <v i ,v j >, calculate the structural similarity s between node pairs: where a k,ij Indicates that in the kth layer, <v i ,v j > is the weight of the edge, D(·) is the degree of the node; Each unlabeled node v i Calculate the node similarity with its neighbor nodes and select the node with v i The node with the largest node similarity is coarsened, and the two nodes are marked at the same time; for unmarked nodes, the coarsening operation is performed in descending order of the node degree; If all neighbors of an unlabeled node are labeled, the node is labeled as a supernode; After marking all nodes, a coarsened graph is obtained; Supernode v i Its neighbor node v k The weight of the edge between them is v k The sum of the edge weights with all neighboring nodes, including node v i ; In the matrix The relationship between the nodes corresponding to the super nodes is retained; in the kth layer, each element t ij Calculate according to the following formula: The node hidden layer is represented as: Formula 3: B k+1 =T k B k ; Among them B k+1 represents the hidden layer representation of the node in the k+1th layer, T k represents the transformation matrix in the kth layer, B k represents the hidden layer representation of the node in the kth layer; The adjacency matrix in each layer of the coarsened graph is calculated as follows: Formula 4: A k+1 = T k A k T k ; Among them A k+1 represents the adjacency matrix of the k+1th layer coarsened graph, A k represents the adjacency matrix of the k-th level coarsened graph, T k represents the transformation matrix in the kth layer; CRF Normalization: In any graph, define a function Used to represent v in the figure i and v j The spatial relationship of Yes i and v j The feature space of the edges in the maximum overlapping subgraph is formed; adding the position encoding formed by the edges between nodes The calculation is as follows: Formula 5: Among them C src ,C trg are the embedding vectors composed of the edge starting point and target point, respectively, k,ij Indicates that in the kth layer, <v i ,v j >is the weight of the edge.
3. The undirected graph semi-supervised node classification method based on hierarchical conditional random fields according to claim 1, It is characterized in that The graph refinement layer includes two steps, namely, graph refinement operation and saving node feature representation, as follows: Graph refinement operation: Using T L-k For M obtained after the coarsening layer k To restore, the specific formula is as follows: Formula 6: H k =T L-k M k +M l-k ; Where k = 1, 2...L represents the current layer number, H k represents the hidden layer representation of the node after the CRF process, T L-k Represents the stored transformation matrix, M k represents the coarsened graph obtained at the kth layer, M l-k represents the kth coarsened graph from the end; Save node feature representation: Add shortcut connections between corresponding coarsening-refining layers to transfer graph structure information between corresponding levels.
