A graph node classification method based on a new graph neural network

Abstract

The present invention is applicable to the technical field of graph structure data, and provides a graph node classification method based on a new graph neural network, comprising the following steps: reading graph data; calculating feature similarity between nodes, and setting a similarity threshold, adding edges between node pairs when the similarity between nodes exceeds the preset threshold; dividing neighbors of different relationships for nodes and adding relationship attributes to the edges of nodes, obtaining a new graph structure after neighborhood reconstruction; inputting the reconstructed graph into MRN‑GNN, aggregating neighbors of the same type with the central node first, and during the aggregation process, calculating the attention coefficient of neighbors and central node features under each spatial relationship, and weightedly aggregating the attention coefficient with neighbor features; splicing the obtained multi-relation features with the central node features, and finally outputting the classification prediction of the node. The present invention can effectively capture the features of distant nodes without deepening the network, and effectively distinguish the relationship structures of different neighbors.

Description

Technical Field

[0001] The present invention belongs to the technical field of graph structured data, and in particular relates to a graph node classification method based on a novel graph neural network. Background Art

[0002] Graph structured data plays an important role in our lives. For example, social networks, traffic networks, knowledge graphs, and protein interaction networks can all be represented as graph data. Graph neural networks (GNNs) are widely used in network analysis, traffic prediction, recommendation systems, and natural language processing. These applications rely on basic graph tasks including node classification, link prediction, and graph classification. Node classification is an important task in graph learning. The low-dimensional embedding vector of each node is learned by aggregating the features of neighboring nodes, and the category of the node is finally predicted. However, most GNNs are based on the assumption of homogamy, that is, nodes with the same label and similar features tend to be connected to each other. For example, in citation networks, research papers in the same field tend to cite each other. However, there are also heterogamous graph structures in life. For example, in social friendship networks, heterogamous user nodes are connected to each other, and in protein structure networks, different types of amino acids are often connected. Therefore, it is necessary to study related issues under heterogamous graph structures.

[0003] Some works have conducted relevant research on the problem of heterogeneity of graph structure. In heterogeneous graphs, there are many nodes with the same label and similar features but far away in the graph structure. In order to effectively capture these distant nodes, some methods try to make the network deeper to aggregate the features of distant neighbors. However, the deeper network has the problem of over-smoothing, which makes it impossible to distinguish node categories. Some works such as JK-Net use skip connections to deepen the network, alleviating the over-smoothing problem; GCNII uses the initial residual and unit mapping method to express it as a k-order polynomial filter with arbitrary coefficients to capture distant nodes, but these methods still face the problem of node structure information loss. In order to capture the different structures of node neighbors, some works such as MixHop combine different powers of the adjacency matrix with different weight matrices, so that the network can learn mixed relationship representations of neighbors of different orders. In order to retain the structural information of neighbor nodes, some works proposed Geom-GCN, which maps the graph to multiple continuous hyperbolic spaces. In these spaces, they identify structural neighbors for each node and adopt a new aggregation method to obtain a comprehensive graph representation. However, the above methods do not explicitly construct neighbors with different spatial relationships for nodes, and ignore the importance of distinguishing neighbors with different structures.

[0004] In view of the shortcomings of the above methods, the present invention proposes a new graph neural network architecture: MRN-GNN (Multi-relational neighbor constructed GNN), which can not only capture the features of distant nodes without deepening the network, but also construct neighbors of different spatial types for nodes, effectively capturing the different structures of node neighbors, and achieving better results in solving the heterogeneity problem of social networks. Summary of the invention

[0005] The purpose of the present invention is to provide a graph node classification method based on a novel graph neural network, aiming to solve the problems raised in the above-mentioned background technology.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A graph node classification method based on a novel graph neural network includes the following steps:

[0008] Step 1, multi-relation neighborhood reconstruction: read the graph data G, including the adjacency matrix of the graph and the feature matrix of the nodes; calculate the feature similarity between the nodes and set the similarity threshold. When the similarity between the nodes exceeds the preset threshold, add edges between the node pairs; divide the neighbors of different relationships for the nodes and add relationship attributes to the edges of the nodes to obtain a new graph structure after neighborhood reconstruction;

[0009] Step 2, two-layer attention aggregation network: The reconstructed graph is input into the graph neural network MRN-GNN. MRN-GNN aggregates neighbors of the same type with the central node first. In this aggregation process, in order to model the importance of neighbors, the attention coefficients of the neighbors and central node features under each spatial relationship are calculated, and then the attention coefficients and neighbor features are weightedly aggregated to obtain the aggregation features of the corresponding spatial relationships; then the features of the various relationships obtained are concatenated with the central node features, and finally, after linear transformation and the prediction layer of the network, the classification prediction of the nodes in the network is output.

[0010] Furthermore, the specific process of step 1 is as follows:

[0011] Step 11: After reading the graph data G, use cosine similarity to measure the similarity between any two nodes in the graph data G. , represents the initial features of nodes i and j, and the similarity of the features of nodes i and j is expressed as:

[0012] Formula 1: ;

[0013] Let λ represent the similarity threshold. If any s i,jIf the value of is greater than λ, an edge is added between nodes i and j, and the node pairs that meet the conditions are represented by the set P:

[0014] Formula 2: ;

[0015] The adjacency matrix of the graph data G is represented by A. To add edges between node pairs that meet the similarity threshold, the reconstructed adjacency matrix is ​​represented as:

[0016] Formula 3: ;

[0017] Step 12: construct a neighborhood of multiple relationship types for the graph data G after adding edges, and obtain a new graph structure with multiple relationships; take any node i as the central node, and define three different relationships R for the central node:

[0018] Formula 4: ;

[0019] Among them, t represents the original topological connection relationship, s represents the connection relationship based on feature similarity, e represents the self-connection of the node, and the neighbor nodes under the corresponding relationship of t, s, and e are represented as N t , N s , N e ; Finally, a new reconstructed graph G' is obtained.

[0020] Furthermore, the specific process of step 2 is as follows:

[0021] Step 21: Two-layer aggregation mechanism under multi-relational neighbors: Reconstruct the graph G’ Any node v is the central node, and the neighbors with the same connection relationship are aggregated first. The characteristics of the central node v after the first layer aggregation are expressed as :

[0022] Formula 5: ;

[0023] Where u is the neighbor node of the central node v in the reconstructed graph G', r represents the different spatial relationships to which the node belongs, and p represents the aggregation function. represents the aggregated features of the neighbors of type r of the central node v, N r is the set of neighbors of v whose relationship is r, is the feature of node u in layer l; in the second layer aggregation, the obtained feature Further aggregation with function q, the feature of the second aggregation is expressed as :

[0024] Formula 6: ;

[0025] Finally, After nonlinear transformation, the feature representation of the next layer is :

[0026] Formula 7: ;

[0027] in is the learnable matrix of the lth layer; activation function Use ReLU;

[0028] Step 22: Attention mechanism for multi-relational neighborhoods: for reconstructed graphs G’ For any node v in , the neighbors of each connection relationship first undergo a linear transformation:

[0029] Formula 8: ;

[0030] Among them, H r is the feature matrix of the neighbors with relationship r, W r is the learnable parameter matrix of relation r, h r is the original neighbor feature matrix with relationship r;

[0031] Then calculate the attention coefficient between nodes under each connection relationship, and use express:

[0032] Formula 9: ;

[0033] in , are the features of nodes i and j under relation r, , represents the node-shared attention mechanism, which is used to calculate the attention coefficient between nodes. The calculated coefficient represents the importance of node j to node i in relation r. ; Finally, perform normalization:

[0034] Formula 10: ;

[0035] in, , LeakyReLU is the activation function, is the neighbor node of node i under relationship r;

[0036] After obtaining the normalized attention coefficient, the linear combination of the corresponding features is calculated as the final output feature of each node; the updated feature of node i is expressed as:

[0037] Formula 11: ;

[0038] in is the updated feature of node i under relation r; if the multi-head attention mechanism is adopted, the updated feature of node i under relation r is expressed as:

[0039] Formula 12: ;

[0040] Where K represents the number of attention heads; represents the attention weight learned between nodes i and j under relation r in the kth attention head; represents the learnable parameter matrix in the kth head, Represents the activation function ReLU;

[0041] Step 23: Double-layer attention aggregator for multi-relation neighborhood: Substitute the aggregation function of p in Formula 5 with the multi-relation attention aggregation function, then It is expressed as:

[0042] Formula 13: ;

[0043] in, For the node with relationship r For Node The attention coefficient, is the learnable parameter matrix of the lth layer under the relation r;

[0044] Final It is expressed as:

[0045] Formula 14: ;

[0046] Among them, || represents Perform splicing operation and activate function Select ReLU, is the learnable parameter matrix of the overall linear transformation in the lth layer; if it is the last round, then Use mean operation between them;

[0047] Finally, the predicted features are used to predict the node category through the SoftMax function. , using cross entropy Loss as:

[0048] Formula 15: ;

[0049] Where N is the number of nodes, C is the number of categories, Indicates that the i-th sample belongs to the true label of the c-th category, It represents the predicted probability that the i-th sample belongs to the c-th category.

[0050] Compared with the prior art, the present invention has the following beneficial effects:

[0051] 1. The present invention has practical significance in the actual application of node classification of multiple heterogeneous social networks. It can not only effectively capture the characteristics of distant nodes without deepening the network, but also effectively distinguish the relationship structures of different neighbors.

[0052] 2. The present invention verifies the newly designed network MRN-GNN, which not only achieves good classification results on real heterogeneous datasets, such as improving the accuracy by 16.21% compared with Geom-GCN-P (a network designed specifically for heterogeneous graphs) on the Cornell dataset, and improving the accuracy by 24.32% compared with the classic GCN method; it also achieves competitive results on real homogeneous datasets. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 Flowchart of the present invention; wherein (a) is a flowchart of multi-relation neighborhood reconstruction, and (b) is a flowchart of a two-layer attention aggregation network.

[0054] Figure 2 Schematic diagram of similarity connection and multi-relation neighborhood construction; (a) is the similarity measurement based on the original graph, (b) is the graph with similarity connection added, and (c) is the multi-relation neighborhood construction.

[0055] Figure 3 An example diagram of multi-relational two-layer attention aggregation.

[0056] Figure 4 The figure is a visualization of classification T-SNE; (a) shows the classification effect of MRN-GNN on the homogeneous dataset Cora, and (b) shows the classification effect of MRN-GNN on the heterogeneous dataset Wisconsin.

[0057] Figure 5 This is the accuracy of this method on the same matching images under different similarity thresholds.

[0058] Figure 6 This is the accuracy of this method on heterogeneous graphs under different similarity thresholds. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0060] The specific implementation of the present invention is described in detail below in conjunction with specific embodiments.

[0061] An embodiment of the present invention provides a graph node classification method based on a novel graph neural network, comprising the following steps:

[0062] Step 1, multi-relation neighborhood reconstruction: read the graph data G, including the adjacency matrix of the graph and the feature matrix of the nodes. In order to enable nodes to aggregate node features that are more similar to themselves, we calculate the feature similarity between nodes, measure the degree of similarity between nodes, and set a similarity threshold. When the similarity between nodes exceeds the preset threshold, edges are added between node pairs, so that nodes with distant distances but similar features can be connected, which can better enable information exchange between similar nodes. Furthermore, in order to distinguish the neighbors of nodes and improve the expressiveness of the network, the present invention divides the neighbors of nodes into different relationships and adds relationship attributes to the edges of nodes to obtain a new graph structure after neighborhood reconstruction;

[0063] Step 2, two-layer attention aggregation network: After the neighborhood is reconstructed, the reconstructed graph is input into the graph neural network MRN-GNN. Different from the traditional GCN (graph convolutional neural network), the aggregation function of MRN-GNN aggregates neighbors of the same type with the central node first. In this aggregation process, in order to model the importance of neighbors, the attention coefficients of the neighbors and central node features under each spatial relationship are calculated, and then the attention coefficients and neighbor features are weightedly aggregated to obtain the aggregation features of the corresponding spatial relationships; then the features of the various relationships obtained are concatenated with the central node features, and finally, after linear transformation and the prediction layer of the network, the classification prediction of the nodes in the network is output.

[0064] like Figure 1 As shown in (a), as a preferred embodiment of the present invention, the specific process of step 1 is as follows:

[0065] Step 11: After reading the graph data G, use cosine similarity to measure the similarity between any two nodes in the graph data G. , represents the initial features of nodes i and j, and the similarity of the features of nodes i and j is expressed as:

[0066] Formula 1: ;

[0067] Let λ represent the similarity threshold. If any s i,j If the value of is greater than λ, an edge is added between nodes i and j, and the node pairs that meet the conditions are represented by the set P:

[0068] Formula 2: ;

[0069] The adjacency matrix of the graph data G is represented by A (excluding self-loops). To add edges between node pairs that meet the similarity threshold, the reconstructed adjacency matrix is ​​represented as:

[0070] Formula 3: ;

[0071] At this time, for graphs with strong heterogeneity, edges can be added between nodes with similar features but far distances, which is conducive to effective message transmission of features.

[0072] Step 12: construct a neighborhood of multiple relationship types for the graph data G after adding edges, and obtain a new graph structure with multiple relationships; take any node i as the central node, and define three different relationships R for the central node:

[0073] Formula 4: ;

[0074] Among them, t represents the original topology connection relationship, s represents the connection relationship based on feature similarity, e represents the ego-connection of the node, and the neighbor nodes under the corresponding relationship of t, s, and e are represented as N t , N s , N e ; Finally, a new reconstructed graph G' is obtained. Figure 2 Schematic diagrams are constructed based on similarity connections and multi-relation neighborhoods.

[0075] like Figure 1 As shown in (b), as a preferred embodiment of the present invention, the specific process of step 2 is as follows:

[0076] Step 21: Two-layer aggregation mechanism under multi-relational neighbors: Reconstruct the graph G’ Any node v is the central node, and the neighbors with the same connection relationship are aggregated first. The characteristics of the central node v after the first layer aggregation are expressed as :

[0077] Formula 5: ;

[0078] Where u is the neighbor node of the central node v in the reconstructed graph G', r represents the different spatial relationships to which the node belongs, and p represents the aggregation function. In this layer, for each node in the graph, the features of all neighbor nodes belonging to the same connection relationship are aggregated together through the aggregation function. represents the aggregated features of the neighbors of type r of the central node v, N r is the set of neighbors of v whose relationship is r, is the feature of node u in layer l; in the second layer aggregation, the obtained feature Further aggregation with function q, the feature of the second aggregation is expressed as :

[0079] Formula 6: ;

[0080] Finally, After nonlinear transformation, the feature representation of the next layer is :

[0081] Formula 7: ;

[0082] in is the learnable matrix of the lth layer; As the activation function, the present invention selects ReLU;

[0083] Step 22: Attention mechanism for multi-relational neighborhoods: for reconstructed graphs G’ For any node v in , the neighbors of each connection relationship first undergo a linear transformation:

[0084] Formula 8: ;

[0085] Among them, H r is the feature matrix of the neighbors with relationship r, W r is the learnable parameter matrix of relation r, h r is the original neighbor feature matrix with relationship r.

[0086] Then calculate the attention coefficient between nodes under each connection relationship, and use express:

[0087] Formula 9: ;

[0088] in , are the features of nodes i and j under relation r, , represents the node-shared attention mechanism, which is used to calculate the attention coefficient between nodes. The calculated coefficient represents the importance of node j to node i in relation r. ; Finally, perform normalization:

[0089] Formula 10: ;

[0090] in , LeakyReLU is the activation function, is the neighbor node of node i under relationship r;

[0091] After obtaining the normalized attention coefficient, the linear combination of the corresponding features is calculated as the final output feature of each node; the updated feature of node i is expressed as:

[0092] Formula 11: ;

[0093] in is the updated feature of node i under relation r; if the multi-head attention mechanism is adopted, the updated feature of node i under relation r is expressed as:

[0094] Formula 12: ;

[0095] Where K represents the number of attention heads; represents the attention weight learned between nodes i and j under relation r in the kth attention head; represents the learnable parameter matrix in the kth head; Represents the activation function ReLU;

[0096] Step 23. Double-layer attention aggregator for multi-relational neighborhoods: In order to realize double-layer aggregation with attention mechanism under multi-relational neighborhoods, the aggregation function of p in Formula 5 is replaced by the multi-relational attention aggregation function, then It is expressed as:

[0097] Formula 13: ;

[0098] in, For the node with relationship r For Node The attention coefficient, is the learnable parameter matrix of the lth layer under the relation r;

[0099] Final It is expressed as:

[0100] Formula 14: ;

[0101] Among them, || represents Perform splicing operation and activate function Select ReLU, is the learnable parameter matrix of the overall linear transformation in the lth layer. It is worth noting that if it is the last round, then The mean operation is used between them. Figure 3 The dual-layer attention aggregation under multiple relations is briefly described in , assuming that the feature corresponding to the central node is , the corresponding features of neighbor nodes based on similarity are , the neighbor features of the original topology correspond to . represents the attention coefficient, such as Indicates that the central node is in the spatial relationship s, and the feature is After the first layer of aggregation, the central node obtains features from three spatial relationships. , and ; In the second layer of aggregation, the features of the three spatial relationships obtained in the first layer are further aggregated to obtain the final feature representation .

[0102] Finally, the predicted features are used to predict the node category through the SoftMax function. , using cross entropy Loss as:

[0103] Formula 15: ;

[0104] Among them, N is the number of nodes, C is the number of categories, c is the variable, Indicates that the i-th sample belongs to the true label of the c-th category, It represents the predicted probability that the i-th sample belongs to the c-th category.

[0105] like Figure 4 This is the classification effect of MRN-GNN on the representative homogamous dataset Cora and heterogamous dataset Wisconsin. We use the T-SNE visualization method to display the network prediction results. Different grayscales represent different categories. The closer the distance between nodes of the same grayscale is, and the farther the distance from nodes of different grayscales is, the better the node classification effect is. Figure 4 It can be seen that this method achieves better classification effect.

[0106] Embodiment 1, experiment is carried out under multiple real data sets, including 3 commonly used homogeneous data sets: Cora, Citeseer, Pubmed, these three data sets belong to the widely used citation data sets, nodes represent papers, edges represent the citation relationship between papers, the characteristics of nodes are the characteristic description of papers, and the labels of nodes are the categories to which the articles belong. 3 data sets with strong heterogeneity: Texas, Cornell, Wisconsin. They are three sub-datasets of the webpage data set webKB collected by Carnegie Mellon University from the computer science departments of various universities. They are all data sets representing the relationship between web pages. Nodes represent web pages, edges represent the links between two web pages, the characteristics of nodes are the characteristic descriptions of web pages, and the labels of nodes are the categories to which web pages belong. The specific data set attributes are detailed in Table 1.

[0107] Table 1 Introduction to data sets and data division

[0108]

[0109] The MRN-GNN proposed in this paper is compared with a variety of representative methods, among which GCN (graph convolutional network) and GAT (graph attention network) are classic graph neural network architectures; Res-GCN (residual graph convolutional network) and JK-Net (graph network with jump connections) use residual and jump connections to deepen the network to obtain the features of distant nodes; SDRF (graph network based on edge combination curvature) analyzes the heterogeneity and over-smoothing problems of graphs from the perspective of curvature; Geom-GCN (graph convolutional network based on geometry) preserves the structural information of nodes by mapping heterogeneous graphs to a variety of hyperbolic spaces; Geom-GCN-I, Geom-GCN-P, and Geom-GCN-S represent methods of embedding graph G into hyperbolic space using Isomap, Poincare, and Struc2vec, respectively. Table 2 shows the comparison results of this method with other methods, with the best results in bold and the suboptimal results marked with gray shading.

[0110] Table 2 Comparison results of this method with other methods

[0111]

[0112] It can be seen that among the methods listed in Table 2, the MRN-GNN proposed in the present invention achieves the best results on heterogeneous graphs, and also achieves very competitive results on homogeneous graphs, which shows that the present invention not only works well on heterogeneous graphs, but can also be well applied to homogeneous graphs.

[0113] In order to better verify the effectiveness of each module designed by the present invention, the present invention conducted an ablation experiment. Table 3 shows the model effect when the similarity threshold λ is 0.6 and the number of attention heads is 1. The present invention takes GCN as the basic network and then gradually adds various designs of the present invention. It can be seen that the method of the present invention can generally and effectively improve the accuracy of the model step by step. Similarly, in order to be more convincing, the present invention also conducted an ablation experiment under the conditions of the similarity threshold λ being 0.7 and the number of attention heads being 8. The experimental results are shown in Table 4. The method of the present invention can still gradually improve the expression ability of the network, verifying the rationality and effectiveness of the multi-relation neighborhood reconstruction method and the two-layer attention aggregation mechanism proposed by the present invention.

[0114] Table 3 Effects of various designs when the similarity threshold is 0.6 and the attention head is 1

[0115]

[0116] Table 4 Effects of various designs when the similarity threshold is 0.7 and the number of attention heads is 8

[0117]

[0118] At the same time, we tested the accuracy of this method on the same matching images and different matching images under different similarity thresholds. For specific results, see Figure 5 and Figure 6 As can be seen from the figure, different data sets have different sensitivities to changes in the similarity threshold; at the same time, as the similarity threshold increases, the accuracy of the model is not completely positively correlated with the increase in the threshold. The accuracy of different data sets shows different upward or downward trends, indicating that the higher the similarity threshold, the better. It is necessary to consider the different structures of specific graph data in order to set a reasonable threshold for the model.

[0119] The above are only preferred embodiments of the present invention. It should be pointed out that, for those skilled in the art, several modifications and improvements can be made without departing from the concept of the present invention. These should also be regarded as the protection scope of the present invention. These will not affect the effect of the implementation of the present invention and the practicality of the patent.

Claims

1. A graph node classification method based on a novel graph neural network, characterized in that: The graph node classification method is applied to citations, where nodes represent papers, edges represent citation relationships between papers, the features of the nodes are feature descriptions of the papers, and the labels of the nodes are the categories to which the papers belong; the method comprises the following steps: Step 1, multi-relation neighborhood reconstruction: read the graph data G, including the adjacency matrix of the graph and the feature matrix of the nodes; calculate the feature similarity between the nodes and set the similarity threshold. When the similarity between the nodes exceeds the preset threshold, add edges between the node pairs; divide the neighbors of different relationships for the nodes and add relationship attributes to the edges of the nodes to obtain a new graph structure after neighborhood reconstruction; Step 2, two-layer attention aggregation network: input the reconstructed graph into the graph neural network MRN-GNN, aggregate the neighbors of the same type with the central node first, calculate the attention coefficient of the neighbors and central node features under each spatial relationship during the aggregation process, and then perform weighted aggregation of the attention coefficient and the neighbor features to obtain the aggregation features of the corresponding spatial relationship; then concatenate the features of the various relationships obtained with the central node features, and finally output the classification prediction of the nodes in the network after linear transformation and the prediction layer of the network; The specific process of step 1 is as follows: Step 11: After reading the graph data G, the cosine similarity is used to measure the similarity between any two nodes in the graph data G. , represents the initial features of nodes i and j, and the similarity of the features of nodes i and j is expressed as: Formula 1: ; use λ Represents the similarity threshold. If any s i,j The value is greater than λ , then add an edge between nodes i and j, and represent the node pairs that meet the conditions with the set P: Formula 2: ; The adjacency matrix of the graph data G is represented by A. To add edges between node pairs that meet the similarity threshold, the reconstructed adjacency matrix is ​​represented as: Formula 3: ; Step 12: construct a neighborhood of multiple relationship types for the graph data G after adding edges, and obtain a new graph structure with multiple relationships; take any node i as the central node, and define three different relationships R for the central node: Formula 4: ; Among them, t represents the original topological connection relationship, s represents the connection relationship based on feature similarity, e represents the self-connection of the node, and the neighbor nodes under the corresponding relationship of t, s, and e are respectively expressed as N t , N s , N e Finally, we get a new reconstruction G’ ; The specific process of step 2 is as follows: Step 21: Two-layer aggregation mechanism under multi-relational neighbors: Reconstruct the graph G’ Any node in v As the central node, the neighbors with the same connection relationship are aggregated first, and the central node v The feature representation after the first layer aggregation is : Formula 5: ; in u Central Node v In the reconstruction G’ The neighbor nodes in r Indicates the different spatial relationships to which the nodes belong, p represents an aggregate function, Represents the central node v The type is r Aggregate features of neighbors, N r is the set of neighbors of v whose relationship is r, For u node l The features of the layer; in the second layer of aggregation, the obtained features Further use of function q Aggregation, the second aggregation feature is expressed as : Formula 6: ; Finally, After nonlinear transformation, the feature representation of the next layer is : Formula 7: ; in For the l Layer learnable matrix; activation function Use ReLU; Step 22: Attention mechanism for multi-relational neighborhoods: for reconstructing graphs G’ Any node in v , the neighbors of each connection relationship are first subjected to a linear transformation: Formula 8: ; in H r is the feature matrix of neighbors with relationship r, W r is the learnable parameter matrix with relation r, h r is the original neighbor feature matrix with relationship r; Then calculate the attention coefficient between nodes under each connection relationship, and use express: Formula 9: ; in , are the features of nodes i and j under relation r, , represents the node-shared attention mechanism, which is used to calculate the attention coefficient between nodes. The calculated coefficient represents the importance of node j to node i in relation r. ; Finally, perform normalization: Formula 10: ; in is the learnable parameter vector in the attention mechanism, LeakyReLU is the activation function, is the neighbor node of node i under relationship r; After obtaining the normalized attention coefficient, the linear combination of the corresponding features is calculated as the final output feature of each node; the updated feature of node i is expressed as: Formula 11: ; in is the updated feature of node i under relation r; if the multi-head attention mechanism is adopted, the updated feature of node i under relation r is expressed as: Formula 12: ; Where K represents the number of attention heads; represents the attention weight learned between nodes i and j under relation r in the kth attention head; represents the learnable parameter matrix in the kth head; ReLU activation function. Step 23: Double-layer attention aggregator for multi-relational neighborhoods: Substitute p The aggregation function of takes the multi-relation attention aggregation function, then the It is expressed as: Formula 13: ; in, is the attention coefficient of node u with relationship r to node v, For the l The learnable parameter matrix of the layer under relation r; Final It is expressed as: Formula 14: ; in, express Perform splicing operations between them and activate the function Select ReLU, For the l The learnable parameter matrix of the overall linear transformation in the layer; if it is the last round, then Use mean operation between them; Finally, the predicted features are used to predict the node category through the SoftMax function. , using cross entropy Loss as: Formula 15: ; Where N is the number of nodes, C is the number of categories, Indicates that the i-th sample belongs to the true label of the c-th category, It represents the predicted probability that the i-th sample belongs to the c-th category.

Images