Heterogeneous graph representation learning method based on meta-path multi-level graph attention network

Abstract

The application relates to a heterogeneous graph representation learning method based on a meta-path multi-level graph attention network, which comprises four modules: a simple graph and hypergraph construction module based on a meta-path, a simple graph feature aggregation module, a hypergraph feature aggregation module and a node type prediction module; the meta-path-based multi-level graph construction module extracts pair and non-pair relationships in the heterogeneous graph through a meta-path, and then constructs corresponding simple graphs and hypergraphs; the simple graph feature aggregation module is used for preliminarily aggregating node features on the simple graph; the hypergraph feature aggregation module further integrates high-order relationships and intermediate path information into the node features; the node type prediction module trains the model by using various types of nodes in the heterogeneous graph, and outputs node representation for downstream classification or clustering tasks. The hypergraph is introduced into the research of the heterogeneous graph representation learning, the hypergraph can directly contain any number of target nodes, so that the multi-element relationship in the heterogeneous graph is completely saved, and semantic loss is not caused.

Description

TECHNICAL FIELD

[0001] The application belongs to the field of computer application, data mining, graph representation learning and graph neural network research, and particularly relates to a heterogeneous graph embedding model based on a multi-level graph attention network. BACKGROUND

[0002] Graph Neural Network (GNN) is a hot topic in recent years. GNN can integrate neighbor information into node embedding representation, thereby converting sparse and high-dimensional adjacency matrix information into dense and low-dimensional node feature representation. This greatly helps downstream tasks based on graph structure, such as node classification and link prediction. In the real world, the nodes and edges in the graph often have multiple types, i.e., the graph is heterogeneous. The heterogeneity of the graph brings difficulties to the work of traditional GNN, and the model needs to consider the weight of different types of relationships in the aggregation process. The features of nodes or edges may also be located in different feature spaces, which makes the heterogeneous graph representation learning pose higher requirements and greater challenges to the graph neural network method.

[0003] Based on the research idea of introducing a hypergraph to provide supplementary structure and content, the application proposes a heterogeneous graph representation learning method based on a multi-level graph attention network. A hypergraph attention network is added to the simple graph neural network, which can integrate the missing intermediate path information of the simple graph. The combination of the hypergraph and the simple graph enables the model to have a flexible receptive field size, thereby being able to capture local and global information in the heterogeneous graph and effectively improve the comprehensive quality of the generated node representation. SUMMARY

[0004] Technical problems to be solved

[0005] In view of the deficiencies and problems in the prior art, such as the inability of traditional meta-path-based methods to consider multiple relationships in the heterogeneous graph and the loss of intermediate path information, the application proposes a heterogeneous graph representation learning method based on a multi-level graph attention network. The original heterogeneous graph is reconstructed using a simple graph and a hypergraph based on a meta-path, and a multi-level attention network is used to learn this structure, so that the model has a flexible receptive field size. The hypergraph is introduced into the research of heterogeneous graph representation learning. The hypergraph can directly contain any number of target nodes, thereby completely preserving the multiple relationships in the heterogeneous graph and not losing semantics.

[0006] Technical scheme

[0007] A heterogeneous graph representation learning method based on a meta-path multi-level graph attention network, characterized in that it comprises the following steps:

[0008] Step 1: Spatial mapping of node attribute features

[0009] Step 1.1, different linear transition matrices are assigned to different types of nodes to map the attribute features of all nodes into the same feature space.

[0010] Step 2, construction of multi-level graph structure based on meta-path;

[0011] Step 2.1, construction of simple graph based on meta-path. The pairs of nodes in the heterogeneous graph are extracted by the meta-path, and all node pairs are combined to form a simple graph based on the meta-path. Change the meta-path type to define multiple simple graphs.

[0012] Step 2.2, construction of hypergraph based on meta-path. Multiple nodes connected by the same meta-path are placed in the same hyperedge, and all hyperedges are constructed to form a hypergraph based on the meta-path. Change the meta-path type to define multiple hypergraphs.

[0013] Step 3, simple graph feature aggregation module;

[0014] Step 3.1, simple graph neighbor node attention coefficient calculation. The features of the target node and its first-order neighbor nodes are connected using the Concatenation operation, the ReLU(·) is used as the activation function, and the normalized features are obtained by the Softmax function. The attention coefficient of each node is obtained when aggregating the node features.

[0015] Step 3.2, simple graph node feature aggregation. The target node and neighbor node features are weighted and summed using the corresponding attention coefficient in step 3.1 to obtain the target node feature.

[0016] Step 3.3, multi-head attention mechanism. Repeat steps 3.1 and 3.2 T times (usually 4 to 8 times), and connect the T features of the same node to obtain a feature.

[0017] Step 4, hypergraph feature aggregation module;

[0018] Step 4.1, intermediate path feature encoding. As shown in Figure 2 , the features of the intermediate nodes in each meta-path are encoded using the relationship rotation encoder RotatE to obtain the intermediate path features.

[0019] Step 4.2, hyperedge internal node attention coefficient calculation. The node features of step 3.3 and the intermediate path features are connected by the connection operation, the LeakyReLU(·) is used as the activation function, and the normalized features are obtained by the Softmax function. The attention coefficient is obtained when aggregating the internal nodes of the hyperedge.

[0020] Step 4.3, super-edge internal node feature aggregation. The node features contained in the super-edge are weighted and summed using the attention coefficients in step 4.2, and are connected with the linearly projected intermediate path features, and the super-edge features are obtained through the Sigmoid(·) activation function.

[0021] Step 4.4, super-edge attention coefficient calculation. The linearly projected super-edge features are connected with the target node features using the connection operation, the LeakyReLU(·) is used as the activation function, and the connected features are normalized through the Softmax function to obtain the attention coefficients of the super-edge aggregation.

[0022] Step 4.5, super-edge feature aggregation. The super-edge features are weighted and summed using the attention coefficients in step 4.4, and are connected with the linearly projected center node features, and the center node features are updated through the Sigmoid(·) activation function.

[0023] Step 4.6, multi-head attention mechanism. Repeat steps 4.2 to 4.5 T times (generally take 4 to 8 times), and connect the obtained T features as the feature f i .

[0024] Step 5, node type prediction module.

[0025] Step 5.1, node type prediction. A multi-class classifier is constructed, and the vector f i is input into the classifier, and the output result is used to predict the node type. The construction of the classifier is generally a multilayer perceptron (MLP) followed by a Sigmoid(·) function, and the output of the classifier is:

[0026] y = Sigmoid(MLP(f)), (1)

[0027] where f is the node feature, and y is the output of the classifier, which can be used to predict the node type.

[0028] Step 5.2, single-task loss function. The model can only use the target node for classification prediction, and the cross-entropy loss function is used as the loss function, and the formula is:

[0029]

[0030] where is the score value in the range of [0, 1] predicted by the target node classifier for each type, and y i is the true type label of the target node.

[0031] Step 5.3, multi-task loss function. The model can use all types of nodes for classification prediction, and the node features on the intermediate path are obtained one by one through the feature fusion method based on element-wise mean pooling (EMP). EMP is to take the mean of the corresponding dimensions of two feature vectors to fuse, represents the feature fusion process, and the calculation process can be represented as:

[0032]

[0033] wherein is the feature of the intermediate node k of type b, e k represents the hyperedge composed of the nodes connected to the intermediate node k, after obtaining all the node features, cross-entropy is used as the loss function, and the formula is:

[0034]

[0035] wherein represents the node type set, is the node set of type .

[0036] In step 2.2, the construction of the meta-path-based hypergraph is specific to:

[0037] For the meta-path that connects more than two nodes in the heterogeneous graph at the same time, the hypergraph can directly represent the multi-element relationship between the nodes. We construct the meta-path-based hypergraph by the following steps:

[0038] (1) Place multiple end nodes connected by the same meta-path instance in the same hyperedge, and encode the intermediate nodes of the meta-path except the end nodes as a single feature.

[0039] (2) Change the meta-path type to construct a hypergraph based on different types of meta-paths.

[0040] In step 3.1, the calculation of the simple graph neighbor node attention coefficient is specific to:

[0041] First, use the connection information between nodes in the adjacency matrix to connect the feature of each node in the simple graph with the feature of its neighbor node, and then use a one-dimensional attention convolution kernel q to convolve the connected features. Through ReLU(·) as the activation function, the similarity η ij between the center node i and the neighbor node j is obtained. The size of the similarity reflects the contribution of the neighbor node to the center node feature in the aggregation process. In order to avoid gradient explosion and speed up the training speed, the Softmax(·) function is used to process the similarity η ijThe normalized attention coefficient of the neighbor node is obtained, and the size thereof reflects the importance of different neighbor nodes in the feature aggregation process.

[0042] In step 4.1, the intermediate path feature encoding is performed, specifically as follows:

[0043] The node features are sequentially encoded according to the order of the intermediate path by using the relationship rotation encoder RotatE, so as to encode the entire sequence structure information into a single vector. For an intermediate path P(t0,t n )={t0,…,t n-1 ,t n}, r i represents the relationship vector between nodes t i-1 and t i , and RotatE uses an iterative encoding process to obtain the encoded features, and the calculation process can be represented as:

[0044]

[0045]

[0046]

[0047] wherein f represents the feature of node i, represents the Hadamard product, and h is the encoded feature of the intermediate path.

[0048] In step 4.3, the hyperedge attention coefficient is calculated, specifically as follows:

[0049] First, the coefficient matrices W n and W e are defined for the target node and the related hyperedge respectively, and the matrix values can be learned through back propagation. The feature f i of the target node and the feature h j of the hyperedge are projected through the coefficient matrices respectively, and then connected through the concatenation operation, and the similarity γ ij between the target node i and the related hyperedge j is obtained through the LeakyReLU(·) as the activation function. The size of the similarity reflects the contribution of the related hyperedge to the target node feature in the aggregation process. In order to avoid gradient explosion and speed up the training speed, the similarity γ ij is normalized by using the Softmax(·) function, and the calculation process can be represented as:

[0050]

[0051] wherein ε ia is a trainable weight vector.

[0052] In step 5.3, the multi-task loss function is specifically:

[0053] By aggregating the multi-level graph attention network, we get the final target node feature vector f i To involve the features of other types of nodes in training, a multi-task loss function is established, and the features of other types of nodes are obtained through an element-wise mean pooling (EMP) based feature fusion method. The pooling process is:

[0054]

[0055] As the input of node classification, the node type is predicted. This method will construct a multi-class classifier, and the vector is input into the classifier, and the value of each dimension of the output vector is the probability value of the current node being of this type. According to the probability distribution, the prediction result of the node type can be output.

[0056] The construction of the classifier adopts a multi-layer perceptron (MLP) followed by a sigmoid function classifier. This classifier is suitable for multi-classification problems. The nodes in the data set can have only one class, and a node can also correspond to multiple types. The output of the classifier is:

[0057] y = Sigmoid(MLP(f)), (8)

[0058] The loss function adopts the cross-entropy function, and the formula is:

[0059]

[0060] wherein represents the node type, represents the number of nodes of this type. Among them, is the score value in the range of [0, 1] predicted by the target node classifier for each type.

[0061] Advantages

[0062] ​The application provides a heterogeneous graph representation learning method based on a multi-level graph attention network.

[0063] 1. The supergraph structure based on the meta path is introduced in the field of heterogeneous graph representation learning, which effectively preserves the non-pair relationship in the heterogeneous graph and provides additional semantic information.

[0064] 2. By sequentially aggregating the features of simple graphs and supergraphs, the model has multiple receptive fields, which is beneficial to simultaneously obtaining local and global information in the graph, enhancing the ability to mine complex semantic relationships in the heterogeneous graph, and improving the quality of node representation vectors.

[0065] 3. In the node classification task of the heterogeneous graph, the model is trained using the prediction results and label values of different types of nodes, which effectively improves the robustness of the model, significantly reduces the risk of overfitting in the model training process, and significantly improves the classification accuracy compared with existing methods.

[0066] Compared with the prior art, the application has the following beneficial effects:

[0067] In the process of heterogeneous graph representation learning, compared with the meta path sequence model used by the traditional method, the supergraph structure based on the meta path can completely preserve the multi-element relationship information in the heterogeneous graph, and improve the learning ability of the model for complex semantics. In addition, the introduction of the supergraph enables the model to well obtain the global information in the graph. In contrast, the traditional method is mostly limited to node local neighbor information. The experimental results on the standard data set prove that the model achieves better performance than existing methods. BRIEF DESCRIPTION OF DRAWINGS

[0068] The accompanying drawings are included to provide a further understanding of the application and are incorporated in and constitute a part of this specification, illustrate embodiments of the application and together with the description serve to explain the principles of the application. In the drawings:

[0069] Figure 1 Heterogeneous graph schematic diagram involved in the examples of the present application.

[0070] Figure 2 Meta-path-based hyperedge construction process of the method described in the examples of the present application.

[0071] Figure 3 Overall model framework diagram of the method described in the examples of the present application.

[0072] Figure 4 Simple graph feature aggregation module and hypergraph feature aggregation module in the examples of the present application. DETAILED DESCRIPTION

[0073] In order to make the objects, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in combination with the drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and not to limit the present application. In addition, the technical features involved in each embodiment of the present application described below can be combined with each other as long as they do not conflict with each other.

[0074] The present application is a heterogeneous graph representation learning method based on meta-path multi-level graph attention network. The model consists of four sub-modules: a meta-path-based multi-level graph construction module, a simple graph feature aggregation module, a hypergraph feature aggregation module, and a node type prediction and multi-task training module. The overall model schematic diagram is shown in Figure 3 as follows:

[0075] 1. Meta-path-based multi-level graph construction

[0076] First, determine the target node type, and then give multiple types of meta-paths. The definition of meta-path requires partial domain knowledge to ensure that it is meaningful in semantics. Take the citation heterogeneous graph DBLP (as shown in Figure 1 ) as an example, which contains three types of nodes: author (A), paper (P), and conference (V); if the target node is determined to be a paper, the meaningful meta-path definition methods include P-A-P and P-V-P, etc. Use meta-paths to extract the pair-wise relationships in the heterogeneous graph as neighbor nodes, thereby constructing a simple graph based on meta-paths.

[0077] Meta-paths can also connect more than two target nodes at the same time. These target nodes together with the intermediate paths form a complete semantic relationship, and the hypergraph is used to directly represent the multi-element relationship between the target nodes. The present application constructs a hypergraph based on meta-paths through the following steps:

[0078] (1) The same meta-path instance connects multiple target nodes in the same hyper-edge, and the intermediate nodes of the meta-path except the target nodes are encoded as a single feature.

[0079] (2) Change the meta-path type to construct a hypergraph based on different types of meta-paths.

[0080] 2. Simple graph feature aggregation module

[0081] First, the attention coefficient between the neighbor nodes is calculated. Using the connection information between the nodes in the adjacency matrix, the feature of each node in the simple graph is connected with the feature of its neighbor node, and then a one-dimensional attention convolution kernel q is used to convolve the connected features, and ReLU(·) is used as the activation function to obtain the similarity η ij between the center node i and the neighbor node j. The size of the similarity reflects the contribution of the neighbor node to the center node feature in the aggregation process. In order to avoid gradient explosion and speed up the training, the similarity η ij is normalized by using the Softmax(·) function to obtain the attention coefficient α of the neighbor node, which reflects the importance of different neighbor nodes in the feature aggregation process.

[0082] The corresponding neighbor node features (including the center node itself) are weighted and summed using the attention coefficient α In order to reduce the bias of the attention process and make the learning process more stable, a multi-head attention mechanism is adopted, and the above attention process is repeated T times to generate the center node representation by connecting the center node features.

[0083] 3. Hypergraph feature aggregation

[0084] In the simple graph feature aggregation module, the preliminary node representation is obtained, but it only contains local pairwise information. In order to further fuse multi-element relationship and intermediate path information, the simple graph aggregation node representation is taken as input for hypergraph feature aggregation, and the overall framework of hypergraph feature aggregation is shown in Figure 4 .

[0085] 3.1 Hyperedge internal node feature aggregation

[0086] The intermediate path information is needed in the hyperedge internal aggregation process, and the relationship rotation encoder RotatE is used to encode the node features in order according to the intermediate path sequence, and the entire sequence structure information is encoded into a single vector. For an intermediate path P(t0,t n )={t0,…,t n-1 ,t n}, r i represents the distance between nodes t i-1 and ti RotatE uses an iterative encoding process to obtain the encoded features, the calculation process of which can be represented as:

[0087]

[0088]

[0089]

[0090] wherein represents the feature of node i, and represents the Hadamard product, is the encoded feature of the intermediate path. Using the preliminary node representation aggregated by the simple graph features and the encoded intermediate path features as input, the multi-head attention mechanism is used to calculate the weight and aggregate to obtain the representation vector of the current hyperedge.

[0091] 3.2 Hyperedge feature aggregation

[0092] For any node i in the hypergraph, it can be contained in multiple hyperedges j e ε i . First, define the coefficient matrix W n and W e for the target node and the related hyperedge, respectively. The matrix values can be learned through backpropagation, and the target node feature f i and the hyperedge feature h j are projected through the coefficient matrix, respectively, and then connected through the concatenation operation. Through LeakyReLU(·) as the activation function, the similarity γ ij between the target node i and the related hyperedge j is obtained. The size of the similarity reflects the contribution of the related hyperedge to the target node feature in the aggregation process. In order to avoid gradient explosion and speed up the training speed, the similarity γ ij is normalized using the Softmax(·) function, and the calculation process can be represented as:

[0093]

[0094] γ j = LeakyReLU(W e · h j || W n · f i ). (13)

[0095] wherein ε i represents the related hyperedge set of the target node i, and a is a trainable weight vector. The attention coefficient β ijThe super-edge features are weighted and summed, and are connected with the center node features after linear projection, and the center node features are updated through a Sigmoid(·) activation function. The multi-head attention mechanism is used to connect the T features of the same node obtained to the target node feature f i .

[0096] 4. Node type prediction and multi-task training

[0097] 4.1 Node type prediction

[0098] The present application uses node classification as a model semi-supervised learning task. A multi-class classifier is constructed, and the vector f is input into the classifier, and the output result is used to predict the node type. The construction of the classifier uses a Multilayer Perceptron (MLP) followed by a Sigmoid(·) function, and according to different data, the model can be applicable to the case where a node has only a single type, or the case where a node corresponds to multiple types. The output of the classifier is:

[0099] y = Sigmoid(MLP(f)). (14)

[0100] Where y is the output of the classifier, which is a prediction vector, each dimension corresponds to a node type, and the value of each dimension represents the prediction probability of the node in this type.

[0101] 4.2 Single-task training

[0102] The model can only use the target node for type prediction, that is, only the predicted value of the target node and the actual label participate in the loss function calculation. Compared with multi-task training, single-task training does not require the labels of all type nodes in the heterogeneous graph, and at the same time, due to the absence of the pooling process, the training rate will be improved to a certain extent, and it is suitable for the case where the node labels are incomplete. The loss function adopts a cross-entropy loss function, and the formula is:

[0103]

[0104] Where is the score value of the target node classifier for each type prediction in the range of [0, 1], and y i is the type label of the target node.

[0105] 4.3 Multi-task training

[0106] For the label-complete graph data, the model can use all types of nodes for classification prediction. By using an element-wise mean pooling (EMP) based feature fusion method, the node features on the intermediate path are obtained one by one. The purpose of this is to generate the representation of other types of nodes through the target node representation. EMP is to fuse two feature vectors by averaging the corresponding dimensions, The feature fusion process is represented, and the calculation process can be represented as:

[0107]

[0108] wherein is the intermediate node feature of type b, after obtaining the features of all types of nodes, the cross-entropy is also used as the loss function, and the loss values of different types of nodes are summed, and the formula is:

[0109]

[0110] wherein represents the node type, represents the number of nodes of the type.

[0111] 5. Experiment and result

[0112] 5.1 Training design

[0113] Before training the model, the construction of the simple graph and hypergraph based on the meta-path has been completed, and the feature vectors of different types of nodes are projected into the same feature space.

[0114] The model parameters are randomly initialized, and the random number used for initialization is fixed by setting the random seed every time the training is performed. During the training process, after completing an epoch of training, the model will be verified on the validation set. If the current epoch result on the validation set is better than the previous epoch result, the model parameters obtained by the current epoch training will replace the previous parameters. The optimization algorithm of the model adopts AdaMax, and the optimization algorithm parameters are all taken as default values. The model uses a multi-task cross-entropy function as the loss function for end-to-end learning, and the formula of the multi-task cross-entropy loss function is:

[0115]

[0116] The prediction value of the classifier for the node on each type is a score value in the range of [0, 1]. In order to prevent overfitting phenomenon in the model training, the present application adds a Dropout layer in the model, and performs gradient clipping in the back propagation to prevent the phenomenon of gradient vanishing or explosion.

[0117] 5.2 Experimental results

[0118] The evaluation of the prediction effect of the model uses F1-score as the model precision measurement index, which takes into account the accuracy (Precision) and recall (Recall) of the classification model. In the multi-classification problem, there are two ways to calculate F1-score, namely micro-F1 and macro-F1, and the calculation formulas are as follows:

[0119]

[0120]

[0121] where N represents the number of node types, Precision i and Recall i respectively represent the accuracy and recall of the i-th type.

[0122] As shown in Table 1, the node classification results of the model under different training rates. The average F1-score of the model under the training rates of "20%", "40%", "60%" and "80%" is "93.05%", "93.73%", "94.08%" and "94.22%", respectively.

[0123] Table 1

[0124]

[0125] Table 2 is the comparison result of F1-score of the model of the present application and other papers. GCN, DHNE, HGT, HAN and MAGNN are experimental results of different models on the same data set. Ours represents the model proposed in the present application, and it can be seen that the performance of the model of the present application has been greatly improved compared with the baseline model.

[0126] Table 2

[0127]

[0128] The above is only a specific embodiment of the present application, but the protection scope of the present application is not limited to this. Any skilled person in the art can easily think of various equivalent modifications or replacements within the technical range disclosed in the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A heterogeneous graph representation learning method based on a meta-path multi-level graph attention network, characterized in that, The heterogeneous graph is a DBLP-based citation heterogeneous graph, containing three types of nodes: author A, paper P, and conference V, and the meta-path includes P-A-P and P-V-P; the target node is a paper; the steps are as follows: Step 1: Spatial mapping of node attribute features Step 1.1: Assign different linear transformation matrices to different types of nodes to map the attribute features of all nodes into the same feature space; Step 2: Construction of multi-level graph structure based on meta-path Step 2.1: Construction of a simple graph based on a meta-path: extract pairs of nodes in the citation heterogeneous graph DBLP through a meta-path, and combine all node pairs to form a simple graph based on a meta-path; change the meta-path type to define multiple simple graphs; Step 2.2: Construction of a hypergraph based on a meta-path: place multiple nodes connected by the same meta-path in the same hyperedge, and construct all hyperedges to form a hypergraph based on a meta-path; change the meta-path type to define multiple hypergraphs; Step 3: Simple graph feature aggregation module Step 3.1: Simple graph neighbor node attention coefficient calculation: concatenate the features of the target node with its first-order neighbor nodes using the connection operation, and use As the activation function, by The function normalizes the concatenated features to obtain the attention coefficients of each node when aggregating node features. Step 3.2: Simple graph node feature aggregation: Use the corresponding attention coefficient in step 3.1 to weight and sum the target node and neighbor node features to obtain the target node features; Step 3.3: Multi-head attention mechanism: repeat Step 3.1 and Step 3.2 Next, the same nodes of the resulting features are concatenated into one feature; Step 4: Hypergraph feature aggregation module Step 4.1: Intermediate path feature encoding: using a relation rotation encoder Encode the features of the intermediate nodes in each meta-path to obtain the intermediate path features; Step 4.2: Hyperedge internal node attention coefficient calculation: concatenate the node features of step 3.3 with the intermediate path features using As the activation function, the concatenated features are normalized by function to obtain the attention coefficient when aggregating the hyperedge internal nodes. Step 4.3: Hyper-edge internal node feature aggregation: Weighted sum the node features contained in the hyper-edge using the attention coefficients in step 4.2, and concatenate with the linear projected intermediate path features, pass through an activation function, to obtain the hyper-edge feature; Step 4.4: Hyperedge attention coefficient calculation: concatenate the linearly projected hyperedge features with the target node features using the concatenate operation, and use As the activation function, by the function normalizes the concatenated features to obtain the attention coefficients when aggregating hyperedges; Step 4.5: Super-edge feature aggregation: Weighted sum the super-edge features using the attention coefficients in Step 4.4, and concatenate with the linear projected center node features, update the center node features through activation function. Step 4.6: Update the attention coefficients by the following equation: where is the attention coefficient of the edge between node i and node j in the k-th iteration, is the attention coefficient of the edge between node i and node j in the (k-1)-th iteration, is the super-edge feature of the edge between node i and node j in the k-th iteration, is the Step 4.6: Multi-head attention mechanism: repeat steps 4.2 to 4.5 Next, the resulting features are concatenated as the target node features ; Step 5: Node type prediction module Step 5.1: Node type prediction: A multi-class classifier is constructed, and the vector is input into the classifier, and the output is used to predict the node type; the classifier is constructed as a multi-layer perceptron followed by function, and the output of the classifier is: wherein is a node feature, is an output of the classifier for predicting a node type; Step 5.2: Single-task loss function: the model only uses the target node for classification prediction, and the loss function uses the cross-entropy loss function, the formula is: wherein is the range of scores predicted by the target node classifier for each type score value, is the true type label of the target node; Step 5.3: Multi-task loss function: The model can use all types of nodes for classification prediction, and the node features on the intermediate path are obtained one by one through the feature fusion method based on the element-wise average pooling model; EMP is to fuse two feature vectors by averaging the corresponding dimensions, represents the feature fusion process, and the calculation process is represented as: wherein is a feature of an intermediate node of type represents a hyperedge of nodes connected to the intermediate node After obtaining all node features, cross-entropy is used as the loss function, and the formula is:​​ wherein represents a set of node types, is a set of nodes of type .

2. The method of claim 1, wherein the method comprises: In step 2.2, the construction of a hypergraph based on a meta-path, specifically: For a meta-path that connects two or more nodes in the citation heterogeneous graph, a hypergraph can directly represent the multi-element relationship between nodes; the hypergraph based on a meta-path is constructed by the following steps: (1) Place multiple end nodes connected by the same meta-path instance in the same hyperedge, and encode the intermediate nodes of the meta-path except the end nodes as a single feature; (2) Change the meta-path type to construct a hypergraph based on different types of meta-paths.

3. The method of claim 1, wherein the method comprises: In step 3.1, the simple graph neighbor node attention coefficient calculation, specifically: First, using the connection information between nodes in the adjacency matrix, the features of each node in the simple graph are connected to the features of its neighboring nodes. Then, a one-dimensional attention convolution kernel is used. Convolution is performed on the connection features, through As an activation function, the center node is obtained. with neighboring nodes Similarity between The similarity score reflects the contribution of neighboring nodes to the features of the central node during the aggregation process. To avoid gradient explosion and speed up training, use Functions on similarity After normalization, the attention coefficients of neighboring nodes are obtained, and their magnitudes reflect the importance of different neighboring nodes during feature aggregation.

4. The method of claim 1, wherein the method comprises: In step 3.3 Take 4 to 8 times.

5. The method of claim 1, wherein the method further comprises: In step 4.1, the intermediate path feature encoding, specifically: Using relational rotary encoders The node features are encoded in turn according to the intermediate path order, thereby encoding the entire sequence structure information into a single vector; for an intermediate path , denotes the relation vector between the node and , The encoded features are obtained using an iterative encoding process, the calculation process of which is represented as: wherein represents a node characteristic, represents a Hadamard Product, is an intermediate path encoded feature.

6. The method of claim 1, wherein the method further comprises: In step 4.3, the hyperedge attention coefficient calculation, specifically: First, define coefficient matrices for the target node and related hyperedges. and The matrix values ​​are learned through backpropagation, which incorporates the features of the target node. and hyperedge features Each part is projected through the coefficient matrix and then connected by the concatenation operation. As an activation function, the target node is obtained. With related hyperedges Similarity between The similarity score reflects the contribution of related hyperedges to the features of the target node during the aggregation process; to avoid gradient explosion and speed up training, the similarity score is used... Functions on similarity The normalization process is performed, and the calculation process is expressed as follows: wherein represents a set of related hyper-edges of a target node , is a trainable weight vector.

7. The method of claim 1, wherein the method further comprises: In step 5.3, the multi-task loss function, specifically: Through the aggregation of the multi-level graph attention network, the final target node feature vector is obtained In order to make the features of other types of nodes participate in training, a multi-task loss function is established, and the features of other types of nodes are obtained through a feature fusion method based on an element-wise average pooling model. The pooling process is: Will The method uses the vector as input for node classification to predict node types; it constructs a multi-class classifier that uses vectors... The data is input into the classifier, and the value of each dimension of the output vector is the probability value of the current node being of that type. Based on the probability distribution, the prediction result of the node type can be output. The classifier is constructed using a multilayer perceptron MLP followed by a Sigmoid function classifier, which can be applied to multi-classification problems, and the nodes in the dataset have only one class, and also a node corresponds to multiple types; the output of the classifier is: The loss function uses the cross-entropy function, the formula is: wherein represents the node type, represents the number of nodes under the type; wherein is the range of score values predicted by the target node classifier for each type.

Images