Heterogeneous attribute network representation learning method based on high-order message passing

By adopting a high-order message delivery mechanism in heterogeneous attribute networks, adaptively aggregating and fusing remote messages, the difficulty of heterogeneous information capture and remote message acquisition in heterogeneous network representation learning is solved, and the accuracy of node classification is improved and Indicates learning ability.

CN119030846APending Publication Date: 2024-11-26CHINA UNIV OF MINING & TECH
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202410996524.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-24
Publication Date
2024-11-26

AI Technical Summary

Technical Problem

Existing heterogeneous network representation learning methods have difficulties in capturing heterogeneous information in heterogeneous networks and acquiring remote messages, resulting in poor representation learning effect and poor metapathic flexibility relying on manual definitions.

Method used

Using a heterogeneous attribute network representation learning method based on advanced message delivery, the remote messages are adaptively aggregated and fused to generate the final representation of the node through heterogeneous linear transformation, network mode-based message delivery mechanism and advanced message attention mechanism. .

Benefits of technology

It significantly improves the distinctive power and representation learning ability of node classification, avoids dependence on manually defined metapaths, and improves the flexibility and universality of the model.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119030846A_ABST
    Figure CN119030846A_ABST
Patent Text Reader

Abstract

The invention discloses a heterogeneous attribute network representation learning method based on high-order message passing, and the method comprises the steps: mapping node features of different dimensions to a shared feature space of the same dimension through linear transformation for a given heterogeneous attribute network; for different nodes and edge types in the direct neighborhood of the target type node, adopting a heterogeneous message passing mechanism based on a network mode and having a double-layer aggregation function, and performing message aggregation according to different node types and edge types; for neighborhood messages of different orders of target type nodes, adaptively allocating message weights of different orders based on a high-order message attention mechanism, and then performing weighted fusion on the messages of different orders to generate final representation of the nodes; a single-layer linear transformation is used as a classifier, and target nodes are assigned into a particular class or category according to the learned node representation. According to the method, the node features and semantic information in the heterogeneous network can be effectively captured, and the distinguishing power of node classification can be remarkably improved.
Need to check novelty before this filing date? Find Prior Art

Description

[0001] Technology Neighborhood

[0002] The present invention belongs to the field of heterogeneous network representation learning technology, and specifically is a heterogeneous attribute network representation learning method based on high-order message passing. Background Art

[0003] With the rapid development of complex networks in the real world, heterogeneous networks have become a common paradigm for describing diversity and complexity. Heterogeneous networks are composed of multiple types of nodes and edges, which are used to represent and propagate different types of information. Nodes in heterogeneous networks represent different types of entities, while edges represent multiple relationships between entities. Representation learning for nodes in heterogeneous networks aims to map nodes and edges to low-dimensional vector spaces while capturing semantic information and relationships between nodes and edges. Traditional graph neural networks face the following problems when dealing with heterogeneous networks: First, there are difficulties in capturing heterogeneous information in heterogeneous networks, resulting in poor representation learning effects. Second, when obtaining remote messages, it is necessary to manually extract meta-paths, which leads to insufficient remote message acquisition.

[0004] In response to the above problems, existing heterogeneous network representation learning methods can be roughly divided into two categories: meta-path-based methods and meta-path-free methods. Meta-path-based methods use pre-defined meta-paths to capture and encode structural and semantic relationships in heterogeneous networks, but this method relies on manually defined meta-paths and has poor flexibility. Meta-path-free methods are more flexible in architectural design, do not require strict meta-path guidance, and can adaptively learn new graph structures based on data and tasks, but this method has limitations in acquiring remote messages. In order to effectively solve the above problems, it is urgent to provide a representation learning method that can adaptively acquire and fuse messages from remote neighbors, thereby improving the model's representation learning ability and node classification accuracy. Summary of the invention

[0005] In view of the problems existing in the above-mentioned prior art, the present invention provides a heterogeneous attribute network representation learning method based on high-order message passing. The method does not rely on manually defined meta-paths, has good flexibility and strong versatility, can adaptively aggregate messages from remote locations, and effectively capture node features and semantic information in heterogeneous networks, thereby significantly improving the discriminatory power of node classification.

[0006] In order to achieve the above object, the present invention provides a heterogeneous attribute network representation learning method based on high-order message passing, comprising the following steps:

[0007] Step 1: Build a heterogeneous graph neural network model based on high-order message passing;

[0008] S11: Build a heterogeneous graph neural network model, and make the heterogeneous graph neural network model include a heterogeneous linear transformation module, a heterogeneous message passing module, a high-order message passing module and a classifier module; wherein the heterogeneous linear transformation module is used to map node features of different dimensions into a shared feature space of the same dimension; the heterogeneous message passing module adopts a message passing mechanism based on a network model, and introduces an attention mechanism to adaptively learn the weights of messages from different types of neighbor nodes, and then combines the feature representations from the previous layer to update the feature representations of nodes in the network; the high-order message passing module introduces an attention mechanism based on high-order messages, which is used to adaptively assign weights of messages of different orders according to the different effects of remote neighbors of different orders on the target node, and then weightedly fuse messages of different orders to generate the final representation of the node; the classifier module is used to adopt a single-layer linear transformation method to assign the target node to a specific class or category according to the learned node representation;

[0009] S12: Divide the selected heterogeneous attribute network benchmark dataset into a training set, a validation set, and a test set according to a set ratio;

[0010] S13: Use the training set to train the heterogeneous graph neural network model to obtain a trained heterogeneous graph neural network model; during the training process, the model parameters are continuously updated through the back propagation algorithm to reduce the value of the loss function, and the optimal parameters are selected through the verification set results;

[0011] S14: Use the test set to test the trained heterogeneous graph neural network model to obtain a heterogeneous graph neural network model based on high-order message passing;

[0012] Step 2: Input a given heterogeneous attribute network and distinguish the nodes in the network;

[0013] S21: perform heterogeneous linear transformation;

[0014] For a given heterogeneous attribute network, the heterogeneous linear transformation module maps the original input features of different types of nodes to a shared feature space through a specific type of linear transformation;

[0015] S22: Perform heterogeneous message transmission based on network mode;

[0016] For different node and edge types in the direct neighborhood of the target type node, the heterogeneous message passing module adopts a network-based heterogeneous message passing mechanism with a double-layer aggregation function to aggregate messages according to different node types and edge types. For messages at the node type level, the first layer generates features of specific node types by mean aggregation. For messages at the relationship type level, the second layer adaptively assigns different weights to messages from different types of nodes based on the attention mechanism, and then combines the feature representations from the previous layer to update the node feature representations in the network.

[0017] S23: Perform high-order message passing based on high-order message attention mechanism;

[0018] For the remote neighborhood messages of different orders of the target type node, the high-order message passing module adaptively aggregates the messages from the remote neighborhood according to the characteristics of all neighboring nodes of each node through the high-order message attention mechanism, and assigns weights to messages of different orders, and then performs weighted fusion on messages of different orders to generate the final representation of the node;

[0019] S24: Use the classifier to classify nodes;

[0020] The classifier module uses a single-layer linear transformation as a classifier to assign target nodes to specific classes or categories based on the learned node representations.

[0021] As a preferred embodiment, in the heterogeneous linear transformation process in step 2 S21, the output feature H after linear transformation is obtained according to formula (1): o ;

[0022] H o =X o Θ o (1);

[0023] Where o∈O is the node type in the heterogeneous attribute network, X o represents the original input feature of node type o, Θ o is the learnable weight parameter of node type o.

[0024] As a preferred embodiment, the process of updating the feature representation of nodes in the network in step 2 S22 is as follows:

[0025] S221: Obtain the representation of the neighbor node r∈R∪{tt} of the target node type t according to formula (2);

[0026]

[0027] In the formula, Represents the adjacency matrix corresponding to the current relationship subgraph; s∈O is a node type instance, Hs Represents the output feature of node s after linear transformation; Θ r is the learnable parameter of the sub-relation type r;

[0028] S222: Obtain the hidden layer representation of the node whose target node type is t∈O according to formula (3);

[0029]

[0030] In the formula, r1, r2, …, r n ∈R is a relation type instance.

[0031] As a preferred embodiment, in step 2 S23, the final representation of the target node is obtained according to formula (4);

[0032]

[0033] In the formula, is the output feature of the target node itself; They are the feature representations of the different order neighborhoods of the target node.

[0034] As a preferred embodiment, in step 1 S13, the loss function The acquisition process is as follows:

[0035] S131: Obtain the predicted label of the target node according to formula (5)

[0036]

[0037] In the formula, Z t is the output feature of the target type node learned by the last layer of the graph neural network; Θ c is a learnable parameter in the classifier;

[0038] S132: Obtain the target node label probability normalized by softmax according to formula (6)

[0039]

[0040] In the formula,

[0041] S13: Obtain the loss function value of the model according to formula (7) And optimize the training of the entire model by minimizing the loss function;

[0042]

[0043] In the formula, Represents some nodes with labels; yic ∈Y is the target node v i The true label of is the target node v i The predicted label of .

[0044] The present invention proposes a representation learning method for heterogeneous attribute networks based on high-order message passing, which fully considers the diversity of heterogeneous graphs and can adaptively obtain messages from distant neighbors. First, a heterogeneous linear transformation is performed first, and the feature maps of nodes of different types can be projected into the same feature space, which can facilitate the subsequent aggregation of messages of different types of nodes in the message passing mechanism. Secondly, through the heterogeneous message passing method based on the network mode, the network topology and relational semantic information contained in the heterogeneous attribute network can be captured, thereby retaining the heterogeneity of the heterogeneous network as much as possible. Then, the adaptive high-order message attention mechanism can be used to adaptively allocate the weights of messages of different orders according to the different effects of neighbors of different orders on the target node, thereby realizing the feature fusion of neighborhood messages of different orders. In this way, not only can the remote neighborhood messages be adaptively obtained and fused, but also the importance of these messages can be distinguished, thereby significantly improving the representation learning ability of the heterogeneous network and the distinguishing power of node classification.

[0045] This method does not rely on manually defined meta-paths and has good flexibility and versatility. It can adaptively aggregate messages from remote locations and effectively capture node features and semantic information in heterogeneous networks, thereby significantly improving the discriminative power of node classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the process of the present invention;

[0047] Figure 2 is a bar diagram of the ablation experiment results in the present invention;

[0048] Figure 3 It is a line graph showing the effect of HRHGNN model depth on model performance in the present invention;

[0049] Figure 4 It is a line graph showing the effect of the propagation order on the model performance in the present invention;

[0050] Figure 5 It is the influence of hidden layer dimension on model performance in the present invention;

[0051] Figure 6 It is a visualization diagram of heterogeneous network representation in the present invention;

[0052] Figure 7 It is a visualization diagram of heterogeneous network representation that varies with model depth in the present invention;

[0053] Figure 8 It is a visualization diagram of heterogeneous network representation according to the present invention that varies with the propagation order. DETAILED DESCRIPTION

[0054] The present invention will be further described below in conjunction with the accompanying drawings.

[0055] like Figure 1 As shown, the present invention provides a heterogeneous attribute network representation learning method based on high-order message passing, comprising the following steps:

[0056] Step 1: Build a heterogeneous graph neural network model based on high-order message passing;

[0057] S11: Build a heterogeneous graph neural network model, and make the heterogeneous graph neural network model include a heterogeneous linear transformation module, a heterogeneous message passing module, a high-order message passing module and a classifier module; wherein the heterogeneous linear transformation module is used to map node features of different dimensions into a shared feature space of the same dimension; the heterogeneous message passing module adopts a message passing mechanism based on a network model, and introduces an attention mechanism to adaptively learn the weights of messages from different types of neighbor nodes, and then combines the feature representations from the previous layer to update the feature representations of nodes in the network; the high-order message passing module introduces an attention mechanism based on high-order messages, which is used to adaptively assign weights of messages of different orders according to the different effects of remote neighbors of different orders on the target node, and then weightedly fuse messages of different orders to generate the final representation of the node; the classifier module is used to adopt a single-layer linear transformation method to assign the target node to a specific class or category according to the learned node representation;

[0058] S12: Divide the selected heterogeneous attribute network benchmark dataset into a training set, a validation set, and a test set according to a set ratio;

[0059] S13: Use the training set to train the heterogeneous graph neural network model to obtain a trained heterogeneous graph neural network model; during the training process, the model parameters are continuously updated through the back propagation algorithm to reduce the value of the loss function, and the optimal parameters are selected through the verification set results;

[0060] S14: Use the test set to test the trained heterogeneous graph neural network model to obtain a heterogeneous graph neural network model based on high-order message passing;

[0061] Step 2: Input a given heterogeneous attribute network and distinguish the nodes in the network;

[0062] S21: perform heterogeneous linear transformation;

[0063] For a given heterogeneous attribute network, the heterogeneous linear transformation module maps the original input features of different types of nodes to a shared feature space through a specific type of linear transformation;

[0064] S22: Perform heterogeneous message transmission based on network mode;

[0065] For different node and edge types in the direct neighborhood of the target type node, the heterogeneous message passing module adopts a network-based heterogeneous message passing mechanism with a double-layer aggregation function to aggregate messages according to different node types and edge types. For messages at the node type level, the first layer generates features of specific node types by mean aggregation. For messages at the relationship type level, the second layer adaptively assigns different weights to messages from different types of nodes based on the attention mechanism, and then combines the feature representations from the previous layer to update the node feature representations in the network.

[0066] S23: Perform high-order message passing based on high-order message attention mechanism;

[0067] For the remote neighborhood messages of different orders of the target type node, the high-order message passing module adaptively aggregates the messages from the remote neighborhood according to the characteristics of all neighboring nodes of each node through the high-order message attention mechanism, and assigns weights to messages of different orders, and then performs weighted fusion on messages of different orders to generate the final representation of the node;

[0068] S24: Use the classifier to classify nodes;

[0069] The classifier module uses a single-layer linear transformation as a classifier to predict the characteristics of the target node based on the learned node representation and assign the target node to a specific class or category.

[0070] As a preferred embodiment, in the heterogeneous linear transformation process in step 2 S21, the output feature H after linear transformation is obtained according to formula (1): o ;

[0071] H o =X o Θ o (1);

[0072] Where o∈O is the node type in the heterogeneous attribute network, X o represents the original input feature of node type o, Θ o is the learnable weight parameter of node type o.

[0073] As a preferred embodiment, the process of updating the feature representation of nodes in the network in step 2 S22 is as follows:

[0074] S221: Obtain the representation of the neighbor node r∈R∪{tt} of the target node type t according to formula (2);

[0075]

[0076] In the formula, Represents the adjacency matrix corresponding to the current relationship subgraph; s∈O is a node type instance, H s represents the output feature of node s after linear transformation; Θr is the learnable parameter of sub-relation type r;

[0077] S222: Obtain the hidden layer representation of the node whose target node type is t∈O according to formula (3);

[0078]

[0079] In the formula, r1, r2, …, r n ∈R is a relation type instance.

[0080] As a preferred embodiment, in step 2 S23, the final representation of the target node is obtained according to formula (4);

[0081]

[0082] In the formula, is the output feature of the target node itself; They are the feature representations of the different order neighborhoods of the target node.

[0083] As a preferred embodiment, in step 1 S13, the loss function The acquisition process is as follows:

[0084] S131: Obtain the predicted label of the target node according to formula (5)

[0085]

[0086] In the formula, Z t is the output feature of the target type node learned by the last layer of the graph neural network; Θ c is a learnable parameter in the classifier;

[0087] S132: Obtain the target node label probability normalized by softmax according to formula (6)

[0088]

[0089] In the formula,

[0090] S13: Obtain the loss function value of the model according to formula (7) And optimize the training of the entire model by minimizing the loss function;

[0091]

[0092] In the formula, Represents some nodes with labels; y ic ∈Y is the target node v i The true label of is the target node v i The predicted label of .

[0093] Verification process:

[0094] 1. Comparison model:

[0095] In order to verify the effectiveness of the method in this application, we first selected competitive attribute network representation learning baseline methods, including GNNs and HGNNs. We conducted node classification experiments and visualization experiments on the heterogeneous attribute network benchmark dataset. The specific comparison models are as follows:

[0096] GCN is a classic network representation learning method with homogeneous attributes, which proposes a simple message passing mechanism for running neural network models directly on network data.

[0097] GAT introduces a local neighborhood attention mechanism, which assigns different importance to different nodes in the direct neighborhood.

[0098] RGCN is an extension of GCN on heterogeneous attribute networks. It can run on complex relational data and is widely used in heterogeneous attribute network representation learning.

[0099] In order to establish a heterogeneous model, HGT designs parameters related to node and edge types to obtain heterogeneous attention on each edge.

[0100] HetGTCN introduces the tree model into heterogeneous graph neural network, which can encode the heterogeneity of node features and graph structure without manually extracting meta-paths.

[0101] HetGTAN further introduces the attention mechanism and explores three different aggregation functions: semantic attention, mean, and weighted function with learnable parameters.

[0102] 2. Dataset:

[0103] When performing node classification tasks, three public heterogeneous attribute network benchmark datasets are used, including two academic datasets (ACM, DBLP) and one movie dataset (IMDB). The academic datasets consist of different types of nodes such as papers, authors, and topics, while the movie dataset consists of three types of nodes: actors, movies, and directors. Table 5-1 summarizes the statistical information of these datasets.

[0104] Table 5-1: Dataset details

[0105]

[0106] The ACM dataset is obtained from the ACM Digital Library and includes representative datasets of computer science conferences. ACM can use three object types to model heterogeneous network patterns: paper (P), author (A), and topic (S). The paper nodes in the dataset are annotated with three research neighborhoods, including data mining, databases, and computer networks. The attributes of the paper nodes are bag-of-words representations of their keywords. The features of authors and topics are encoded as one-hot encodings.

[0107] The DBLP dataset is provided by the authors of HAN and is extracted from the four research neighborhoods of the DBLP bibliography. Then, a heterogeneous attribute network containing four object types is constructed: paper (P), author (A), conference (C), and term (T). The author (A) object is labeled into four research neighborhoods according to the conference where the author published the paper: data mining (DM), database (DB), artificial intelligence (AI), and information retrieval (IR). The attributes of the author node are described as a bag-of-words representation of the keywords of the published paper, and the attributes of other types of nodes are represented as one-hot encoding.

[0108] The IMDB dataset comes from an online database of movies. A subset of it is used to extract the relationship between actors, movies, and directors. The dataset consists of 5257 actors (A), 4278 movies (M), and 2081 directors (D). The target node movies are divided into three categories, namely action, comedy, and drama. The attribute features of the movies are their bag-of-words representation.

[0109] 3. Model training

[0110] For all comparison methods, the same dataset split (training set / validation set / test set) was used on each benchmark dataset. For fair comparison, the comparison method used the original code shared by the original author and made necessary modifications to the code in its GitHub code base. Then, the proposed HRHGNN (High-Order Message Passing for Heterogeneous Graph Neural Networks) and some necessary baseline methods were implemented using the PyTorch framework. For fair comparison, the hyperparameters of all models were adjusted individually on each dataset to make their experimental results consistent with or even better than the best settings in the original paper. As shown in Table 5-2, grid search was used to adjust the hyperparameters: learning rate {0.001, 0.01, 0.1}, weight decay {5e-2, 5e-3, 5e-5}, Dropout {0.2, 0.5, 0.8}. For fair comparison, each experimental setting was run 10 times with different random seeds and then averaged. The best hyperparameters were adjusted in the experimental run and used in all 10 runs. In addition, an early termination strategy is used on the validation set to select the model with the highest accuracy on the validation set as the optimal model for testing, and the threshold for early termination is set to 100. The optimizer is Adam and the activation function is ReLU. In the heterogeneous network node classification task, Micro-F1 and Macro-F1 are used as evaluation indicators.

[0111] Table 5-2: Hyperparameter search space

[0112]

[0113] 4. Analysis of experimental results

[0114] In this section, a comparative experiment is conducted on the proposed HRHGNN heterogeneous attribute network representation learning model to verify that the proposed method has good expression and discrimination capabilities in heterogeneous attribute networks. First, node classification tasks are performed on three heterogeneous network benchmark datasets, and two classification evaluation indicators Micro-F1 and Macro-F1 are statistically analyzed. In order to verify the effectiveness of each module for the classification task, a detailed ablation experiment is conducted. By adjusting the parameters, the parameter sensitivity of the model is analyzed. Finally, the discrimination ability of the model is verified through visualization tasks.

[0115] 4.1 Node Classification

[0116] Micro-F1 and Macro-F1 indicators were compared on three heterogeneous attribute network benchmark datasets. For fair comparison, each method was randomly run 10 times and the average was taken. The experimental results are shown in Table 5-3. HRHGNN proposed in this application achieved the best classification results, followed by HetGTAN. In the ACM data, the homogeneous attribute network representation learning methods GCN and GAT achieved good results. The heterogeneous attribute network representation learning methods RGCN, HetGTCN and HetGTAN are slightly better than GCN. The results of the HGT method on the ACM dataset are slightly worse. However, on the DBLP dataset, the results achieved by the heterogeneous attribute network representation learning methods RGCN, HGT, HetGTCN and HetGTAN far exceed those of the homogeneous attribute network representation learning methods. On the IMDB dataset, the classification results of GCN and HGT are comparable. The RGCN, HetGTCN and HetGTAN methods are slightly better than GCN and GAT.

[0117] Table 5-3: Classification results of heterogeneous network nodes

[0118]

[0119] 4.2 Ablation Experiment

[0120] In order to verify the effectiveness of each component of the proposed method, ablation experiments are further conducted on three heterogeneous network benchmark datasets. First, the proposed framework (HRHGNN) is compared with variants that remove type attention (without RMI, w / o RMI), remove high-order attention (without RMA, w / o RMA), and both (without ALL, w / o ALL) and remove attention (without Att, w / o Att).

[0121] like Figure 2As shown, the Micro-F1 indicator analysis of the experimental results. The results on the three data sets of ACM, DBLP and IMDB are basically the same, and the HRHGNN model proposed in this application has the best effect. When the high-order message attention is removed, the classification effect decreases. The relative effect decreases more significantly on the IMDB data set, which shows that messages from high-order neighborhoods enhance the model's representation ability. When a specific type of attention mechanism is removed, a greater performance decrease occurs, which shows that it is more important for downstream tasks for heterogeneous network representation learning to retain its heterogeneity. When the previous two are removed, the classification ability of the model is the worst. For the Micro-F1 indicator, the performance drop on the ACM data is smaller than that on the DBLP and IMDB data. After removing the high-order message attention, the model classification results decrease. After removing the specific type of attention, the performance also deteriorates. Finally, the two core models are removed, and the worst result is obtained. It can be seen that the core components of the method proposed in this application play an important role in the representation ability of the model.

[0122] 4.3 Parameter Sensitivity Analysis

[0123] This section studies the sensitivity of parameters during HRHGNN model training, including the number of neural network layers, the order of high-order message passing, and the dimension of the hidden layer. Parameter sensitivity analysis experiments are conducted on three benchmark datasets: ACM, DBLP, and IMDB.

[0124] First, the number of neural network layers is studied. The number of neural network layers of HRHGNN is increased from 2 to 10 layers. The experimental results are as follows: Figure 3 As shown. Generally speaking, the more layers a neural network has, the more complex and richer semantics it can capture. However, due to the associations between network data nodes, over-smoothing can easily occur. On the three datasets, the model achieved higher evaluation indicators at layers 2-4, and the corresponding model had the strongest representation ability. On the ACM and DBLP datasets, when the neural network layer data exceeded 5 layers, the model classification ability began to decline; on the IMDB dataset, the model ability dropped sharply after exceeding 3 layers. In general, when the number of neural network layers exceeds 4, the model's node distinction ability gradually decreases as the number of neural network layers increases, and eventually the model cannot distinguish the labels of nodes in the network. This is the inherent over-smoothing problem of the graph neural network model.

[0125] Secondly, we study the order of high-order message passing. When other hyperparameters are fixed, we increase the number of propagation hops (i.e., order) of the model's high-order message passing from 1 to 80. The experimental results are shown in Figure 2. Figure 4As shown in the figure. From the results of the three data sets, when the order of high-order message transmission is 5 to 7, the model performance gradually improves and eventually reaches the optimal level. Then, as the transmission order increases further, the model capability does not drop as sharply as the number of neural network layers, but decreases slowly. It can be seen that the proposed high-order message transmission mechanism helps to alleviate the over-smoothing phenomenon. When the transmission order exceeds 20 layers, the classification ability of the model begins to gradually decline, and eventually over-smoothing occurs.

[0126] Finally, the dimension of the hidden layer is studied. By changing the dimension of the node features in the hidden layer, the changes in the model classification accuracy are analyzed. The results are as follows Figure 5 As shown. It can be seen that the performance on the ACM dataset is very sensitive to the dimension of the hidden layer. When the dimensions go from 8 to 64, the node classification accuracy increases successively, and decreases when the dimensions are 128. In contrast, on DBLP and IMDB, the changes in accuracy indicators are not particularly obvious, but the overall trend is consistent. At first, as the dimension gradually increases, the performance improves; then, the performance begins to decline, which may be due to the overfitting phenomenon caused by the increase in the number of parameters. The overall inflection point occurs when the dimension is 64. Therefore, the best effect is achieved when the hidden layer dimensions of the model are all set to 64.

[0127] 4.4 Visualization

[0128] This section conducts visualization experiments. Since the proposed model involves model depth and propagation depth, different embedding representation visualization results brought about by the increasing model depth and propagation order are added, which can enable more intuitive visualization analysis. In order to more intuitively demonstrate the representation capabilities of different models, the embedding vectors on the three benchmark datasets are projected into a two-dimensional space through dimensionality reduction using the t-SNE algorithm. As shown in the figure, nodes of different colors represent different labels. Nodes of different colors are far apart, and nodes of the same color are closer, which means that this method has a higher representation capability.

[0129] (1) Embedded visualization experiments. Figure 6As shown in the figure, different models (GCN, GAT, HGT, HetGTAN and the proposed HRHGNN) are used to represent the embedding visualization on the three datasets of ACM, DBLP and IMDB. The basic model GCN shows good representation ability on the ACM data and can show three different category boundaries, but there are a large number of misclassified nodes at the intersection of the category boundaries. GAT and HGT have mixed category boundaries on the ACM dataset, while the HetGTAN model has a relatively wide category distance. In the visualization results obtained by the proposed HRHGNN model, there are obvious boundaries between the three categories, and the nodes within the category are close to each other to form clusters. Due to the complex network of the IMDB dataset, the category visualization is not ideal, and the nodes between categories are mixed. Despite this, the proposed HRHGNN can cluster the nodes of the same category when classifying the IMDB dataset. Compared with ACM and IMDB, the models on the DBLP data show ideal visualization effects. Although GCN, GAT, and HGT can show four types, there are many mixed nodes on the category boundaries. The HetGTAN and HRHGNN models performed well, showing that the categories are clustered and the category boundaries are clear. It can be seen that the proposed HRHGNN has a stronger representation ability in the visualization of heterogeneous network representation and can distinguish different types of nodes.

[0130] (2) Visualization of heterogeneous network representations as the number of neural network layers changes. In order to more intuitively study the impact of model depth on representation capabilities, the visualization results of the HRHGNN model on three benchmark datasets are shown in the figure below. Figure 7 As shown in the figure. For the three benchmark datasets, when the neural network layer is stacked with 2 and 4 layers, the output representation visualization results show obvious category aggregation and category boundaries. However, when the model depth is greater than 4 layers, the representation visualization categories are mixed and the effect gradually deteriorates. At 8 and 10 layers, the node distribution in the visualization is complex and the categories of network nodes cannot be distinguished. This is consistent with the conclusion in 4.3 that the effect of node classification task becomes worse and worse as the model depth increases.

[0131] (3) Visualization of heterogeneous network representations as the order of high-order message transmission changes. In order to more intuitively study the impact of the transmission order on the representation ability, the visualization results of the HRHGNN model on three benchmark datasets are shown in the figure below. Figure 8As shown in the figure. Compared with the effect of model depth on representation visualization, the increase in propagation order helps to enhance the representation ability of the model to a certain extent. After reaching a certain threshold of 5 to 6 (consistent with the optimal result of model depth in 4.3), the representation ability of the model gradually decreases and becomes worse and worse. On the three benchmark datasets, when the propagation order is 5, the best visualization effect is shown, the categories form obvious clusters, and the category boundaries are obvious, especially on ACM and DBLP data. When the propagation order is further increased, although the model representation ability decreases, the rate of decrease is slower than that of the model depth. When the model depth of HRHGNN is 10, the nodes of different categories in the visualization results are abnormally chaotic, while the model still shows good representation ability when the propagation order is 10. When it is greater than 20 layers, the representation ability of the model deteriorates. Although DBLP can still be classified at 80 layers, the ACM dataset has become mixed.

[0132] It can be seen that through the analysis and comparison of the visualization results from three angles, the proposed HRHGNN model has outstanding performance in heterogeneous network representation visualization. Compared with the baseline model, HRHGNN can better show the clusters formed by nodes of different categories in heterogeneous networks, and the boundaries between categories are obvious.

[0133] The present invention evaluates the effectiveness of HRHGNN on a heterogeneous network benchmark dataset, verifies the role of each core component of the HRHGNN model through node classification experiments and ablation experiments, and demonstrates the superiority of this method; through parameter sensitivity analysis and visualization analysis of HRHGNN's representation ability at different model depths and different propagation orders, it intuitively demonstrates HRHGNN's strong representation ability and discrimination. The experimental results show that the performance of HRHGNN is better than the representative baseline method, successfully verifying the effectiveness of HRHGNN in this application.

[0134] Compared with homogeneous networks, heterogeneous networks have more complex heterogeneity. Therefore, directly extending the homogeneous attribute network representation learning method to heterogeneous graphs cannot obtain good results. At the same time, in view of the problem that the existing heterogeneous attribute network representation learning method cannot effectively obtain remote messages, the heterogeneous attribute network representation learning method proposed in the present invention fully considers the diversity of heterogeneous graphs and can also adaptively obtain messages from distant neighbors. First, heterogeneous feature transformation is performed. The features of different types of nodes are projected into a shared feature space, which facilitates the subsequent aggregation of messages in the message passing mechanism. Secondly, the feature representation of the nodes is iteratively aggregated and updated through a two-level aggregation method based on the network model. The heterogeneous message passing mechanism based on the network model can pass messages according to the type of nodes and the type of relationships, thereby retaining the heterogeneous characteristics of the heterogeneous network. Finally, an adaptive high-order message attention mechanism is proposed, which can perform feature fusion on neighborhood messages from different orders. Neighbors of different orders may have different effects on the central node, and the high-order attention mechanism assigns different weights to messages of different orders. This high-order message passing can adaptively obtain messages from distant neighbors while distinguishing the importance of these messages, thereby improving the model's representation learning ability and improving the discriminative power of node classification.

Claims

1. A heterogeneous attribute network representation learning method based on high-order message passing, characterized in that: The following steps are involved: Step 1: Build a heterogeneous graph neural network model based on high-order message passing; S11: Build a heterogeneous graph neural network model, and make the heterogeneous graph neural network model include a heterogeneous linear transformation module, a heterogeneous message passing module, a high-order message passing module and a classifier module; wherein the heterogeneous linear transformation module is used to map node features of different dimensions into a shared feature space of the same dimension; the heterogeneous message passing module adopts a message passing mechanism based on a network model, and introduces an attention mechanism to adaptively learn the weights of messages from different types of neighbor nodes, and then combines the feature representations from the previous layer to update the feature representations of nodes in the network; the high-order message passing module introduces an attention mechanism based on high-order messages, which is used to adaptively assign weights of messages of different orders according to the different effects of remote neighbors of different orders on the target node, and then weightedly fuse messages of different orders to generate the final representation of the node; the classifier module is used to adopt a single-layer linear transformation method to assign the target node to a specific class or category according to the learned node representation; S12: Divide the selected heterogeneous attribute network benchmark dataset into a training set, a validation set, and a test set according to a set ratio; S13: Use the training set to train the heterogeneous graph neural network model to obtain a trained heterogeneous graph neural network model; during the training process, the model parameters are continuously updated through the back propagation algorithm to reduce the value of the loss function, and the optimal parameters are selected through the verification set results; S14: Use the test set to test the trained heterogeneous graph neural network model to obtain a heterogeneous graph neural network model based on high-order message passing; Step 2: Input a given heterogeneous attribute network and distinguish the nodes in the network; S21: perform heterogeneous linear transformation; For a given heterogeneous attribute network, the heterogeneous linear transformation module maps the original input features of different types of nodes to a shared feature space through a specific type of linear transformation; S22: Perform heterogeneous message transmission based on network mode; For different node and edge types in the direct neighborhood of the target type node, the heterogeneous message passing module adopts a network-based heterogeneous message passing mechanism with a double-layer aggregation function to aggregate messages according to different node types and edge types. For messages at the node type level, the first layer generates features of specific node types by mean aggregation. For messages at the relationship type level, the second layer adaptively assigns different weights to messages from different types of nodes based on the attention mechanism, and then combines the feature representations from the previous layer to update the node feature representations in the network. S23: Perform high-order message passing based on high-order message attention mechanism; For the remote neighborhood messages of different orders of the target type node, the high-order message passing module adaptively aggregates the messages from the remote neighborhood according to the characteristics of all neighboring nodes of each node through the high-order message attention mechanism, and assigns weights to the messages of different orders, and then performs weighted fusion on the messages of different orders to generate the final representation of the node; S24: Use the classifier to classify nodes; The classifier module uses a single-layer linear transformation as a classifier to assign target nodes to specific classes or categories based on the learned node representations.

2. According to claim 1, a heterogeneous attribute network representation learning method based on high-order message passing is characterized in that: In the heterogeneous linear transformation process in step 2 S21, the output feature H after linear transformation is obtained according to formula (1): o ; H o =X o I o (1); Where o∈O is the node type in the heterogeneous attribute network, Xo represents the original input feature of the node type o, and Θo is the learnable weight parameter of the node type o.

3. According to claim 1, a heterogeneous attribute network representation learning method based on high-order message passing is characterized in that: The process of updating the feature representation of nodes in the network in step 2 S22 is as follows: S221: Obtain the representation of the neighbor node r∈R∪{tt} of the target node type t according to formula (2); In the formula, Represents the adjacency matrix corresponding to the current relationship subgraph; s∈O is a node type instance, H s Represents the output features of node s after linear transformation; Θr is the learnable parameter of sub-relation type r; S222: Obtain the hidden layer representation of the node whose target node type is t∈O according to formula (3); In the formula, r1, r2, ..., r n ∈R is a relation type instance.

4. According to claim 1, a heterogeneous attribute network representation learning method based on high-order message passing is characterized in that: In step 2 S23, the final representation of the target node is obtained according to formula (4); In the formula, is the output feature of the target node itself; They are the feature representations of the different order neighborhoods of the target node.

5. According to claim 1, a heterogeneous attribute network representation learning method based on high-order message passing is characterized in that: In step 1 S13, the loss function The acquisition process is as follows: S131: Obtain the predicted label of the target node according to formula (5) In the formula, Z t is the output feature of the target type node learned by the last layer of the graph neural network; Θ c is a learnable parameter in the classifier; S132: Obtain the target node label probability normalized by softmax according to formula (6) In the formula, S13: Obtain the loss function value of the model according to formula (7) And optimize the training of the entire model by minimizing the loss function; In the formula, Represents some nodes with labels; y ic ∈Y is the target node v i The true label of is the target node v i The predicted label of .

Citation Information

Cited By

  • Spectrogram convolutional network heterogeneous graph representation learning method based on path collaborative graph enhancement

    CN121787494A

  • Spectral graph convolution network heterogeneous graph representation learning method based on path collaborative graph enhancement

    CN121787494B