Role-based node attribute multi-relation graph representation learning method

Abstract

The invention discloses a role-based node attribute multi-relation graph representation learning method, which comprises the following steps of: generating node pairs with different types of information to obtain node representations retaining the information; optimization of structural role node pairs is included in a graph representation learning framework, and the model performance is enhanced through additional information; learning the embedding of nodes in each relation graph, and unifying the embedding from different relations to form final node representation; according to the invention, an innovative role-based method is introduced, and node structure information and node attribute characteristics are integrated.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-relational graph representation learning, and particularly to a role-based multi-relational graph representation learning method for node attributes. Background Art

[0002] Multi-relational graph embedding: In recent years, multi-relational graph representation learning methods have received extensive attention. For example, PMNE proposed three learning strategies: network aggregation, result aggregation, and inter-layer collaborative analysis to learn the node embeddings of multi-way networks. OhmNet is an unsupervised hierarchical multi-way network embedding method. MNE and CrossMNA simultaneously learn different-dimensional embeddings for each node and then combine them to obtain the final node representation. MELL learns node embeddings at each layer and combines them with layer embeddings to obtain the final node embeddings. GATNE integrates basic embeddings, edge embeddings, and attribute embeddings to learn node representations, aiming to address the challenges brought by multi-layer heterogeneous networks. HDMI optimizes the joint supervision signal and uses the attention mechanism to integrate node embeddings. HMNE learns node embeddings by considering high-order node dependencies. The multi-view collaborative network embedding method MANE derives node embeddings by considering the low-order collaboration between nodes. MHGCN parameterizes the specific relationship representations of nodes and constructs two complementary coupling frameworks to capture the importance of various context factors of different node and edge types in heterogeneous graphs. Recently, some methods have designed role-based multi-relational graph learning methods. For example, RMNE was the first to explore role-based multi-layer network representation learning.

[0003] Graph neural networks: Graph neural networks (GNNs) use deep neural networks to solve graph representation learning problems and have achieved remarkable success in various graph-related tasks including node classification, link prediction, and community detection. GNNs also show remarkable performance in many traditional scenarios, such as in the fields of recommendation systems, natural language processing, etc. For example, Hu et al. incorporated the attention mechanism based on Transformer into GNNs to process large academic networks. Xu et al. created scene graphs from original input images and used GNNs to obtain structured scene representations. Initially, GNNs mainly dealt with homogeneous graphs, such as the graph convolutional network (GCN), in which the target node uniformly collects information from adjacent nodes. From a technical perspective, the application of the attention mechanism is one of the most common methods in GNNs. The graph attention network (GATs) implements an attention mechanism that enables GNNs to autonomously determine the importance of adjacent nodes.

[0004] Node Role: Node roles divide the nodes in a graph into different sets, and nodes in the same set have high semantic similarity. If two nodes in a graph have the same neighbor connection pattern and similar attribute features, they are considered to have the same node role. In a unified embedding space, the vector representations of nodes within the same role set should be close to each other.

[0005] The formal definition of node role is as follows:

[0006] Node Role: Given a graph G(V, E) and node attribute information x, first extract the structural features F = [f1, f2,..., f m , and then, map each node to a set of node roles through a mapping function Φ. This is achieved by explicitly defining a function that maps vertices to M node role sets W = [ω1, ω2,..., ω M (where M ≤ |V|). Let u and v be arbitrary nodes, and the specific mapping process is as follows:

[0007]

[0008] Multi-relational graph representation learning aims to obtain low-dimensional vector representations of nodes and edges and has made significant progress in recent years. In particular, graph neural networks have been successfully applied to learn node and edge representations for multi-relational graph embedding. For example, MNE, GATNE, and DMGI learn node representations for each specific relation respectively and then aggregate them to obtain the final node representation. DualHGNN uses a spectral hypergraph convolution operator and adopts internal and cross-domain information transfer strategies to promote intra-domain and cross-domain information exchange, thereby learning effective node embeddings. FAME and MHGCN can automatically learn valuable heterogeneous meta-path interactions of different lengths in multi-way heterogeneous networks by leveraging multi-layer convolutional aggregation. Examples are Figure 1 shown, where (a) is an example of a two-layer undirected multi-relational graph with node attributes. (b) is a simple example of node roles in a multi-relational attribute graph. (c) is an example of node role definition, where both the structural features and node attribute features of nodes 5 and 6 in layer 1 are considered when defining node roles. The above models have shown good performance in multi-relational graph representation learning.

[0009] Most existing methods ignore node role information, which can be used to determine the similarity between nodes, often resulting in suboptimal performance. To address this issue, some researchers have begun to design role-based multi-relational graph representation learning methods, aiming to retain the corresponding role knowledge when learning node representations. However, most methods only focus on leveraging structural information to learn role knowledge. In fact, in most multi-relational graphs, nodes usually have concise attributes, and existing methods cannot make good use of these attributes. Summary of the Invention

[0010] To solve the problems existing in the prior art, the object of the present invention is to provide a role-based multi-relational graph representation learning method for node attributes. The present invention introduces an innovative role-based method that integrates node structure information and node attribute features.

[0011] To achieve the above object, the technical solution adopted by the present invention is: a role-based multi-relational graph representation learning method for node attributes, including:

[0012] Generating node pairs with different types of information to obtain node representations that retain this information; learning the embeddings of nodes in each relational graph and unifying the embeddings from different relations to form the final node representation.

[0013] As a further improvement of the present invention, given an L-layer multi-relational graph and the generated node pairs, the corresponding loss of each module is as follows:

[0014] (1) Node pairs within the same relational graph network: In each layer of the network, first use the DeepWalk random walk method to generate a node sequence, and then generate node pairs based on the walk sequence; the central node u and the node v ∈ N e (u) form an intra-layer node pair (u, v), and the objective function corresponding to the node pairs within the same relational network is as follows:

[0015]

[0016] Use the softmax function to calculate logP(v∣u; θ), then:

[0017]

[0018] where, N e (u) is the set of context nodes of node u within a specific window size; f v and f u represent the embedding vectors of nodes v and u in each layer of the network, respectively;

[0019] (2) Node pairs with a one-to-one correspondence: The corresponding objective function is:

[0020]

[0021] Among them, Intra(u) represents the set of nodes corresponding one-to-one to node u; there is:

[0022]

[0023] (3) Cross-layer cross-node pairs: In different network layers l and l', the possibility of second-order cooperation in the real network is measured using the following equation:

[0024]

[0025] Among them, e uv represents the edge connecting nodes u and v;

[0026] The objective function corresponding to the cross-layer cross-node pair is as follows:

[0027]

[0028]

[0029] Among them, cross(u) is used to represent the set of second-order cooperative nodes of the central node u;

[0030] (4) Node role pairs: Role(u) is used to represent the set of nodes with the same role as node u; Role(u) includes nodes with the same role as node u in the same layer and different layers; therefore, the objective function corresponding to the node role pair is:

[0031]

[0032] Then the comprehensive loss function is expressed as follows:

[0033]

[0034] Among them, α, β, and γ (≥0) are adjustable hyperparameters used to control the influence of each module on the overall loss function.

[0035] The present invention aims to solve the problems existing in the existing multi-relational graph representation learning methods, that is, most methods ignore node role information and fail to fully utilize node attribute features, resulting in the inability to achieve optimal performance in tasks such as node classification and link prediction. By proposing a role-based multi-relational graph representation learning method (RMGRL), combining node role information with node structure and attribute features, and unifying different levels of information into an optimization framework, the similarity between nodes can be accurately captured, and the performance in node classification and link prediction tasks can be improved, which is superior to the existing state-of-the-art methods.

[0036] The beneficial effects of the present invention are as follows:

[0037] More comprehensive information capture: Most existing methods only focus on node structure information or ignore node role information. However, the RMGRL method of the present invention can not only capture the structure information of nodes, but also incorporate node attribute information, coupling node role information and various structural features into a unified graph learning framework, more comprehensively reflecting the characteristics of nodes and obtaining a more comprehensive node vector representation.

[0038] Effectively maintain structural similarity: The embedding vectors generated by RMGRL can effectively maintain the structural similarity within and between layers. In the representation learning of multi-relational graphs, it can better retain the structural features of the graph, providing a more accurate basis for subsequent tasks.

[0039] Significant performance improvement: In the experimental evaluation of tasks such as node classification and link prediction, on two real-world multi-relational graphs, the RMGRL method significantly outperforms the existing state-of-the-art methods, such as a series of related methods like MNE and GATNE, demonstrating its effectiveness and superiority in practical applications and improving the performance of multi-relational graph representation learning in related tasks.

[0040] Linear scalability: From the complexity analysis, RMGRL has the characteristic of linear scalability, and its overall complexity is O(|E|·d·H). This means that when dealing with large-scale multi-relational graph data, this method has good adaptability and scalability, and can cope with the growth of data scale in practical applications, having important economic and technical value. Brief Description of the Drawings

[0041] Figure 1 It is an example graph of a traditional multi-relational graph;

[0042] Figure 2 It is a framework graph of role-based multi-relational graph representation learning in an embodiment of the present invention;

[0043] Figure 3 It is a schematic diagram showing the influence of different parameters on the node classification performance of RMGRL on the IMDB dataset in an embodiment of the present invention;

[0044] Figure 4 It is a schematic diagram showing the influence of different parameters on the link prediction performance of RMGRL on the Amazon dataset in an embodiment of the present invention;

[0045] Figure 5 It is a schematic diagram of different parameters of the link prediction performance of RMGRL on the IMDB dataset in an embodiment of the present invention. Detailed Embodiment

[0046] The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0047] Example

[0048] A Role-based Node Attribute Multi-relational Graph Representation Learning method (RMGRL). The proposed RMGRL integrates node attribute information to develop a role-based multi-relational graph representation learning framework. The comprehensive framework of RMGRL is as Figure 1 shown.

[0049] Similar to the Skip-gram model, RMGRL aims to optimize node representations by maximizing the co-occurrence probability of node pairs. Therefore, the RMGRL method generates node pairs with different types of information to obtain node representations that retain this information. Then, it learns the embeddings of nodes in each relational graph and unifies the embeddings from different relations to form the final node representation. Given an L-layer multi-relational graph and the generated node pairs, the corresponding losses for each module in RMGRL are as follows:

[0050] (1) Node pairs within the same relational graph network: In each layer of the network, first use the DeepWalk random walk method to generate a node sequence, and then generate node pairs based on the walk sequence. The central node u and the node v ∈ N e (u) form an intra-layer node pair (u, v), as shown by the node pairs (1, 2) and (9, 12) in Figure 2 . The objective function corresponding to this module is as follows:

[0051]

[0052] where the softmax function is used to calculate logP(v∣u; θ), so the equation becomes:

[0053]

[0054] where N e (u) is the set of context nodes of node u within a specific window size. f v and f u represent the embedding vectors of nodes v and u in each layer of the network, respectively.

[0055] (2) Node pairs with a one-to-one correspondence: This type of node pair involves the case where the same node is represented in different network layers. The purpose of optimizing these node pairs is to ensure that although the same node is in different network layers, its representation is as similar as possible. This method is consistent with the existing research results. The objective function corresponding to this module is:

[0056]

[0057] where Intra(u) represents the set of nodes that correspond one-to-one with node u. For example, asFigure 2 As shown, Intra(2) = {8, 14}. Taking node 2 as the target node, node pairs (2, 8) and (2, 14) can be formed. Similarly, there are:

[0058]

[0059] (3) Cross - layer and cross - node pairs: According to existing research, in different network layers l and l', the likelihood of second - order collaboration in the real network can be measured using the following equation:

[0060]

[0061] where e uv represents the edge connecting nodes u and v. A higher ratio indicates that nodes connected in one network layer are more likely to be connected in another layer. If the value is 1, it means that the formation of a link between two nodes in one layer is independent of their connection in another layer. Therefore, for the two real - world datasets used in this embodiment (as shown in Table 3), the ratios between different network layers are shown in Tables 1 and 2. It can be observed that in both datasets, the corresponding ratios are not equal to 1, indicating the existence of a second - order collaboration effect in the datasets. This means that compared with node pairs that are not connected in layer l, node pairs that are connected in layer l are more likely to form connections in layer l'.

[0062] Table 1. Second - order collaboration between different layers in IMDB

[0063]

[0064]

[0065] Table 2. Second - order collaboration between different layers in Amazon

[0066]

[0067] Table 3. Statistical information of the datasets

[0068]

[0069] The symbol cross(u) is used to represent the set of second - order collaborative nodes of the central node u. As Figure 2 shown, in the second layer of the multi - attribute network, there is an edge connecting nodes 9 and 12. This connection increases the likelihood of forming an edge between them in the third layer (i.e., nodes 15 and 18 are more likely to be connected), thus creating cross - layer and cross - node pairs, such as (3, 18). The second - order collaboration relationship reflects the second - order proximity between nodes to a certain extent. The objective function of this module is as follows:

[0070]

[0071] (4) Node role pairs: The optimization of node role pairs is the main contribution of this embodiment. The core idea is that if two nodes have the same role, their representations in the embedding space should be close, regardless of whether they are in the same network layer or different network layers. Therefore, by considering node attributes and structural information and identifying node role information, node role pairs can be constructed. Different from previous studies, this embodiment not only considers the structural information of nodes but also considers the attribute characteristics of nodes. Use Role(u) to represent the set of nodes that have the same role as node u. Therefore, nodes sharing the same node role form a role pair (for example, Figure 2 the node pair (3, 10) in

[0072]

[0073] Therefore, there is:

[0074]

[0075] Therefore, by integrating the above modules, the comprehensive loss function of multi-attribute network modeling can be expressed as follows:

[0076]

[0077] where α, β, and γ (≥0) are adjustable hyperparameters used to control the impact of each module on the overall loss function.

[0078] To implement RMGRL, node role pairs are generated and obtained from the same role set. This embodiment also uses negative sampling techniques and the Adam optimizer for optimization. Finally, the node representations from different relational networks are concatenated to form the final node representation.

[0079] Complexity analysis: Based on existing work, the time complexity of intra-layer pairs is O(|E|·d / |L|·H), where |E| is the total number of edges in all layers of the multi-relational graph, d / |L| is the embedding dimension of nodes in a specific layer, and H is the number of negative samples. The time complexity of cross-layer intra-node pairs and cross-layer inter-node pairs is O(|E|·d / |L|·H|L|). The time complexity of structural role pairs is O(|E|·d·H). Therefore, the overall complexity is O(|E|·d·H). It can be seen that RMGRL has linear scalability.

[0080] In this embodiment, the performance of RMGRL is evaluated through node classification and link prediction tasks. Specifically, link prediction tasks are carried out on Amazon and IMDB datasets, and node classification tasks are carried out on the IMDB dataset.

[0081] Datasets:

[0082] Two real-world multi-relational attributed graphs, Amazon and IMDB, are used to verify the performance of the proposed method. Table 3 shows the basic statistics of the multi-relational graphs.

[0083] (1) The Amazon dataset includes product reviews and metadata from the Amazon website. Yu et al. constructed the multi-relational graph used in this embodiment using only product metadata, including product attributes and connections between products. The graph consists of two layers, representing co-browsing and co-purchasing relationships between various products respectively. Node attributes include external product features such as product category, sales rank, brand, and price information.

[0084] (2) The IMDB dataset is a heterogeneous graph containing three different types of nodes: movies, directors, and actors. For the node classification task, this embodiment uses movie genres (e.g., romance, action, science fiction) as node labels and represents node attributes through bag-of-words feature vectors.

[0085] Baseline methods:

[0086] To verify the effectiveness of the proposed RMGRL method, it is compared with 14 baseline models. The specific details are as follows:

[0087] (1) Node2vec: It generates node sequences on the graph through random walks and learns graph representations.

[0088] (2) RandNE: It designs an iterative Gaussian random projection method to transform the network into a low-dimensional embedding space.

[0089] (3) FastRP: It explicitly constructs a node similarity matrix to capture the propagation relationships in the graph. In addition, it uses highly sparse random projection for dimensionality reduction, which helps to iteratively calculate node embeddings.

[0090] (4) SGC: It simplifies graph convolution by removing the non-linear projection in the network during the inter-layer information propagation process of the graph.

[0091] (5) R-GCN: It considers various edge types related to nodes and adopts weight sharing and coefficient constraints in heterogeneous networks.

[0092] (6) MAGNN: It contains three main components: node content transformation, internal meta-path aggregation, and cross-meta-path aggregation.

[0093] (7) HPN: It designs a semantic propagation mechanism to alleviate semantic ambiguity and a semantic fusion mechanism to learn the importance of meta-paths.

[0094] (8) PMNE: It contains three different models for integrating multiple networks to generate a comprehensive embedding for each node. The model has three variants: PMNE-n, PMNE-r, and PMNE-c.

[0095] (9) MNE: It derives the final embedding by combining high-dimensional common embeddings with low-dimensional specific layer embeddings.

[0096] (10) GATNE: It proposes using base embeddings as well as edge and attribute representations to generate overall node embeddings. Edge embeddings are generated by self-attention fusion of neighborhood information.

[0097] (11) DMGI: It introduces a consensus regularization framework to address the interrelationships between specific types of node embeddings. Additionally, it employs an attention mechanism to aggregate signals from each relational context.

[0098] (12) FAME: It is a network embedding method for AMHENs that utilizes random projection and spectral graph transformation to capture meta-paths, significantly improving efficiency by applying random projection techniques.

[0099] (13) MHGCN: It parameterizes the specific relation representation of nodes and develops two complementary coupling structures to model the multi-context importance of different types of nodes and edges in heterogeneous graphs.

[0100] (14) RMNE: It generates structural roles based on the structural similarity of nodes and adopts a role-based random walk strategy to learn node embeddings.

[0101] Table 4. Link Prediction

[0102]

[0103] The best results for each dataset are in bold; the sub-optimal scores for each algorithm are underlined. Some of the experimental results in the table are from MHGCN.

[0104] Link prediction task:

[0105] The results of link prediction are summarized in Table 4, indicating that the proposed RMGRL outperforms all baseline models. The analysis of the results in Table 4 shows that the single-layer network embedding method without considering multiple networks performs poorly in link prediction on both datasets. In addition, among the multiple network embedding methods, MHGCN performs relatively poorly, while RMNE performs even worse in terms of AUC and F1 score. The possible reason is that RMNE only considers the structural role based on structural features and does not consider the features of the nodes themselves. This further shows that it is more realistic to consider the structural role of nodes with attribute features and can enhance the model's representation learning ability.

[0106] Node classification task:

[0107] The results of the node classification experiment are shown in Table 5. As shown in Table 5, compared with the baseline models, RMGRL achieves the best results. Specifically, the RMGRL proposed in this embodiment improves by 6.4% on Macro-F1.

[0108] Table 5. Performance evaluation of node classification task

[0109]

[0110] The best results for each dataset are shown in bold. The results of these benchmark models are from MHGCN.

[0111] Compared with the sub-optimal RMNE, Micro-F1 improves by 6.3%. Compared with FAME, both RMNE and RMGRL show significant performance improvements, indicating that considering global role information can enhance the model's ability to distinguish nodes. In addition, compared with RMNE, it further illustrates that considering node attribute information in node roles is beneficial. Defining node roles from both topological structure and node attributes can filter out noise information with the same topological structure but different node attributes, thereby further improving the model's representation learning ability. It should be noted here that this embodiment does not use supervised or semi-supervised methods to compare node classification performance because RMGRL is unsupervised. However, it is believed that applying RMGRL in the semi-supervised or supervised learning paradigm can also further improve the model's performance.

[0112] Parameter sensitivity analysis:

[0113] For the RMGRL method, this embodiment conducts parameter sensitivity analysis on two datasets and two tasks. The RMGRL method contains three hyperparameters. In the experiment, this embodiment systematically changes one hyperparameter at a time while keeping other parameters unchanged to evaluate its impact on the model performance. The results are as Figure 3As shown. Among the hyperparameters, β has the least impact on the F1 score of the model, while α and γ significantly affect the F1 score. This may be because in the IMDB dataset, the information of cross-layer same-node pairs and node role pairs is more important, while the importance of second-order collaborative information is relatively low. For the node classification task, the optimal values are determined as α = 1, β = 1, and γ = 0.5.

[0114] In the Amazon link prediction task, the results of the parameter sensitivity experiment are as Figure 4 shown. According to Figure 4 , the optimal values of parameters α and β are both 0, while the optimal value of γ is 0.5. As α and β increase, the performance of the model gradually decreases. On the contrary, as γ increases, the performance of the model first improves and then decreases. This behavior may be attributed to the fact that in the Amazon dataset, the information of cross-layer same-node pairs and second-order adjacencies contributes relatively little to the model performance. In contrast, the information of node role pairs has a significant impact on the link prediction accuracy of the model.

[0115] In the IMDB link prediction task, Figure 5 shows the sensitivity of the model to different parameters. The results show that the optimal values of α and β are both 0.5, while the optimal value of γ is 1. Specifically, as α and β increase, the performance of the model initially improves but then decreases. On the contrary, as γ increases, the performance of the model continuously enhances. Generally speaking, the performance of the model fluctuates relatively little on the IMDB dataset, while it shows greater variability on the Amazon dataset.

[0116] In the RMGRL method, in this embodiment, the impact of different node role discovery methods on the model performance is further studied through experiments. Table 6 shows the results of the node classification task on the IMDB dataset and the link prediction task on the Amazon dataset. As shown in Table 6, the choice of node role discovery method significantly affects the model performance, especially on the IMDB dataset. On the Amazon dataset, different node role discovery methods all show good performance. Specifically, the degree-based node role method performs the worst, while the motif-based model performs the best, followed by the WL-based method. This is consistent with the expectation because the degree-based node role method is the most basic and rough topological structure representation method, while the motif-based role discovery method can include node degree information. Therefore, this embodiment recommends using the motif-based node role discovery method in the multi-relational attribute graph representation learning process because the model performance will be better.

[0117] Table 6. Node Classification Performance of RMGRL under Different Node Role Discovery Algorithms

[0118]

[0119] The above-described embodiments merely represent specific implementation manners of the present invention. The description thereof is relatively specific and detailed, but it should not be construed as a limitation to the scope of the patent of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present invention, several modifications and improvements can still be made, and these all fall within the protection scope of the present invention.

Claims

1. A role-based multi-relationship graph representation learning method for node attributes, characterized in that Including: Generating node pairs with different types of information to obtain node representations that retain this information; Learning the embeddings of nodes in each relational graph and unifying the embeddings from different relations to form the final node representations.

2. The role-based node attribute multi-relationship graph representation learning method according to claim 1, wherein Given an L-layer multi-relational graph and the generated node pairs, the corresponding losses for each module are as follows: (1) Node pairs within the same relationship graph network: In each layer of the network, first use the DeepWalk random walk method to generate a node sequence, and then generate node pairs based on the walk sequence; the central node u and the node v ∈ N e (u) form an intra-layer node pair (u, v), and the objective function corresponding to the node pairs within the same relationship network is as follows: Using the softmax function to calculate logP(v∣u; θ), then: Among them, N e (u) is the set of context nodes of node u within a specific window size; f v and f u respectively represent the embedding vectors of nodes v and u in each layer of the network; (2) Node pairs with a one-to-one correspondence: The corresponding objective function is: where Intra(u) represents the set of nodes that correspond one-to-one with node u; there is: (3) Cross-layer cross nodes pair: Consistent with existing literature, in different network layers l and l', the probability of second-order collaboration in the real network is measured using the following equation: is measured using the following equation: Among them, e uv represents the edge connecting nodes u and v; The objective function corresponding to the cross-layer cross node pairs is as follows: where cross(u) is used to represent the set of second-order collaborative nodes of the central node u; (4) Node role pairs: Using Role(u) to represent the set of nodes that have the same role as node u; Role(u) includes nodes that have the same role as node u in the same layer and different layers; therefore, the objective function corresponding to the node role pairs is: Then the combined loss function is expressed as follows: where α, β, and γ (≥0) are adjustable hyperparameters used to control the influence of each module on the overall loss function.

Images