Heterogeneous graph neural network representation method based on meta-path migration and multi-level attention fusion

Abstract

The invention discloses a heterogeneous graph neural network representation method based on meta-path walk and multistage attention fusion, which comprises the following steps: acquiring original node features and relation type information in a heterogeneous graph, and performing feature mapping on the original node features to obtain an initial node representation vector of a target node; executing random walk constrained by a preset meta-path on a target node along the preset meta-path to obtain a meta-path instance sequence, and then performing sequence coding modeling to generate a corresponding instance representation vector; and executing instance-level attention weighted fusion to obtain a meta-path semantic representation, executing meta-path-level attention fusion to generate a final node representation vector, inputting a downstream task model to update model parameters of the final node representation vector, and completing representation learning of a target node in the heterogeneous graph. The method has the beneficial effects that an end-to-end heterogeneous graph node representation learning process is formed, and the structure robustness and the adaptation capability can be improved while the representation precision is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of heterogeneous graph representation learning, and particularly relates to a heterogeneous graph neural network representation method based on meta-path walking and multi-level attention fusion. BACKGROUND

[0002] With the increasing application of heterogeneous information networks (HIN) in recommendation systems, knowledge graphs, academic networks, financial risk control and other scenarios, how to learn high-quality node representations from heterogeneous graphs has become a core problem in the field of graph representation learning. A heterogeneous graph is a graph structure that contains multiple types of nodes and multiple types of relationships. For example, in the IMDB movie dataset, common node types include movies, actors and directors, and edge types include different semantic relationships such as "act" and "direct". In the DBLP academic dataset, typical node types include authors, papers and conferences / journals, and edge types include "write", "publish" and "cite". Such graph structures carry rich high-order semantics, but also pose challenges for representation learning.

[0003] In the prior art, a common approach is to "homogenize" the heterogeneous graph, that is, to treat different types of nodes and edges as the same class, and then use embedding algorithms or graph neural networks for homogeneous graphs for representation learning. This type of method is simple to implement, but it loses the semantic information carried by node types and relationship types, resulting in node vectors that cannot fully distinguish neighbors under different semantic relationships. For tasks that have strong semantic dependencies between multiple types of entities, such as determining movie similarity based on common actors or determining author research direction based on co-authored papers, this loss of information can lead to a decline in the effectiveness of downstream tasks such as classification and clustering.

[0004] To preserve the semantic relationships in a heterogeneous graph, existing research has proposed the concept of "meta-path". A meta-path can be understood as a relationship pattern that spans multiple node types and edge types, such as "movie-actor-movie", "author-paper-author", "author-paper-conference-paper-author", etc. Meta-path-based representation learning methods typically sample or perform random walks along pre-defined meta-paths to obtain node sequences (also referred to as meta-path instances) that conform to the semantic pattern, and then learn node representations based on these sequences. Compared to homogenization, this type of method can preserve information about "which semantic relationship is associated with" to some extent. However, this type of method often only models these sequences as context windows, lacking differentiation of the importance of different meta-path instances; meanwhile, the semantics between different meta-paths are often handled separately, then simply concatenated or weighted, lacking a unified fusion mechanism.

[0005] To further improve the representation quality of heterogeneous graphs, Graph Neural Networks (GNNs) have been introduced into heterogeneous graph scenarios, with typical examples including Heterogeneous Graph Attention Networks (HANs). These methods typically construct a corresponding neighborhood or subgraph for each meta-path, aggregate features within that neighborhood, and then distribute weights across different meta-paths (i.e., different semantic channels) using an attention mechanism to achieve semantic fusion across multiple meta-paths. Compared to simple concatenation or averaging, these methods can adaptively emphasize more useful meta-path semantics. However, existing approaches still have the following shortcomings: 1. Existing methods often treat all neighborhoods or instances corresponding to a meta-path as a whole, without fine-grained differentiation of different instances under the same meta-path (e.g., different "movie-actor-movie" links), failing to highlight the more representative semantics of the target node; 2. In many methods, the neighborhood corresponding to a meta-path is aggregated as an "unordered set," and the temporal information such as the order of occurrence of nodes and relationships in the path instances and contextual dependencies is not fully utilized, which limits the model's ability to express higher-order semantic relationships; 3. The strategies for obtaining meta-path instances, the weight allocation strategies for instances, and the fusion strategies between different meta-paths are often designed independently, lacking joint modeling or collaborative optimization mechanisms, resulting in low information utilization, which in turn affects the accuracy and robustness of tasks such as node classification and node clustering, especially when the annotation ratio is low or noisy edges exist.

[0006] In summary, current heterogeneous graph representation learning methods still have room for improvement in terms of preserving multi-type semantics, utilizing path order information, distinguishing the importance of different meta-path instances, and effectively integrating multiple meta-paths. Summary of the Invention

[0007] To fully utilize the structural features and contextual semantics inherent in multi-type nodes and multi-semantic relationships in heterogeneous information networks, and to overcome the problems in existing technologies such as failure to distinguish the importance of different instances under the same meta-path, inability to effectively model path order information, and the disconnect between meta-path instance evaluation and multi-path semantic fusion, this invention provides a heterogeneous graph neural network representation method based on meta-path walking and multi-level attention fusion. This method generates semantically related path instances through random walks based on meta-paths, performs temporal modeling on the instance sequence and learns instance-level attention weights. Furthermore, it utilizes a meta-path-level attention mechanism to achieve adaptive fusion of multi-semantic information, thereby obtaining node representations that take into account both local semantics and global structure. This effectively improves the representation accuracy and robustness of heterogeneous graphs in tasks such as node classification and clustering.

[0008] To achieve the aforementioned objectives, this invention provides a heterogeneous graph neural network representation method based on meta-path walking and multi-level attention fusion. This method includes sequentially performing the following steps with each node in the heterogeneous graph as the target node: Step S1: Obtain the original node features and relationship type information in the heterogeneous graph, perform feature mapping on the original node features of different types in the heterogeneous graph, map different types of nodes to a unified representation space, and obtain the initial node representation vector of the target node; wherein, the different types of the nodes are distinguished by the pattern structure of the heterogeneous graph.

[0009] Step S2: Based on the pattern structure of the heterogeneous graph, at least one preset meta-path is predetermined. A random walk constrained by the preset meta-path is performed on the target node along each preset meta-path to obtain multiple meta-path instance sequences associated with the target node. Each meta-path instance sequence is an ordered sequence of nodes and relationships connected sequentially according to the corresponding preset meta-path. Step S3: Perform sequence encoding modeling on the multiple meta-path instance sequences respectively, generate corresponding instance representation vectors based on the order of nodes and relations in each meta-path instance sequence, and associate the obtained multiple instance representation vectors with the initial node representation vector corresponding to the target node; Step S4: Perform instance-level attention weighted fusion on multiple instance representation vectors under the same preset meta-path to obtain the meta-path semantic representation of the preset meta-path for the target node; Step S5: Perform meta-path-level attention fusion on the meta-path semantic representations obtained for each preset meta-path to generate the final node representation vector of the target node; Step S6: Input the final node representation vector into the downstream task model to perform node classification and / or clustering tasks, and update the model parameters used to generate the final node representation vector in steps S1 to S5 based on the training results of the downstream task model, thereby completing the representation learning of the target node in the heterogeneous graph; the model parameters include learnable parameters in the process of feature mapping, sequence encoding, instance-level attention, and meta-path-level attention.

[0010] Preferably, the feature mapping in step S1 specifically involves: setting corresponding feature mapping parameters for each node type in the heterogeneous graph, and mapping the original node features of each node type to a node representation vector with a unified vector dimension based on the feature mapping parameters corresponding to each node type.

[0011] Preferably, the preset metapath in step S2 is a sequence of node types and a sequence of relation types determined according to the pattern structure of the heterogeneous graph. The random walk constrained by the preset metapath specifically involves: when performing a random walk on the target node, selecting the next hop node in sequence according to the predefined node type order in the preset metapath to generate the metapath instance sequence that conforms to the preset metapath.

[0012] Preferably, the sequence encoding modeling in step S3 includes: inputting each of the meta-path instance sequences as an ordered sequence of nodes and relations into a recurrent neural network model; modeling the temporal dependencies of the ordered sequence using the recurrent neural network model; and obtaining the corresponding instance representation vector based on the output of the recurrent neural network model. The recurrent neural network model is a gated recurrent unit network or a long short-term memory network. The recurrent neural network model sequentially receives the representations of each node and its relations in the meta-path instance sequence and updates the hidden state, using the hidden state at the final time step as the instance representation vector.

[0013] Preferably, the instance-level attention-weighted fusion in step S4 is performed by a path instance fusion mechanism. The path instance fusion mechanism is implemented. With the target node i in the same preset metapath The instance representation vector of the corresponding multiple meta-path instance sequences (in () as input, to output the preset metapath Meta-path semantic representation for the target node i and satisfy the following form: .

[0014] The specific process for this step is as follows: For the same preset metapath Below and the target node i For each of the associated metapath instance sequences, an instance representation vector corresponding to that metapath instance sequence is obtained. and based on the target node i The representation vector Calculate the unnormalized importance coefficient for each instance. The unnormalized importance coefficients satisfy the following formula:

[0015] in, This represents a vector concatenation operation. To be consistent with the preset metapath The associated trainable parameter vector, It is a linear rectified activation function with a negative slope term; j Represents the target node i In the preset metapath The neighboring nodes under the target node i In metapath The next j One neighboring node; For the unnormalized importance coefficient Normalization is performed to obtain attention weights The attention weights satisfy the following formula:

[0016] in, Indicates targeting the target node i According to the preset metapath A set of indices for the generated sequence of multiple metapath instances; This is the index variable used in the softmax normalized summation for iteration. All neighboring nodes; And based on the attention weights The instance representation vector Perform a weighted summation to obtain the preset meta-path. For the target node i Meta-path semantic representation The weighted sum satisfies the following formula:

[0017] Preferably, the meta-path-level attention fusion in step S5 includes: multiple preset meta-paths for the target node 𝑖. Obtain each of the preset metapaths For the target node i Meta-path semantic representation Based on the meta-path semantic representation Calculate the preset metapath Attention weight coefficient and the attention weight coefficients Used for semantic representation of each of the metapaths Perform weighted fusion to generate the target node. i The final node represents the vector. The weighted fusion satisfies the following formula:

[0018] Meta-path-level attention fusion further details the following computational process: For the target node i Multiple preset metapaths ( This represents the k-th preset meta-path, used to define specific semantic connection patterns between nodes in a heterogeneous graph. Each preset meta-path... All are predetermined in step S2, wherein k Indicates the first k The system includes a total of 10 preset metapath indexes. K Preset metapath, therefore k ∈{1,…, K}), and each of the preset metapaths For the target node i Meta-path semantic representation As input, utilize the meta-path-level fusion mechanism Generate the target node i The final node represents the vector. The meta-path-level fusion mechanism satisfies the following form:

[0019] By providing each of the preset metapaths Calculate the corresponding correlation coefficient The multiple meta-path semantic representations are weighted and fused, and the relevance coefficient is... Satisfy the following formula:

[0020] in, Indicates transpose; This represents the index variable used in the softmax normalization summation process, which is used to traverse all preset metapaths when calculating the metapath-level relevance coefficient. to This is only used for index summation and does not specifically refer to a particular metapath.

[0021] and the correlation coefficient As attention weights, they are used for weighted fusion of the semantic representations of each metapath. To obtain the final node representation vector The weighted fusion satisfies the following formula:

[0022] Preferably, training the downstream task model in step S6 includes: for a semi-supervised node classification task, minimizing the cross-entropy loss function based on the final node representation vector of the target node to update the model parameters used in steps S1 to S5, wherein the cross-entropy loss function 𝐿 satisfies the following formula:

[0023] in, This represents a set of nodes with known labels. Represents a node l The actual labels or label vectors, Represents a node l The final node represents a vector. C This represents the classifier mapping that maps the final node representation vector to the classification result; Based on the minimization result of the cross-entropy loss function 𝐿, the model parameters used to generate the final node representation vector are updated.

[0024] The beneficial effects of this invention are: it establishes an end-to-end heterogeneous graph node representation learning process, which can improve structural robustness and adaptability while ensuring representation accuracy. This provides an efficient and scalable technical solution to address the problems of existing heterogeneous graph modeling methods relying on manual paths and struggling to integrate multiple semantic information. Specifically: (1) The present invention introduces a two-layer attention fusion structure of "instance level - meta-path level" in heterogeneous graphs: First, under the same preset meta-path, instance-level attention weighting is performed on multiple meta-path instance sequences generated based on random walk, making full use of the fine-grained semantic structure information carried by different instances; then, meta-path level self-attention fusion is performed between multiple preset meta-paths to automatically discover the correlation and difference between the semantics of different meta-paths, thereby realizing the joint modeling of multi-level semantic information. Different preset meta-path sets can be selected according to the pattern structure of different heterogeneous graphs, adapting to various application scenarios such as e-commerce recommendation, financial risk control, and academic networks.

[0025] (2) Based on an adaptive attention weighting mechanism, this invention automatically learns the contribution of each meta-path to the representation of the target node within a pre-determined set of preset meta-paths. It does not require manual setting of fixed weights for each meta-path, nor is it limited to the manual design of a single path. Compared with heterogeneous graph representation learning methods that rely on manual design of meta-paths or fixed weights, it has better flexibility and generalization ability and can more fully explore the potential structural patterns in the graph.

[0026] (3) This invention generates and models multiple random walk instances under the same meta-path. By using instance-level attention to weight and fuse the importance of different instances, it can effectively reduce the sampling bias caused by a single random walk path, making the obtained node representation more stable and robust. It is not sensitive to the influence of noise samples and some missing structures, and is suitable for deployment on complex heterogeneous graph data in real-world scenarios. Attached Figure Description

[0027] Figure 1 This is a flowchart of Embodiment 1 of the present invention. Detailed Implementation

[0028] To clearly illustrate the technical features of this solution, the following detailed implementation method will be used to explain the solution.

[0029] Example 1 See Figure 1 This invention provides a method for representing heterogeneous graphs using a neural network based on meta-path walking and multi-level attention fusion. The method includes sequentially performing the following steps with each node in the heterogeneous graph as the target node: Step S1: Obtain the original node features and relationship type information in the heterogeneous graph. Perform feature mapping on the original node features of different types in the heterogeneous graph, mapping different types of nodes to a unified representation space to obtain the initial node representation vector of the target node. The different types of nodes are distinguished by the pattern structure of the heterogeneous graph. Specifically, set corresponding feature mapping parameters for each node type in the heterogeneous graph, and based on the feature mapping parameters corresponding to each node type, map the original node features of that node type into a node representation vector with a unified vector dimension.

[0030] Step S2: Based on the schema structure of the heterogeneous graph, at least one preset meta-path is pre-determined. A random walk constrained by the preset meta-path is performed on the target node along each preset meta-path, resulting in multiple meta-path instance sequences associated with the target node. Each meta-path instance sequence is an ordered sequence of nodes and relations connected sequentially according to the corresponding preset meta-path. The preset meta-path is a sequence of node types and relation types determined according to the schema structure of the heterogeneous graph. Specifically, the random walk constrained by the preset meta-path involves selecting the next-hop node sequentially according to the predefined node type order in the preset meta-path during the random walk, thereby generating a meta-path instance sequence that conforms to the preset meta-path.

[0031] Step S3: Perform sequence encoding modeling on multiple meta-path instance sequences. Generate corresponding instance representation vectors based on the order of nodes and relations in each meta-path instance sequence, and associate the obtained instance representation vectors with the initial node representation vector corresponding to the target node. This sequence encoding modeling includes: inputting each meta-path instance sequence as an ordered sequence of nodes and relations into a recurrent neural network model; the recurrent neural network model models the temporal dependencies of this ordered sequence; and obtaining the corresponding instance representation vector based on the output of the recurrent neural network model. The recurrent neural network model is a gated recurrent unit network or a long short-term memory network. The recurrent neural network model sequentially receives the representations of each node and its relations in the meta-path instance sequence and updates the hidden state, using the hidden state at the final time step as the instance representation vector.

[0032] Step S4: Perform instance-level attention weighted fusion on multiple instance representation vectors under the same preset meta-path to obtain the meta-path semantic representation of the preset meta-path for the target node; In this step, instance-level attention-weighted fusion is achieved through a path-instance fusion mechanism. Implementation, Path Instance Fusion Mechanism With target node i in the same preset metapath The instance representation vector of the corresponding multiple meta-path instance sequences (in () as input, to output a preset metapath Meta-path semantic representation for target node i and satisfy the following form: .

[0033] The specific process for this step is as follows: For the same preset metapath Down to target node i For each of the associated metapath instance sequences, obtain the instance representation vector corresponding to that metapath instance sequence. And based on the target node i The representation vector Calculate the unnormalized importance coefficient for each instance. The unnormalized importance coefficients satisfy the following formula:

[0034] in, This represents a vector concatenation operation. To match the preset metapath The associated trainable parameter vector, It is a linear rectified activation function with a negative slope term; j Represents the target node i In the preset metapath The neighboring nodes under the target node i In metapath The next j One neighboring node; For unnormalized importance coefficients Normalization is performed to obtain attention weights The attention weights satisfy the following formula:

[0035] in, Indicates targeting the target node i Based on the preset meta path A set of indices for the generated sequence of multiple metapath instances; This is the index variable used in the softmax normalized summation for iteration. All neighboring nodes; And based on attention weights Instance representation vector Perform a weighted summation to obtain the preset meta-path. For the target node i Meta-path semantic representation The weighted sum satisfies the following formula:

[0036] Step S5: Perform meta-path-level attention fusion on the meta-path semantic representations obtained from each preset meta-path to generate the final node representation vector of the target node; this meta-path-level attention fusion includes: multiple preset meta-paths for the target node 𝑖 Obtain each preset metapath For the target node i Meta-path semantic representation Based on meta-path semantic representation Calculate the preset metapath Attention weight coefficient And the attention weight coefficient Used for semantic representation of each metapath Perform weighted fusion to generate the target node. i The final node represents the vector. The weighted fusion satisfies the following formula:

[0037] Meta-path-level attention fusion further details the following computational process: For the target node i Multiple preset metapaths ( This represents the k-th preset meta-path, used to define specific semantic connection patterns between nodes in a heterogeneous graph. Each preset meta-path... All are predetermined in step S2, wherein k Indicates the first k The system includes a total of 10 preset metapath indexes. K Preset metapath, therefore k ∈{1,…, K}), and each preset metapath For the target node i Meta-path semantic representation As input, utilize the meta-path-level fusion mechanism Generate target nodei The final node represents the vector. The meta-path-level fusion mechanism satisfies the following form:

[0038] And by providing each preset metapath Calculate the corresponding correlation coefficient Weighted fusion of multiple meta-path semantic representations, relevance coefficient Satisfy the following formula:

[0039] in, Indicates transpose; This represents the index variable used in the softmax normalization summation process, which is used to traverse all preset metapaths when calculating the metapath-level relevance coefficient. to This is only used for index summation and does not specifically refer to a particular metapath.

[0040] and correlation coefficient As attention weights, they are used to weight and fuse the semantic representations of each metapath. To obtain the final node representation vector The weighted fusion satisfies the following formula:

[0041] Step S6: Input the final node representation vector into the downstream task model to perform node classification and / or clustering tasks, and update the model parameters used in steps S1 to S5 to generate the final node representation vector based on the training results of the downstream task model, completing the representation learning of the target node in the heterogeneous graph; the model parameters include learnable parameters in the process of feature mapping, sequence encoding, instance-level attention, and meta-path-level attention. The training of the downstream task model includes: for the semi-supervised node classification task, minimizing the cross-entropy loss function based on the final node representation vector of the target node to update the model parameters used in steps S1 to S5, where the cross-entropy loss function satisfies the following formula:

[0042] in, This represents a set of nodes with known labels. Represents a node l The actual labels or label vectors, Represents a node l The final node represents a vector. C This represents the classifier mapping that maps the final node representation vector to the classification result; Based on the minimization result of the cross-entropy loss function 𝐿, the model parameters used to generate the final node representation vector are updated.

[0043] Example 2 This invention provides a heterogeneous graph neural network algorithm based on meta-path walking, comprising the following steps: I. Meta-path instances and their semantic representations are generated using a random walk algorithm, and the structural information of the meta-paths is used to describe the fine-grained semantic structure around the nodes. After the meta-paths are generated, a soft attention mechanism is used to fuse the structural information of multiple meta-paths to enhance the representation ability of the semantic structure around the nodes. At the same time, a self-attention mechanism is introduced to explore the semantic correlation and difference information between multiple paths. The representation of heterogeneous graph nodes is learned through adaptive weighted fusion to improve the algorithm performance. The meta-path definition and model hyperparameters can be flexibly adjusted for different application scenarios. For example, in the e-commerce recommendation scenario, the focus is on meta-paths such as "user-product-review" and "user-favorite-product". In the financial risk identification scenario, meta-paths such as "enterprise-guarantee-enterprise" and "enterprise-related enterprise-bank" are modeled first.

[0044] II. By designing a node feature transformation module, the original features of different types of nodes are mapped to a unified low-dimensional feature space, and the node features are mapped through linear transformation:

[0045] in, , Representing nodes respectively i Feature vectors before and after projection transformation , Representing nodes respectively i The type and its corresponding feature transformation matrix.

[0046] This module can adapt to different node feature types in different application scenarios. For example, in social network scenarios, it can uniformly map user behavior sequence features and product attribute features (price, category); in bioinformatics scenarios, it can convert gene sequence features and protein structure features into the same space, solving the problem of inconsistent feature spaces of different types of nodes in heterogeneous graphs and the inability to directly propagate and aggregate information.

[0047] Third, by constructing a meta-path instance-level modeling module, the classic random walk algorithm is used to generate meta-path instances and their semantic representations. A soft attention mechanism is then used to fuse structural information from multiple meta-paths to meticulously describe the fine-grained semantic structure surrounding nodes. In practical applications, this module can specifically capture key semantics of a scenario. For example, in intelligent transportation scenarios, random walks generate meta-path instances of "vehicle-road segment-traffic light-road segment-vehicle" to capture the semantic associations between road segments; in academic literature classification scenarios, meta-path instances of "author-paper-keyword-paper-author" are generated to mine the semantic associations between authors' research fields. This allows for better understanding and analysis of the complex semantic structures in heterogeneous graphs, thereby improving the model's performance and interpretability.

[0048] Fourth, by constructing meta-path instances and their representation modules, in order to better capture the order of sampling neighbors in a random walk sequence based on meta-paths, meta-path instances are generated through random walks. Encoding examples using recurrent neural networks (RNNs):

[0049] In e-commerce recommendation scenarios, this module can capture user preferences for different products by encoding meta-path instances of "user-product-review-product". In financial risk identification scenarios, encoding meta-path instances of "enterprise-guarantee-enterprise-loan" can uncover risk transmission relationships between enterprises and achieve coverage of all structures around the node, thereby improving the accuracy of node representation. By providing different neighbor structure information through multiple random walks, the content of node representation is further enriched, while reducing the impact of random walk paths, making node representation more robust and stable, and adapting to dynamic changes in data in the scenario.

[0050] V. A path instance fusion mechanism was designed by constructing a meta-path instance representation fusion module. By using a soft attention mechanism to fully consider the walking sequence The differences are calculated and the resulting metapath walk instances are evaluated for nodes. i The importance of feature representation is shown in the following formula: .

[0051] A more comprehensive capture of all structures surrounding the node and achievement of differential fusion, calculating i and instance Similarity between them:

[0052] in, This represents a vector concatenation operation. This represents the transpose operation. It is a non-linear activation function used to learn the weights of metapath instances.

[0053] The importance coefficients are then normalized using softmax.

[0054] The path to the element is obtained through weighted aggregation. The next node represents:

[0055] In knowledge graph construction scenarios, this mechanism can assign different weights to "entity-relationship-entity" meta-path instances and prioritize the fusion of instance information corresponding to key relationships (such as "disease-cause-gene"). In intelligent transportation scenarios, "vehicle-congested road segment-vehicle" meta-path instances are given higher weights to improve the accuracy of congestion association analysis.

[0056] VI. Fusion is performed using a self-attention mechanism through the meta-path-level modeling module. k Individual path representation :

[0057] Calculate the correlation coefficient between metapaths :

[0058] The final node representation obtained by weighted fusion is:

[0059] In e-commerce recommendation scenarios, this module can automatically increase the weight of meta-paths such as "user-product-purchase" and "user-review-product" while weakening the impact of weakly related meta-paths such as "user-browse-product". In financial risk identification scenarios, it prioritizes the integration of semantic information from meta-paths such as "enterprise-guarantee-enterprise" and "enterprise-non-performing loan-bank", enabling meta-path instance traversal to capture the structural information contained in the meta-paths and improve the decision-making accuracy of the scenario task.

[0060] 7. By optimizing the loss function module, minimize the cross-entropy loss for semi-supervised node classification tasks:

[0061] in, The node represents the mapping to the classifier. Represents the actual label of the node. and These represent the label and vector representation of the heterogeneous node to be classified, respectively. In the academic literature author domain classification scenario, by optimizing the loss function, the classification characteristics of authors in fields such as "database" and "artificial intelligence" can be accurately learned. In the bioinformatics scenario, the gene function classification model can be optimized to improve the classification accuracy of categories such as "oncogene" and "tumor suppressor gene".

[0062] Example 3 This invention provides a heterogeneous graph neural network algorithm based on meta-path walking, specifically a graph neural network method applied to node representation learning in heterogeneous graphs. It includes steps such as node feature transformation, meta-path instance generation, meta-path instance encoding, meta-path instance fusion, meta-path level fusion, and loss function optimization.

[0063] This embodiment illustrates the application of this method in a terminal device. It is understood that this method is also applicable to deployment on a server, and can be used to build a heterogeneous graph analysis system that includes collaborative work between the terminal and the server, achieved through interaction between the terminal and the server. The method described in this embodiment includes the following steps: 1. Data Collection and Preprocessing Download and prepare heterogeneous graph datasets, including the IMDB Movies dataset and the DBLP Academic dataset. The IMDB dataset contains three types of nodes: Movies (M), Actors (A), and Directors (D), and two types of edges: Movie-Director and Movie-Actor. The DBLP dataset contains four types of nodes: Authors (A), Papers (P), Keywords (T), and Conferences (V), and three types of edges: Author-Paper, Paper-Keyword, and Paper-Conference. Standardize the original features of the nodes, such as converting the bag-of-words features of the IMDB Movies plot and the bag-of-words features of the DBLP Authors, Papers, and Keywords into vectors with uniform dimensions to ensure the consistency of the input data.

[0064] 2. Metapath instance generation To address the issue of heterogeneous feature spaces for different types of nodes in heterogeneous graphs, a dedicated feature mapping matrix is ​​designed for each node type. Taking the IMDB dataset as an example, movie node features are mapped using matrix MM, actor node features using matrix MA, and director node features using matrix MD, transforming all node features into a unified low-dimensional space.

[0065] 3. Metapath instances and their representations Based on preset meta-paths (the IMDB dataset selects meta-paths such as MDM and MAM, and the DBLP dataset selects meta-paths such as APA and APVPA), a random walk algorithm is used to generate meta-path instances. Taking the IMDB dataset meta-path MAM (movie-actor-movie) as an example, starting from the movie node, a random walk is performed to the actor node, and then back to the movie node to obtain meta-path instances. A recurrent neural network is used to encode the meta-path instances, converting the node sequence features into semantic representation vectors.

[0066] 4. Meta-path instance representation fusion To address the semantic differences among multiple instances within the same metapath, a soft attention mechanism is employed to calculate the importance weights of each instance to the feature representation of the starting node. The starting node features are concatenated with the instance representations of the metapath, and an importance coefficient is obtained through a nonlinear transformation. This is then normalized using softmax to obtain the weight coefficients, and finally, a weighted sum is used to obtain the feature representation of the node within that metapath.

[0067] 5. Meta-path level modeling To address the semantic complementarity of multi-path systems, a self-attention mechanism is employed to learn the weight coefficients of different meta-paths. The correlation between the feature representations of nodes under different meta-paths is calculated, and the meta-path weights are obtained through softmax normalization. Finally, the node's final feature representation is obtained through weighted fusion.

[0068] 6. Loss Function Optimization A semi-supervised learning framework is adopted, with the cross-entropy loss of labeled nodes as the optimization objective, and model parameters are updated through backpropagation. In the designed MPRW-HGNN, the batch size is 8, the maximum number of training iterations is 100, each node samples a maximum of 100 neighbors, the dropout ratio is 0.5, the learning rate is 0.005, and the L2 regularization is 0.001. For random walk models such as DeepWalk, ESim, metapath2vec, and HERec, the window size is set to 5, the walk length is 100, the number of walks is 40, and the number of negative samples is 5. For GCN and GAT, parameters are tuned using the validation set. For the MPRW-HGNN model, the learning rate is 0.005, the regularization coefficient is 0.001, the semantic level attention vector q dimension is 128, the number of attention heads K is 8, and the random dropout ratio is 0.6. All semi-supervised (heterogeneous) graph neural network models were used with identical datasets, and to ensure fairness, all algorithms learned node representations of 64 dimensions. To prevent overfitting, we used an early stopping strategy, stopping training when the model's loss function on the validation set did not decrease for five consecutive epochs.

[0069] Through the detailed implementation steps described above, this invention significantly improves the learning effect of heterogeneous graph node representation by using meta-path instance modeling and a two-level attention mechanism, and can be widely applied in fields such as social network analysis and knowledge graph reasoning.

[0070] To verify the effectiveness and practicality of the proposed method, examples are given on the IMDB and DBLP datasets. Tables 1 and 2 show the node classification results of the examples on the test set, measured by Macro-F1 and Micro-F1; Tables 3 and 4 show the node clustering results of the examples on the test set, measured by NMI and ARI. Detailed information for each category is as follows: Table 1. Validation results of the example on the IMDB dataset for node classification.

[0071] As shown in Table 1, with a 20% training label ratio in the IMDB dataset, the Macro-F1 score of MPRW-HGNN in this invention is 59.09%, which is 14.93% higher than the homogeneous graph model LINE (44.16%) and 2.78% higher than the heterogeneous graph model HAN (56.31%). With an 80% training label ratio, the Micro-F1 score of MPRW-HGNN reaches 60.55%, continuing to outperform other comparative models and verifying its advantages in node classification tasks.

[0072] Table 2 shows the validation results of the example on the DBLP dataset for node classification.

[0073] As shown in Table 2, with a 20% training label ratio in the DBLP dataset, MPRW-HGNN's Macro-F1 score is 93.09%, which is 4.62% higher than metapath2vec (88.47%) and 2.04% higher than GAT (91.05%). With an 80% training label ratio, its Micro-F1 score reaches 94.44%, further demonstrating the applicability of the model in academic heterogeneous graph scenarios.

[0074] Table 3. Validation results of node clustering on the IMDB dataset.

[0075] As shown in Table 3, in the clustering task of the IMDB dataset, MPRW-HGNN has an NMI of 14.12% and an ARI of 15.26%, which are 8.81% and 9.15% higher than node2vec (NMI 5.31% and ARI 6.11%), respectively, and are significantly better than other unsupervised and semi-supervised comparison models.

[0076] Table 4 shows the validation results of node clustering on the DBLP dataset.

[0077] As shown in Table 4, in the DBLP dataset clustering task, MPRW-HGNN has an NMI of 79.36% and an ARI of 84.14%, which are 1.87% and 1.19% higher than HAN (NMI 77.49% and ARI 82.95%), respectively, demonstrating its stability and superiority in node clustering tasks. Experimental results on the publicly available heterogeneous graph datasets IMDB and DBLP in this embodiment demonstrate that the MPRW-HGNN algorithm proposed in this invention outperforms traditional homogeneous graph models (LINE, node2vec) and mainstream heterogeneous graph models (metapath2vec, HAN) in all evaluation metrics for node classification and clustering tasks. Especially when dealing with heterogeneous graphs containing multiple node types and complex semantic relationships, the algorithm significantly improves the semantic richness and structural robustness of node representations through the fusion of instance-level and semantic-level attention in metapath tasks. It also exhibits good transferability and extensibility, making it suitable for other heterogeneous graph analysis frameworks and graph mining tasks, demonstrating high adaptability and practical value.

[0078] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A heterogeneous graph neural network representation method based on meta-path walking and multi-level attention fusion, characterized in that, The method includes sequentially performing the following steps with each node in the heterogeneous graph as the target node: Step S1: Obtain the original node features and relationship type information in the heterogeneous graph, perform feature mapping on the original node features of different types in the heterogeneous graph, map different types of nodes to a unified representation space, and obtain the initial node representation vector of the target node. Step S2: Based on the pattern structure of the heterogeneous graph, at least one preset meta-path is predetermined, and a random walk constrained by the preset meta-path is performed on the target node along each preset meta-path to obtain a sequence of multiple meta-path instance instances associated with the target node; Step S3: Perform sequence encoding modeling on the multiple meta-path instance sequences respectively, and generate corresponding instance representation vectors based on the order of nodes and relations in each meta-path instance sequence; Step S4: Perform instance-level attention weighted fusion on multiple instance representation vectors under the same preset meta-path to obtain the meta-path semantic representation of the preset meta-path for the target node; Step S5: Perform meta-path-level attention fusion on the meta-path semantic representations obtained for each preset meta-path to generate the final node representation vector of the target node; Step S6: Input the final node representation vector into the downstream task model to perform node classification and / or clustering tasks, and update the model parameters used to generate the final node representation vector in steps S1 to S5 based on the training results of the downstream task model, thereby completing the representation learning of the target node in the heterogeneous graph.

2. The heterogeneous graph neural network representation method according to claim 1, characterized in that, The feature mapping in step S1 specifically involves setting corresponding feature mapping parameters for each node type in the heterogeneous graph, and mapping the original node features of each node type to a node representation vector with a unified vector dimension based on the feature mapping parameters corresponding to each node type.

3. The heterogeneous graph neural network representation method according to claim 1, characterized in that, The preset metapath in step S2 is a sequence of node types and a sequence of relation types determined according to the pattern structure of the heterogeneous graph. The random walk constrained by the preset metapath specifically involves: when performing a random walk on the target node, selecting the next hop node in sequence according to the predefined node type order in the preset metapath to generate the metapath instance sequence that conforms to the preset metapath.

4. The heterogeneous graph neural network representation method according to claim 1, characterized in that, The sequence encoding modeling in step S3 includes: inputting each meta-path instance sequence as an ordered sequence of nodes and relationships into a recurrent neural network model, modeling the temporal dependencies of the ordered sequence by the recurrent neural network model, and obtaining the corresponding instance representation vector based on the output of the recurrent neural network model.

5. The heterogeneous graph neural network representation method according to claim 1, characterized in that, The instance-level attention weighted fusion in step S4 includes: for multiple instance representation vectors under the same preset metapath, calculating the corresponding attention weights based on the initial node representation vector of the target node and each instance representation vector, and performing a weighted summation of the multiple instance representation vectors according to the attention weights to obtain the metapath semantic representation of the preset metapath for the target node.

6. The heterogeneous graph neural network representation method according to claim 5, characterized in that, The instance-level attention-weighted fusion in step S4 includes the following calculation process: For the same preset metapath Below and the target node i For each of the associated metapath instance sequences, an instance representation vector corresponding to that metapath instance sequence is obtained. and based on the target node i Representation vector Calculate the unnormalized importance coefficient for each instance. The unnormalized importance coefficients satisfy the following formula: in, This represents a vector concatenation operation. To be consistent with the preset metapath The associated trainable parameter vector, It is a linear rectified activation function with a negative slope term; j Represents the target node i In the preset metapath The neighboring nodes under the target node i In metapath The next j One neighboring node; For the unnormalized importance coefficient Normalization is performed to obtain attention weights The attention weights satisfy the following formula: in, Indicates targeting the target node i According to the preset metapath A set of indices for the generated sequence of multiple metapath instances; This is the index variable used in the softmax normalized summation for iteration. All neighboring nodes; And based on the attention weights The instance representation vector Perform a weighted summation to obtain the preset meta-path. For the target node i Meta-path semantic representation The weighted sum satisfies the following formula: 。 7. The heterogeneous graph neural network representation method according to claim 1, characterized in that, The instance-level attention-weighted fusion in step S4 is achieved through a path instance fusion mechanism. The path instance fusion mechanism is implemented. With the target node i in the same preset metapath The instance representation vector of the corresponding multiple meta-path instance sequences (in () as input, to output the preset metapath Meta-path semantic representation for the target node i and satisfy the following form: 。 8. The heterogeneous graph neural network representation method according to claim 1, characterized in that, The meta-path-level attention fusion in step S5 includes: multiple preset meta-paths for the target node 𝑖. Obtain each of the preset metapaths For the target node i Meta-path semantic representation Based on the meta-path semantic representation Calculate the preset metapath Attention weight coefficient and the attention weight coefficients Used for semantic representation of each of the metapaths Perform weighted fusion to generate the target node. i The final node represents the vector. The weighted fusion satisfies the following formula: 。 9. The heterogeneous graph neural network representation method according to claim 1, characterized in that, Step S6, training the downstream task model, includes: for a semi-supervised node classification task, minimizing the cross-entropy loss function based on the final node representation vector of the target node to update the model parameters used in steps S1 to S5. The cross-entropy loss function φ satisfies the following formula: in, This represents a set of nodes with known labels. Represents a node l The actual labels or label vectors, Represents a node l The final node represents a vector. C This represents the classifier mapping that maps the final node representation vector to the classification result; Based on the minimization result of the cross-entropy loss function 𝐿, the model parameters used to generate the final node representation vector are updated.

Images