Self-supervised heterogeneous graph node classification method based on latent pattern embedding

Abstract

The application discloses a self-supervised heterogeneous graph node classification method based on latent pattern embedding, and mainly solves the problem of poor heterogeneous graph node classification effect in the prior art. The method comprises the following steps: 1) designing a latent repeated pattern embedding module, and constructing a subgraph sequence by selecting the k-hop neighborhood of the M most similar nodes; 2) capturing subgraph features by a GCN network, and constructing a subgraph feature sequence; 3) taking the subgraph feature sequence as the input of a Bi-LSTM to obtain a latent repeated pattern representation; 4) designing a neighborhood structure embedding (NSE) encoder to capture the high-order neighborhood representation of a node from a sequence obtained by random walk; 5) generating a negative sample sequence by a random strategy to construct a contrastive learning loss; and 6) fusing the obtained latent repeated pattern representation, high-order neighborhood representation and node self-feature to obtain a final node representation, and implementing classification by using a classifier. The application can better capture information and effectively improve the heterogeneous graph node classification accuracy.

Description

Technical Field

[0001] This invention belongs to the field of node classification technology of graph neural networks, and further relates to heterogeneous graph node classification technology. Specifically, it is a self-supervised heterogeneous graph node classification method based on latent pattern embedding, which can be used in social networks to classify different types of nodes such as users and topics to identify potential hot topics or key users. Background Technology

[0002] With the widespread adoption of the internet, in addition to popular data types such as images, text, and audio, network or graph data has become another important data type. Effective analysis and mining of graph data can significantly promote the development of related industries. Recently, graph representation learning methods have demonstrated outstanding performance in areas such as social network analysis, drug discovery, and traffic flow prediction. In social networks, graph representation learning methods can identify potential communities and key nodes, contributing to accurate recommendations and information dissemination prediction. In drug discovery, it can uncover relationships between drug molecules, accelerating new drug development. In traffic flow prediction, graph representation learning methods can effectively handle the complex structure of urban traffic networks, improving the accuracy of traffic flow prediction. With continuous technological development, graph representation learning is expected to play a significant role in more fields, driving innovation in data analysis and decision-making, and bringing more efficient and intelligent solutions to various industries. In the field of graph representation learning, many graph neural network methods have achieved good results in node classification. However, most of these methods are based on the homomorphism assumption, that is, connected nodes tend to have similar attributes, which leads to poor performance in heterogeneous graph network tasks. Compared to homogeneous network graphs, heterogeneous network graphs typically contain multiple node types and relationships, making their structure more complex. In heterogeneous network graphs, connections between different node types represent different relationships, and each relationship expresses different semantics. Compared to homogeneous network graphs, heterogeneous network graphs are closer to the diversity and complexity of data in the real world. Heterogeneous network graphs encompass richer information, containing a wider variety of node objects and relationships, thus making their representation more relevant to real-world applications, but also presenting greater challenges. Currently, most heterogeneous graph node classification methods are limited to supervised learning, while most heterogeneous graph networks in the real world lack labeled information, making these methods unsuitable for tasks lacking labels. Although some unsupervised heterogeneous network graph node classification methods exist, these methods can handle a limited number of node and relationship types, failing to fully utilize the rich semantic and structural information in the network graph.

[0003] Tang et al. proposed a novel graph decoder that reconstructs the entire neighborhood information regarding proximity and structure through Neighborhood Wasserstein Reconstruction (NWR). This decoder computes the neighborhood reconstruction loss by reconstructing node degree, node features, and neighborhood distribution. This decoder demonstrates good performance in heterogeneous graphs. Fan et al. proposed a heterogeneous graph representation method that maximizes the mutual information between feature views and topological views, utilizing the mutual information maximization between feature views and topological views for heterogeneous graph node representation. This method first constructs a feature graph, capturing the underlying structure of nodes in the feature space by measuring the distance between node pairs. Then, a cross-view representation learning module is used to capture local and global information content on the feature views and topological views of the graph. Finally, the model is forced to learn shared information in the feature space and topological space by minimizing the reconstruction loss. These methods enable the model to learn higher-quality node representations by decoding from multiple perspectives or extracting features from multiple views, which can improve the heterogeneous graph node classification performance to some extent, but they ignore the complex semantic structural relationships in heterogeneous graphs, i.e., the pattern information in heterogeneous graphs.

[0004] Existing graph neural network methods based on autoencoders focus solely on decoding node features to reconstruct direct connections. However, due to the inherently oversimplified connection reconstructions, these methods often discard a significant amount of information in the learned node representations, resulting in poor performance in heterogeneous graph network node classification tasks. Heterogeneous graph neural network methods based on contrastive learning use data augmentation (such as structural perturbations, attribute perturbations, and graph diffusion) to design views and compare them with the original graph. Accordingly, relevant view pairs (positive) are pulled together, while irrelevant view pairs (negative) are pushed apart in the latent space. However, these methods learn structural information in graph networks by disrupting the original graph topology or node attributes, failing to capture the complex structural and semantic information in heterogeneous graphs, thus impacting the success rate of node classification in heterogeneous graph networks. Traditional graph neural networks, primarily based on the homogeneity assumption, perform poorly in heterogeneous graph networks. In such networks, connected nodes may belong to different categories, and nodes of the same category are far apart. The complex structure and rich semantic information of heterogeneous graphs are crucial for node classification. Furthermore, heterogeneous graph node annotation is time-consuming and expensive, making the development of self-supervised methods particularly important. Current self-supervised heterogeneous graph methods mainly focus on learning homogeneous node representations, failing to effectively utilize the structural and semantic information of heterogeneous graphs. Therefore, how to classify heterogeneous graph nodes in the absence of labels and effectively utilize the rich structural and semantic information in heterogeneous graphs to learn more distinguishable node representations has become a technical problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0005] The purpose of this invention is to address the shortcomings of existing technologies by proposing a self-supervised heterogeneous graph node classification method based on latent pattern embedding. This method solves the problem that existing technologies cannot effectively utilize the structural and semantic information of heterogeneous graphs, resulting in poor node classification performance in the absence of labels. This invention utilizes a latent repeating pattern embedding module to capture repeating pattern information in heterogeneous graphs and embeds it into the node representation. This allows for better capture of structural and semantic information in heterogeneous graphs, thereby improving the accuracy of heterogeneous graph network node classification.

[0006] The approach to implementing this invention is as follows: First, a Latent Repeating Pattern Embedding (LRPE) module is designed. This module constructs a subgraph sequence by selecting the k-hop neighborhoods of the M most similar nodes. A GCN network is then used to capture the features of each subgraph, thus constructing a subgraph feature sequence. This sequence is used as input to a Bi-LSTM to obtain the latent repeating pattern representation. Next, a Neighborhood Structure Embedding (NSE) encoder is designed to capture the higher-order neighborhood representations of nodes from the sequence obtained from random walks. Furthermore, a negative sample sequence is generated using a random strategy to construct a contrastive learning loss, thereby improving the discriminative ability of the NSE encoder. Finally, the obtained latent repeating pattern representation, higher-order neighborhood representation, and node features are fused to obtain the final node representation. In addition, this invention also includes a neighborhood structure preservation decoder and a feature reconstruction decoder, which are used to guide the neighborhood structure extraction network and the feature fusion network, respectively.

[0007] The specific steps of this invention to achieve the above objectives are as follows:

[0008] (1) Convert the topology and node attributes of the heterogeneous graph network into an adjacency matrix A and a node feature matrix X, respectively, and construct the input dataset G = (A, X);

[0009] (2) A heterogeneous graph representation learning model is constructed, consisting of a Latent Repeating Pattern Embedding (LRPE) module, a contrastive learning module, a feature fusion module, and a decoder module. The LRPE module uses K-hop neighborhoods to construct subgraphs around nodes with the most similar attributes and inputs them into a Bidirectional Long Short-Term Memory (BiLSTM) network to capture latent repeating pattern information and thus represent semantic relationships in the heterogeneous graph network. The contrastive learning module includes positive and negative sample construction, a neighborhood structure embedding (NSE) encoder, and contrastive loss calculation, used to capture higher-order neighborhood representations of nodes and calculate contrastive loss. The feature fusion module is used to integrate the latent repeating pattern information and higher-order neighborhood representations of nodes obtained from the LRPE module and the NSE encoder with the node's self-features to obtain the final representation of the node. The decoder includes a neighborhood structure preservation decoder and a feature reconstruction decoder, used to guide the neighborhood structure extraction network and the feature fusion network, respectively, and to calculate the neighborhood structure preservation loss and the feature reconstruction loss.

[0010] (3) Use the cosine function to measure the similarity between nodes and obtain the node similarity matrix. Where N represents the number of nodes in the heterogeneous graph network; based on the node similarity matrix S, M nodes with the most similar attributes to each node are selected to form a node sequence, and a subgraph sequence is generated by sampling them;

[0011] (4) Extract the features of each subgraph to obtain the representation sequences of M subgraphs, and use Bi-LSTM to capture the repeating pattern information around the M similar nodes from the subgraph representation sequences to generate the potential repeating pattern representation h. p ;

[0012] (5) Sample the higher-order neighbors of each node in the heterogeneous graph network using a random walk strategy to generate a neighborhood topology sequence L. p and L p The positive samples are set as the negative samples; simultaneously, the same number of nodes as the positive samples are randomly sampled from the heterogeneous graph using a random strategy to form the negative sample sequence L. n Then L respectively p and L n As input to the neighborhood structure embedded NSE encoder, the node neighborhood representation h is generated using the NSE encoder. l The negative node represents h;

[0013] (6) Using a multilayer perceptron (MLP), the latent repeating pattern representation h was obtained through steps 3 and 4. p and the neighborhood representation of the node h l Integrating with the node's self-feature X, we obtain the node's final representation Z;

[0014] (7) h is represented by the node neighborhood l The learning loss L is calculated and compared with the negative node representation h. cl Simultaneously, the feature reconstruction decoder and the neighborhood structure preservation decoder are used to reconstruct node features and neighborhood structure features respectively, and the feature reconstruction loss L is calculated. f Total loss L and neighborhood structure preservation ns According to L cl L f and L ns Calculate the total loss L of the model and use the gradient descent algorithm to train and update the model;

[0015] (8) Use the trained model to obtain the final representation of the nodes in the heterogeneous graph;

[0016] (9) Use a classifier to classify nodes based on the final node representations to obtain the classification results.

[0017] Compared with the prior art, the present invention has the following advantages:

[0018] First, since this invention is the first to attempt to generate node representations in an unsupervised manner by utilizing recurring patterns in heterogeneous graphs, a Latent Repeating Pattern Embedding (LRPE) module is designed to explore recurring pattern information around similar nodes and embed it into the node representation; this makes the learned node representation contain more semantic structure information, thereby effectively improving the heterogeneous graph node classification performance.

[0019] Secondly, because this invention designs a neighborhood structure embedded NSE encoder to capture high-order neighborhood representations, it eliminates the need to use multi-layer GCNs to capture high-order neighborhood representations of nodes, thus solving the over-smoothing problem existing in current methods. Furthermore, this invention also equips the NSE encoder with a contrastive learning module to improve its discriminative ability, enabling the NSE to capture high-quality node neighborhood representations.

[0020] Third, because this invention equips the NSE encoder with a novel graph decoder to reconstruct node neighborhood information, the representation learned by the NSE encoder contains as much neighborhood information as possible, thus making it easier to distinguish nodes. Attached image description:

[0021] Figure 1 This is a flowchart illustrating the implementation of the present invention;

[0022] Figure 2 A schematic diagram of the overall architecture for implementing classification using the method of this invention;

[0023] Figure 3 Here is an example image of a repeating pattern in the WebKB network;

[0024] Figure 4 This is a schematic diagram of the LRPE module processing procedure for embedding potential repeating patterns in this invention; Detailed Implementation

[0025] The present invention will now be further described with reference to the accompanying drawings.

[0026] Example 1: Refer to Appendix Figure 1 and Figure 2 The present invention proposes a self-supervised heterogeneous graph node classification method based on latent pattern embedding, comprising the following steps:

[0027] Step 1. Convert the topology and node attributes of the heterogeneous graph network into an adjacency matrix A and a node feature matrix X, respectively, and construct the input dataset G = (A, X);

[0028] Step 2. Refer to Figure 2A heterogeneous graph node classification network model is constructed, consisting of a Latent Repeating Pattern Embedding (LRPE) module, a contrastive learning module, a feature fusion module, and a decoder module. The LRPE module uses repeating patterns around similar nodes to represent semantic relationships in the heterogeneous graph network. It constructs a subgraph around the node with the most similar attributes using K-hop neighborhoods and inputs it into a BiLSTM to capture latent repeating pattern information. The contrastive learning module includes positive and negative sample construction, a neighborhood structure embedding (NSE) encoder, and contrastive loss calculation, used to capture higher-order neighborhood representations of nodes and calculate the contrastive loss. The feature fusion module integrates the latent repeating pattern information and higher-order neighborhood representations of nodes obtained from the LRPE module and NSE encoder, respectively, with the node's self-features to obtain the final representation of the node. The decoder includes a neighborhood structure preservation decoder and a feature reconstruction decoder, used to guide the neighborhood structure extraction network and the feature fusion network, respectively, and to calculate the neighborhood structure preservation loss and the feature reconstruction loss.

[0029] The proposed LRPE module explores recurring patterns around similar nodes and embeds them into node representations to aid in node differentiation. This is the first attempt in the art to utilize recurring patterns in heterogeneous graphs in an unsupervised manner to generate node representations. A neighborhood structure embedding NSE encoder is used to capture higher-order neighborhood representations, thus addressing the oversmoothing problem, and contrastive learning is used to improve the discriminative power of the NSE encoder. To ensure that the learned representations not only capture information to differentiate nodes but also capture as much information as possible to reconstruct the features and structure of the neighborhood, this invention equips the contrastive learning framework with a novel graph decoder to reconstruct node neighborhood information. By combining the advantages of contrastive learning and autoencoders, node neighborhood information is extracted more effectively, improving node classification performance.

[0030] Step 3. Use the cosine function to measure the similarity between nodes and obtain the node similarity matrix. Where N represents the number of nodes in the heterogeneous graph network; based on the node similarity matrix S, select M nodes with the most similar attributes to each node to form a node sequence; generate a subgraph sequence by obtaining the K-hop neighborhood of each node in the node sequence, as follows:

[0031] (3.1) Measure the similarity between nodes using the cosine function:

[0032]

[0033] Among them, S i,j Let x represent the similarity between nodes i and j, ||·|| denote the second norm, and x i and x j These are the initial features of nodes i and j, respectively; i, j ∈ N and i ≠ j;

[0034] (3.2) Select M nodes with the most similar attributes from the heterogeneous graph based on the node similarity matrix to form a node sequence L. node ={n1,n2,...,n M};

[0035] (3.3) The subgraph sequence L is obtained according to the following formula. subgraph :

[0036] L subgraph =KHop(L node ;X;A),

[0037] Where X represents the node feature matrix, A is the adjacency matrix, and KHop(·) represents the function used to generate the K-hop neighborhood of a node. Here, the sampling of the subgraph uses the K-hop neighborhood, i.e., the k-hop neighbor generation method. In addition, random walk sampling can also be used to obtain the subgraph sequence.

[0038] Step 4. Extract features from each subgraph to obtain subgraph representation sequences of M subgraphs. Then, use a bidirectional long short-term memory network (Bi-LSTM) to capture repeating pattern information around these M similar nodes from the subgraph representation sequences, generating a potential repeating pattern representation h. p This embodiment extracts features for each subgraph using a graph convolutional neural network (GCN) encoder with shared weights. Of course, other methods, such as variants, can also be used for subgraph feature extraction.

[0039] Step 5. Sample the higher-order neighbors of each node in the heterogeneous graph network using a random walk strategy to generate a neighborhood topology sequence L. p and L p The positive samples are set as the negative samples; simultaneously, the same number of nodes as the positive samples are randomly sampled from the heterogeneous graph using a random strategy to form the negative sample sequence L. n Then L respectively p and L n As input to the neighborhood structure embedded NSE encoder, the node neighborhood representation h is generated using the NSE encoder. l The negative node represents h;

[0040] Step 6. Using a multilayer perceptron (MLP), the latent repeating pattern representation h obtained through steps 3 and 4 is transformed. p and the neighborhood representation of the node h l By integrating with the node's self-features, the final representation Z of the node is obtained;

[0041] Step 7. Based on the node neighborhood representation h l The learning loss L is calculated and compared with the negative node representation h. clSimultaneously, the feature reconstruction decoder and the neighborhood structure preservation decoder are used to reconstruct node features and neighborhood structure features respectively, and the feature reconstruction loss L is calculated. f Total loss L and neighborhood structure preservation ns According to L cl L f and L ns The total loss L of the model is calculated, and the gradient descent algorithm is used to train and update the model.

[0042] The contrastive learning loss L cl The mutual information between positive and negative sample representations is calculated using the MINE method of mutual information neural estimation, and the neural discriminator is trained using the Jensen-Shannon JS divergent mutual information estimator. The contrastive learning loss L is calculated according to the following formula. cl :

[0043]

[0044] in, Let represent the neighborhood representation and the negative representation of the i-th node, respectively; σ represents a neural discriminator used to provide probability scores for sampled pairs, where σ is a non-linear activation.

[0045] The feature reconstruction loss L f It is through the feature reconstruction decoder ψ F Obtain reconstructed node features Then minimize the original node features X and the reconstructed node features. The difference between them is used to determine the target loss for node feature reconstruction, i.e., the feature reconstruction loss L. f :

[0046]

[0047] Where V represents the node set, and ||·|| represents the second norm.

[0048] The total loss L of the neighborhood structure retention ns The following steps were taken to obtain the result:

[0049] (7.1) Design a neighborhood structure preservation decoder High-order neighborhood information is used to reconstruct features and structure; among which To reconstruct the neighbor node similarity decoder, To reconstruct the node feature decoder;

[0050] (7.2) Construct the loss function for reconstructing the similarity of neighboring nodes:

[0051]

[0052] Among them, X ( i ) Let be the initial features of the i-th node. Let N be the neighborhood representation of the i-th node; sim (·) is a function used to generate a sequence of node proximity similarities;

[0053] (7.3) Define the loss function for reconstructing node features as follows:

[0054]

[0055] (7.4) Calculate the total loss L of the neighborhood structure-preserving decoder according to the following formula. ns :

[0056] L ns =λ s ·L sim +λ t ·L nt ,

[0057] Where, λ s and λ t These are the similarity loss for reconstructing neighboring nodes, L. sim and reconstructed node feature loss L nt Hyperparameters.

[0058] The total loss L of the model is as follows:

[0059] L=λ cl L cl +λ ns ·L ns +λ f ·L f ,

[0060] Where, λ cl , λ ns and λ f These are used to adjust L in the total loss. cl L ns and L f The hyperparameter of the scale.

[0061] Step 8. Use the trained model to obtain the final representation of the nodes in the heterogeneous graph;

[0062] Step 9. Use a classifier to classify the nodes based on the final node representations to obtain the classification results.

[0063] Example 2: The overall implementation steps of the method proposed in this example are the same as those in Example 1. Now, in conjunction with the appendix... Figure 2-4 Specific examples are provided to further describe the implementation process of the method of the present invention in detail:

[0064] Step A, Embedding of Potential Repeating Patterns:

[0065] This invention takes into account that heterogeneous graph networks always contain a large number of recurring patterns, such as Figure 3 As shown, recurring patterns refer to connections that frequently occur between nodes of different categories. For example, a course always connects a teacher, multiple students, and a department. These connections between nodes of different categories constitute a recurring pattern. In heterogeneous graphs, these recurring patterns contain a wealth of semantic information, helping to distinguish nodes.

[0066] Due to the large size and complexity of heterogeneous graph networks, learning potential recurring patterns in an unsupervised manner is quite challenging. To address this challenge, this invention chooses to learn potential recurring pattern information from subgraphs rather than the entire heterogeneous network. However, arbitrarily partitioning the subgraph does not necessarily preserve recurring patterns and may even destroy structural information. Through observation and analysis, it has been found that nodes with similar properties in heterogeneous graph networks often exhibit similar structures in their neighborhoods, which contain recurring patterns. Inspired by this observation, this invention involves a potential recurring pattern embedding module that attempts to explore these recurring patterns in subgraphs surrounding similar nodes. The structure of this potential recurring pattern embedding module is as follows: Figure 4 As shown.

[0067] First, the similarity between every two nodes is measured using the cosine function, thus obtaining the similarity matrix.

[0068]

[0069] Here, ||·|| is the second-order norm, x i and x j These are the initial features of nodes i and j. This invention uses a similarity matrix S to select the M most similar nodes to generate a node sequence L. node ={n1,n2,...,n M Then, the subgraph sequence L is generated by obtaining the k-hop neighbors, i.e., the K-hop neighborhood, of each node in the node sequence. subgraph :

[0070] L subgraph =KHop(L node (4-2)

[0071] Here, X is the node feature matrix, A is the adjacency matrix, and KHop(·) is a function used to generate the k-hop neighborhood of a node. Next, we learn the latent repeating pattern embedding from these M subgraphs. To this end, this invention first obtains the subgraph representation sequence of the M subgraphs using a graph convolutional neural network (GCN) encoder with shared weights.

[0072]

[0073] Here, W1 is the parameter matrix of the GCN encoder. Since the Bidirectional Long Short-Term Memory (BiLSTM) network excels at computing and propagating global structural information across subgraphs, this invention utilizes BiLSTM to learn latent repeating pattern representations across subgraphs by aggregating subgraph embeddings.

[0074]

[0075] Here, W2 is the parameter matrix of the BiLSTM. The LRPE module proposed in this invention captures latent cyclic patterns in heterogeneous graphs with a relatively small input dimension, without explicit pre-settings or annotations.

[0076] Step B: Comparative Learning

[0077] In addition to representing potential recurring patterns across subgraphs, attention should also be paid to node neighborhood representations. In heterogeneous graph networks, nodes with similar characteristics may appear in higher-order neighborhoods. Most methods utilize multi-hop GCNs as encoders to expand the neighborhood, covering more distant nodes. However, multi-hop GCNs suffer from oversmoothing. To address this oversmoothing problem, this invention designs a Neighborhood Structure Embedding (NSE) encoder to capture higher-order neighborhood representations. Furthermore, this invention implements contrastive learning to improve the discriminative power of the NSE encoder.

[0078] Specifically, a random walk strategy is first used to sample the higher-order neighbors of each node in the heterogeneous graph network, thereby generating a neighborhood topology sequence.

[0079]

[0080] Compared to GCN networks, BiLSTM, when extracting node information, can preserve the features of homogeneous nodes within the neighborhood through memory gates and reduce the influence of other types of nodes through forget gates. This gating mechanism allows for different levels of attention to nodes within key regions, thus addressing the problem of over-smoothing. Therefore, this invention utilizes BiLSTM as a Neighborhood Structure Embedding (NSE) encoder to capture the neighborhood representation of nodes.

[0081]

[0082] Here, W3 is the parameter matrix of BiLSTM. The final LSTM layer hidden representation contains the representation of each node in the sequence. To train the NSE encoder to have stronger discriminative power, this invention also utilizes self-supervised contrastive learning between the representations of positive samples that capture neighborhood structure and negative samples that do not. This invention uses the neighborhood topology sequence obtained through random walk as described above. The samples were set as positive samples. In addition to the positive samples, a series of negative samples were also sampled using a random strategy. These negative samples disrupt the topology of the heterogeneous graph.

[0083] Negative samples are generated as follows:

[0084]

[0085] Subsequently, the negative sample sequence is fed into the NSE encoder, which shares weights with the positive sample encoder, thereby deriving the node negative representation h. (i) :

[0086]

[0087] Here h (i) This is the node representation generated from the negative samples. After obtaining the node representations of both positive and negative samples, this invention designs a contrastive learning loss function by calculating the mutual information between the positive and negative sample representations. Here, this invention follows the Mutual Information Neural Estimation (MINE) method, using a Jensen-Shannon (JS) divergent mutual information estimator to train the neural discriminator to approximate the mutual information and maximize the lower bound of the mutual information. The contrastive loss function is as follows:

[0088]

[0089] here σ represents the neural discriminator, which provides probability scores for sampled pairs, and σ is the nonlinear activation.

[0090] Step C, Feature Fusion:

[0091] This invention obtains latent repeating pattern representations from the aforementioned LRPE module and NSE encoder, respectively. and node neighborhood representation Then they are compared with their self-feature X using a multilayer perceptron (MLP). (i) By integrating, we obtain the final representation Z of the node. (i) :

[0092]

[0093] Here, || represents the connection operator. By embedding the potential recurrence pattern information across subgraphs and the information of higher-order neighborhood structures into the node representation, the expressive power of the node representation is greatly enhanced.

[0094] Step D, Decoder Module:

[0095] To ensure that node representations not only capture information distinguishing nodes but also capture as much information as possible to reconstruct the features and structure of the neighborhood, this invention equips contrastive learning with a powerful graph decoder capable of fully reconstructing feature and neighborhood structure information. The graph decoder of this invention consists of two parts: a feature reconstruction decoder ψ... F and neighborhood structure preservation decoder ψ n .

[0096] The feature reconstruction decoder is used to reconstruct the input node features from the computed final node representation Z. This is achieved through the feature reconstruction decoder ψ. F Obtain reconstructed node features The objective loss for node feature reconstruction is to minimize the difference between the original node features X and the reconstructed node features. It is determined by the difference between them. The specific calculation formula is as follows:

[0097]

[0098] Furthermore, this invention designs a neighborhood structure-preserving decoder to reconstruct high-order neighborhood information regarding features and structure. Neighborhood structure-preserving decoder ψ n It is further divided into two parts. decoder Reconstructing the similarity of nodes in the neighborhood Reconstructing node features. This is for decoders that reconstruct the similarity of neighboring nodes. This invention utilizes computational nodes and topological sequence L t Cosine similarity N between each node sim To obtain similarity sequences, the objective function for reconstructing the similarity of neighboring nodes is as follows:

[0099]

[0100] Here, N is the number of nodes, X(i) ) These are the initial characteristics of the nodes. This represents the neighborhood embedding representation. sim (·) is a function used to generate a sequence of node neighbor similarities. For the decoder $\psi^F_n$, it aims to reconstruct node features from the neighborhood representation, and the reconstruction loss can be defined as:

[0101]

[0102] Therefore, the neighborhood structure preserves the overall loss function L of the decoder. ns The format is as follows:

[0103] L ns =λs ·L sim +λ t ·L nt (4-14)

[0104] Step E, Overall Objective Function:

[0105] In summary, the final objective function of the proposed method consists of three parts: (1) the objective function L used for contrastive learning. cl (2) Loss function L used for neighborhood structure-preserving decoder ns (3) Loss function L used for node feature reconstruction decoder f The overall objective function of this invention is defined as follows:

[0106] L=λ cl L cl +λ ns ·L ns +λ f ·L f (4-15)

[0107] Here λ cl , λ ns and λ f These are the hyperparameters that represent the trade-offs between contrast loss, neighborhood structure preservation decoder loss, and node feature reconstruction decoder loss. This invention can easily minimize this objective using the stochastic standard gradient descent algorithm.

[0108] Step F, Model Training

[0109] First, this invention uses an adjacency matrix and a node feature matrix to represent a heterogeneous graph network. The adjacency matrix represents the connections between nodes, and the node feature matrix represents the feature information of each node. Next, the model parameters, including weights and biases, are initialized. Then, the adjacency matrix and node feature matrix are input into the model to obtain repeating pattern embeddings, positive and negative sample neighborhood representations of nodes, and the final node representation. The outputs of these networks are then used to calculate the loss, and the gradient of the loss with respect to the network parameters is calculated using the backpropagation algorithm. These gradients are used to update the network parameters to minimize the loss function. Finally, optimization algorithms such as gradient descent are used to update the network parameters, and forward propagation, loss calculation, backpropagation, and parameter updates are repeated until the network converges or reaches a predetermined number of training iterations. Finally, the test data is input into the trained model to obtain the final node representation.

[0110] Step G, Node Classification

[0111] This invention inputs the final node representation into a trained classifier to perform node classification. The classifier can be a simple multilayer perceptron (MLP) network. The classifier then determines the category of nodes in the heterogeneous graph network.

[0112] Step H, Model Deployment

[0113] The machine device in this embodiment includes a processor and a memory. The present invention stores the trained model in the memory of the machine device. After inputting heterogeneous graph data, the machine device calls the trained model in the memory to process the unlabeled heterogeneous graph node classification task and outputs the final classification result.

[0114] The effects of the present invention will be further explained below with reference to simulation experiments.

[0115] 1. Simulation conditions:

[0116] All settings in this experiment used the Adam optimizer and backpropagation algorithm from the PyTorch Python package, with the dimension size being the same as the feature size of the graph nodes. All experiments were conducted on a 24GB NVIDIA GeForce RTX 4090 GPU.

[0117] 2. Simulation content:

[0118] To demonstrate the effectiveness of this invention, its node classification performance was compared with state-of-the-art graph neural network methods, including NWR-GAE and SELENE based on graph autoencoders, and GraphCL, GRACE, and MVMI-FT based on contrastive learning. This invention used four real-world datasets: Cornell, Texas, Wisconsin, and Actor for testing. The Cornell, Texas, and Wisconsin datasets are WebKB webpage datasets, where nodes represent webpages from different university computer science departments, and edges represent links between webpages. Actor is an actor co-occurrence network, where nodes represent actors, and edges indicate that two actors co-occur on the same Wikipedia page.

[0119] 3. Simulation results:

[0120] As shown in Table 1, compared with all baselines, the method of this invention achieves significantly better results on four heterogeneous graph network datasets. The error bars (±) in the figure represent the standard deviation of the results of 5 trials. Specifically, the method of this invention significantly outperforms the state-of-the-art (SOTA) unsupervised graph embedding methods, achieving improvements of 9.56% on the Cornell dataset, 3.35% on the Texas dataset, 4.39% on the Wisconsin dataset, and 3.46% on the Actor dataset.

[0121] Table 1 shows the node classification accuracy (%) of various methods on three real-world datasets.

[0122]

[0123] The above simulation analysis proves the correctness and effectiveness of the method proposed in this invention.

[0124] The parts of this invention not described in detail are common knowledge to those skilled in the art.

[0125] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, those skilled in the art, after understanding the content and principle of the present invention, may make various modifications and changes in form and detail without departing from the principle and structure of the present invention. However, these modifications and changes based on the concept of the present invention are still within the scope of protection of the claims of the present invention.

Claims

1. A self-supervised heterogeneous graph node classification method based on latent pattern embedding, characterized in that, Includes the following steps: (1) Convert the topology and node attributes of the heterogeneous graph network into an adjacency matrix A and a node feature matrix X, respectively, and construct the input dataset G = (A, X); (2) Construct a heterogeneous graph representation learning model consisting of a latent repeating pattern embedding (LRPE) module, a contrastive learning module, a feature fusion module, and a decoder module; The LRPE module utilizes K-hop neighborhoods to construct subgraphs around nodes with the most similar attributes and inputs them into a bidirectional long short-term memory network (BiLSTM) to capture potential repeating pattern information, thereby characterizing semantic relationships in heterogeneous graph networks. The contrastive learning module includes positive and negative sample construction, neighborhood structure embedding NSE encoder, and contrastive loss calculation, used to capture the higher-order neighborhood representation of nodes and calculate the contrastive loss; the feature fusion module is used to integrate the latent repeating pattern information and the higher-order neighborhood representation of nodes obtained from the LRPE module and NSE encoder respectively with the node's self-features to obtain the final representation of the node; the decoder includes a neighborhood structure preservation decoder and a feature reconstruction decoder, used to guide the neighborhood structure extraction network and the feature fusion network respectively, and to calculate the neighborhood structure preservation loss and the feature reconstruction loss; (3) Use the cosine function to measure the similarity between nodes and obtain the node similarity matrix. Where N represents the number of nodes in the heterogeneous graph network; based on the node similarity matrix S, M nodes with the most similar attributes to each node are selected to form a node sequence, and a subgraph sequence is generated by sampling them; (4) Extract the features of each subgraph to obtain the representation sequences of M subgraphs, and use Bi-LSTM to capture the repeating pattern information around the M similar nodes from the subgraph representation sequences to generate the potential repeating pattern representation h. p ; (5) Sample the higher-order neighbors of each node in the heterogeneous graph network using a random walk strategy to generate a neighborhood topology sequence L. p and L p The positive samples are set as positive samples; simultaneously, a negative sample sequence L is constructed by randomly sampling the same number of nodes as the positive samples from the heterogeneous graph using a random strategy. n Then L respectively p and L n As input to the neighborhood structure embedded NSE encoder, the node neighborhood representation h is generated using the NSE encoder. l The negative node represents h; (6) Using a multilayer perceptron (MLP), the latent repeating pattern representation h was obtained through steps 3 and 4. p and the neighborhood representation of the node h l Integrating with the node's self-feature X, we obtain the node's final representation Z; (7) h is represented by the node neighborhood l The learning loss L is calculated and compared with the negative node representation h. cl Simultaneously, the feature reconstruction decoder and the neighborhood structure preservation decoder are used to reconstruct node features and neighborhood structure features respectively, and the feature reconstruction loss L is calculated. f Total loss L and neighborhood structure preservation ns According to L cl L f and L ns Calculate the total loss L of the model and use the gradient descent algorithm to train and update the model; (8) Use the trained model to obtain the final representation of the nodes in the heterogeneous graph; (9) Use a classifier to classify nodes based on the final node representations to obtain the classification results.

2. The method according to claim 1, characterized in that: The subgraph sequence is generated in step (3), and the specific implementation steps are as follows: (3.1) Measure the similarity between nodes using the cosine function: Among them, S i,j Let x represent the similarity between nodes i and j, ||·|| denote the second norm, and x i and x j These are the initial features of nodes i and j, respectively; i, j ∈ N and i ≠ j; (3.2) Select M nodes with the most similar attributes from the heterogeneous graph based on the node similarity matrix to form a node sequence L. node ={n1,n2,...,n M }; (3.3) The subgraph sequence L is obtained according to the following formula. subgraph : L subgraph = KHop(L) node ;DISTANT), Where X represents the node feature matrix, A is the adjacency matrix, and KHop(·) represents the function used to generate the K-hop neighborhood of a node.

3. The method according to claim 2, characterized in that: The subgraph sequence in step (3.3) can also be generated by random walk sampling.

4. The method according to claim 1, characterized in that: In step (4), features of each subgraph are extracted by a graph convolutional neural network (GCN) encoder or variant with shared weights.

5. The method according to claim 1, characterized in that: The contrastive learning loss L mentioned in step (7) cl The mutual information between positive and negative sample representations is calculated using the MINE method of mutual information neural estimation, and the neural discriminator is trained using the Jensen-Shannon JS divergent mutual information estimator. The contrastive learning loss L is calculated according to the following formula. cl : in, h (i) Let represent the neighborhood representation and the negative representation of the i-th node, respectively; σ represents a neural discriminator used to provide probability scores for sampled pairs, where σ is a non-linear activation.

6. The method according to claim 1, characterized in that: The feature reconstruction loss L mentioned in step (7) f It is through the feature reconstruction decoder ψ F Obtain reconstructed node features Then minimize the original node features X and the reconstructed node features. The difference between them is used to determine the target loss for node feature reconstruction, i.e., the feature reconstruction loss L. f : Where V represents the node set, and ||·|| represents the second norm.

7. The method according to claim 1, characterized in that: In step (7), the total loss L for neighborhood structure preservation is... ns The following steps were taken to obtain the result: (7.1) Design a neighborhood structure preservation decoder High-order neighborhood information is used to reconstruct features and structure; among which To reconstruct the neighbor node similarity decoder, To reconstruct the node feature decoder; (7.2) Construct the loss function for reconstructing the similarity of neighboring nodes: Among them, X (i) Let be the initial features of the i-th node. Let N be the neighborhood representation of the i-th node; sim (·) is a function used to generate a sequence of node proximity similarities; (7.3) Define the loss function for reconstructing node features as follows: (7.4) Calculate the total loss L of the neighborhood structure-preserving decoder according to the following formula. ns : L ns =λ s ·L sim +λ t ·L nt , Where, λ s and λ t These are the similarity loss L for reconstructing neighboring nodes. sim and reconstructed node feature loss L nt Hyperparameters.

8. The method according to claim 1, characterized in that: The total loss L of the model described in step (7) is as follows: L=λ cl L cl +λ ns ·L ns +λ f ·L f , Where, λ cl , λ ns and λ f These are used to adjust L in the total loss. cl L ns and L f The hyperparameter of the scale.

Images