A heterogeneous information network link prediction method based on a contrastive learning mechanism

Abstract

The application discloses a heterogeneous information network link prediction method based on a contrast learning mechanism, and comprises the following steps: a text encoder is used to encode a text into a low-dimensional vector to generate a text representation; a graph encoder is used to encode structural features, heterogeneous features and self-supervised information of the heterogeneous information network to obtain a graph representation; the text representation and the graph representation are pre-trained and aligned through contrast learning; a self-generated learnable and continuous prompt vector is introduced, a natural language sentence is provided to the text encoder, and the natural language sentence is compared with structural and heterogeneous feature representations generated by the graph encoder to generate weights during classification, so that a single representation is obtained through fusion; and the obtained single representation is used for link prediction of the heterogeneous information network. The method can obtain more excellent and significant prediction performance in the link prediction task of the heterogeneous information network.

Description

Technical Field

[0001] This invention relates to the field of knowledge graph network technology in natural language processing, and in particular to a method for predicting links in heterogeneous information networks based on a contrastive learning mechanism. Background Technology

[0002] Heterogeneous information networks are ubiquitous. Interactions between users and items in social networks, knowledge graphs, and search and recommendation systems can be modeled as networks with various types of nodes and edges. Textual heterogeneous information networks are networks with textual information, such as the titles and abstracts of paper nodes in academic networks, which can provide productive auxiliary information for downstream tasks. Most current work on heterogeneous information networks neglects this textual information and maps graph nodes to low-dimensional representations based solely on structural information. To fill this gap, some models for mining heterogeneous information networks propose integrating textual information into node representations. They primarily design a framework that combines the structural and textual information of nodes to generate a single node representation.

[0003] The text network embedding models mentioned above face several limitations. First, they can only classify nodes using pre-trained labels; in other words, they are unsuitable for few-shot learning settings. In few-shot learning, we need to transfer the pre-trained model to classify nodes with unseen labels during the testing phase. In practice, typically only a few labels are available, posing a significant challenge to maintaining performance. Second, previous methods using textual information were originally designed for homogeneous information networks, and no work has yet attempted to address the few-shot learning problem on heterogeneous textual information networks.

[0004] To address the few-shot learning problem, natural language processing research (e.g., ChatGPT) has proposed cue learning, which re-engineers downstream tasks to resemble pre-trained tasks. Whether or not fine-tuning is used, cue learning facilitates the rapid application of prior knowledge to new tasks, thereby enhancing few-shot learning. Recently, cue learning has also been employed in multimodal scenarios for aligning image and text data. However, currently, no cue learning-based techniques have been used to process both graph and text data.

[0005] In view of this, a heterogeneous information network link prediction method based on contrastive learning mechanism is proposed. This method applies cue learning to graph data to solve the few-sample learning problem in textual heterogeneous information networks and obtains more efficient and accurate results for heterogeneous information network link prediction tasks. Summary of the Invention

[0006] This invention aims to address at least one of the technical problems existing in the prior art. To this end, this invention discloses a link prediction method for heterogeneous information networks based on a contrastive learning mechanism. The method employs a text encoder to encode text information; a graph encoder to encode structural and heterogeneous features as well as self-supervised information; a contrastive learning mechanism to align text and network representations; and a learnable continuous vector-based cue learning framework to address the few-sample problem in textual heterogeneous information networks.

[0007] A method for predicting links in heterogeneous information networks based on a contrastive learning mechanism, the method comprising:

[0008] Step 1: Use a text encoder to encode the text into a low-dimensional vector to generate a text representation;

[0009] Step 2: Use a graph encoder to encode the structural features, heterogeneous features, and self-supervised information of the heterogeneous information network to obtain a graph representation;

[0010] Step 3: Align the text representation and the graph representation through comparative learning during pre-training.

[0011] Step 4: Introduce automatically generated learnable and continuous cue vectors, provide the labeled natural language statements to the text encoder, and compare the natural language statements with the structural and heterogeneous feature representations generated by the graph encoder to generate weights for classification, and fuse them to obtain a single representation;

[0012] Step 5: Use the obtained single representation to perform link prediction for heterogeneous information networks.

[0013] Specifically, the text encoder uses the Sentence-BERT model to generate fixed-size text representations.

[0014] Specifically, step 2 includes the following steps:

[0015] Step 201: Sample the heterogeneous subgraph. For a given node, the subgraph surrounding the node needs to be sampled first.

[0016] Step 202: An autoencoder is used to capture the structural information of the subgraph. Given the adjacency matrix A of the subgraph, it will first be processed by the encoder to generate a multi-layered latent representation. Then, the decoder reverses the above process to obtain the reconstructed output. Autoencoders aim to minimize the reconstruction error of the input and output, enabling nodes with similar structures to have similar representations, with a loss function L. structure The calculation formula is as follows:

[0017]

[0018] Where B is the penalty sparsity imposed on non-zero elements to mitigate sparsity problems, and ⊙ represents bitwise multiplication. Indicates a regularization operation;

[0019] Step 203: Explore the heterogeneous features of the heterogeneous information network by grouping nodes of the same type together and applying Bi-LSTM to each group to model type-specific features, given a type T. j Node group Representation of node v The calculation is as follows:

[0020]

[0021] Where Bi-LSTM{v} represents applying Bi-LSTM to the type grouping of node v. Represents a node group Quantity;

[0022] Then, an attention mechanism is applied to aggregate all groups of types to generate a representation h for a given node. v ,

[0023]

[0024] Where δ represents the activation function, using LeakyReLU, u∈R d It is the weight parameter, u T This represents the transpose of u. Let {T} represent the node v, and {T} represent the set of types, α v,j Indicates attention weight;

[0025] Step 204: Based on the self-supervised information pre-trained subgraph, two pre-training tasks are introduced: mask node modeling task and edge reconstruction task, to achieve node-level and edge-level graph exploration.

[0026] Specifically, in the masked node modeling task, nodes are sorted according to their ranking, and a predetermined proportion of nodes are randomly selected and labeled with [MASK]. The sorted nodes are then fed into the Transformer encoder. The representation generated by Bi-LSTM is used as the identifier, and the sorting information is used as the position vector. The hidden layers learned by the Transformer encoder are then processed. It will be fed into the feedforward layer to predict the target node, mathematically represented as:

[0027]

[0028] p v =softmax(W MNM z v ),

[0029] Among them, z v This is the output of the feedforward layer. `Feedforward()` indicates the output from the feedforward layer, `softmax()` represents the activation function, and `W`... MNM ∈V v ×d is the weight shared with the input node representation matrix for classification, V v d is the number of nodes in the subgraph, d is the dimension of the hidden layer vector, and p is the number of nodes in the subgraph. v This is the predicted distribution of v across all nodes, using one-hot labels during training. and prediction Cross-entropy and loss function L MNM The calculation is as follows:

[0030]

[0031] Among them, y i and p i It is y i and p i The i-th component, y i p represents a set of labels i A set representing predicted probabilities;

[0032] The edge reconstruction task involves sampling positive and negative edges in the subgraph. Positive edges are those that actually exist in the original subgraph, while negative edges do not. Given a merged set N of positive and negative edges... S The score for edge reconstruction is calculated using the inner product between a pair of nodes. It calculates the score, h v Let 'v' be the representation of node v, '·' be the inner product, and 'h' be the value of node v. u The representation of node u uses the binary cross-entropy between predicted and true edges to calculate the loss function L for edge reconstruction. ER :

[0033]

[0034] |N S | represents the number of node pairs, BinaryCrossEntropy() represents the binary cross-entropy, e uv Let (u,v) represent the actual scores of nodes u and v, and (u,v) represent the edge connecting nodes u and v.

[0035] Furthermore, the sampling strategy for sampling the subgraph around a node employs a random walk with restart, which iteratively traverses the neighborhood of a given node v and returns to the starting node v with a certain probability. In order to sample nodes with higher importance, the random walk strategy first reaches nodes with higher ranking. In order to make the graph encoder heterogeneous, the traversal is restricted to sampling all types of nodes.

[0036] Specifically, the contrastive learning is used to align text representations and graph representations during training. The learning objective is designed as a contrastive loss function that, given a set of text-subgraph pairs, maximizes the similarity score of matching text-subgraph pairs while minimizing the score of non-matching text-subgraph pairs.

[0037] In the contrastive learning process, given a node v, the node representation learned by the graph encoder is H, and the weight vector generated by the text encoder is denoted as H. Where K represents the number of categories, and each weight w i They all learn from the prompts, and the predicted probability can be calculated as follows:

[0038]

[0039] Where τ is the learned temperature hyperparameter, and <·,·> represent the similarity score. <w i H> represents the text weight vector w i The similarity score between nodes represents the vector H.

[0040] Specifically, the introduction of automatically generated, learnable, and continuous cue vectors in step 4 involves replacing discrete text words with continuous vectors learned end-to-end from the data. The cue P input to the text encoder is designed as follows:

[0041] P = [V1][V2]...[V M [CLASS]

[0042] Where [CLASS] represents the node's category label, [V M [] is a word vector with the same dimension as the word representation in the training phase. M is a hyperparameter representing the number of consecutive text vectors in the prompt. After inputting the consecutive prompts P into the text encoder Text(·), the classification weight vectors representing the node concepts can be obtained. The predicted probability is calculated as follows:

[0043]

[0044] Among them, each prompt P i The category markers in Text(P) are replaced by the word vector representation of the i-th category name. i ) indicates that a prompt will be made for P. iThe vector obtained after being fed into the text encoder.

[0045] Preferably, to obtain a more accurate cue vector in step 4, a residual connection based on the text encoder and the graph encoder is used to utilize the context subgraph of a given node. The text representation of the category label and the node representation in the subgraph are input into the text-subgraph self-attention layer to help the text features find the most relevant context node of a given node.

[0046] Obtain the output D of the text-subgraph comparator e Then, the text features are updated using residual connections.

[0047] Text(P)←Text(P)+λD e

[0048] Here, λ is a learnable parameter used to control the degree of residual connectivity.

[0049] Preferably, λ is initialized to 10. -4 A very small value, so that prior linguistic knowledge from text features can be preserved to the greatest extent possible.

[0050] Compared with existing methods, the advantages of this invention are: it proposes a cue learning framework to utilize textual information in heterogeneous textual networks while simultaneously addressing few-shot learning problems; and it introduces a graph encoder that captures the structure and heterogeneous features of the heterogeneous network while preserving the node-level and edge-level self-supervised information of the network subgraphs. Therefore, this invention's heterogeneous network link prediction method based on a contrastive learning mechanism achieves significantly superior prediction performance in heterogeneous network link prediction tasks. Attached Figure Description

[0051] Figure 1 A flowchart illustrating an embodiment of the present invention is shown;

[0052] Figure 2 A schematic diagram of the pre-training framework in an embodiment of the present invention is shown;

[0053] Figure 3 A schematic diagram of the prompting learning optimization framework according to an embodiment of the present invention is shown. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this invention, and not all embodiments. Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this invention.

[0055] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0056] Prerequisites:

[0057] Let G = (V, E, T) denote a heterogeneous information network, where V and E represent the node set and edge set, respectively; T V and T E These represent the set of node types and the set of edge types, respectively. A heterogeneous information network is |T V |>1 and / or|T E The network is 1.

[0058] like Figure 1 The present invention provides a heterogeneous information network link prediction method based on a contrastive learning mechanism, the method comprising:

[0059] Step 1: Use a text encoder to encode the text into a low-dimensional vector to generate a text representation;

[0060] Step 2: Use a graph encoder to encode the structural features, heterogeneous features, and self-supervised information of the heterogeneous information network to obtain a graph representation;

[0061] Step 3: Align the text representation and the graph representation through comparative learning during pre-training.

[0062] Step 4: Introduce automatically generated learnable and continuous cue vectors, provide the labeled natural language statements to the text encoder, and compare the natural language statements with the structural and heterogeneous feature representations generated by the graph encoder to generate weights for classification, and fuse them to obtain a single representation;

[0063] Step 5: Use the obtained single representation to perform link prediction task for heterogeneous information networks.

[0064] The method of this invention mainly includes a text encoder and a graph encoder, which encode text and network subgraphs into low-dimensional vectors, respectively. In the embodiment, Sentence-Bert (SBERT) is used as the text encoder to generate text representations; for the graph encoder, the subgraph to be processed is first sampled, and all types of nodes are forced to be sampled to ensure heterogeneity. Then, an autoencoder mechanism is applied to explore structural features, and Bi-LSTM is applied to nodes grouped by type to characterize the heterogeneity of the graph.

[0065] Two graph pre-training tasks, masked node modeling and edge reconstruction, are introduced to leverage self-supervised information at the node and edge levels. A contrastive learning framework is then introduced to align these two representations. Specifically, given a pair of text and subgraphs, they are considered matched if they both belong to a given node. The contrastive learning framework maximizes the similarity score of matched text-subgraph pairs and minimizes the similarity score of mismatched text-subgraph pairs.

[0066] The pre-trained model needs to be transferred to downstream tasks to adapt to few-shot settings. Specifically, during the optimization phase, for each new prediction task, classification weights are generated by providing natural language statements describing the categories of interest to the text encoder and comparing them with the structural and heterogeneous feature representations generated by the network encoder. How then should we design prompts that are crucial for downstream tasks? Even subtle changes to the words in the prompts can affect the model's performance. This embodiment does not use handwritten prompts like "a paper of [CLASS] domain," but instead introduces automatically generated, learnable, and continuous prompt vectors. This automated prompting mechanism can bring more task-relevant and efficient transfer effects to the pre-trained model.

[0067] The specific technical solution is as follows.

[0068] Text Encoder

[0069] The pre-training framework in this embodiment is as follows: Figure 2 As shown, it consists of two encoders: a text encoder and a graph encoder. The text encoder maps natural language text to low-dimensional representation vectors. The Sentence-BERT (SBERT) model is used to generate fixed-size text representations.

[0070] Map Encoder

[0071] Graph encoders map network data into low-dimensional representations.

[0072] Heterogeneous graph sampling

[0073] For a given node, the subgraph surrounding the node is first sampled, and then the subgraph is processed by a graph encoder to generate a node representation. After sampling the subgraph, the nodes in the subgraph are ranked using a centrality metric that evaluates the importance of the nodes.

[0074] A sampling strategy of random walk with restart is employed. It iteratively traverses the neighborhood of a given node v and returns to the starting node v with a certain probability. To sample nodes of higher importance, the walk strategy first reaches nodes with higher ranking. To make the encoder heterogeneous, the traversal is restricted to sampling nodes of all types.

[0075] Structural modules

[0076] First, an autoencoder is used to capture the structural information of the subgraph. Given the adjacency matrix A of the subgraph, it is first processed by the encoder to generate a multi-layered latent representation. Then, the decoder reverses the above process to obtain the reconstructed output. An autoencoder aims to minimize the reconstruction error between the input and output, enabling nodes with similar structures to have similar representations. Mathematically,

[0077]

[0078] Where B is the penalty sparsity imposed on non-zero elements to mitigate sparsity problems, and ⊙ represents bitwise multiplication. This indicates a regularization operation.

[0079] Heterogeneous modules

[0080] To explore the heterogeneous features of the network, nodes of the same type are first grouped together. This operation may disrupt the structure of the subgraphs; however, the previously employed autoencoder has already preserved structural features. Then, a Bi-LSTM is applied to each group to model type-specific features. Bi-LSTM is able to capture the interactions of node features and has extensive sequence representation capabilities. Given type T... j Node group Representation of node v The calculation is as follows:

[0081]

[0082] Where Bi-LSTM{v} represents applying Bi-LSTM to the type grouping of node v. Represents a node group Quantity;

[0083] Then, an attention mechanism is applied to aggregate all groups of types to generate a representation h for a given node. v .

[0084]

[0085] Where δ represents the activation function, using LeakyReLU, u∈R d It is the weight parameter, u T This represents the transpose of u. Let {T} represent the node v, and {T} represent the set of types, α v,j This represents the attention weight.

[0086] Self-supervised pre-training

[0087] Furthermore, subgraphs are pre-trained based on self-supervised information. Specifically, two pre-training tasks are introduced: Masked Node Modeling (MNM) and Edge Reconstruction (ER), to achieve node-level and edge-level graph exploration.

[0088] For the masked node modeling task, we sort the nodes according to their ranking and randomly select 15% of the nodes to be labeled with [MASK]. The sorted nodes are then fed into the Transformer encoder, where the representation generated by Bi-LSTM is used as the label representation, and the ranking information is used as the position vector. The hidden layers learned by the Transformer encoder... It will be fed into the feedforward layer to predict the target node, mathematically.

[0089]

[0090] p v =softmax(W MNM z v ), (6)

[0091] Among them, z v This is the output of the feedforward layer. `Feedforward()` indicates the output from the feedforward layer, `softmax()` represents the activation function, and `W`... MNM ∈V v ×d is the weight shared with the input node representation matrix for classification, V v d is the number of nodes in the subgraph, d is the dimension of the hidden layer vector, and p is the number of nodes in the subgraph. v This is the predicted distribution of v across all nodes, using one-hot labels during training. and prediction Cross-entropy and loss function L MNM The calculation is as follows:

[0092]

[0093] Among them, y i and p i It is y i and p i The i-th component, y i p represents a set of labels i A set representing predicted probabilities.

[0094] The edge reconstruction task involves sampling positive and negative edges in the subgraph. Positive edges are those that actually exist in the original subgraph, while negative edges do not. In practice, |N can be set. S | = 6, and the number of positive edges and negative edges are the same. Given the merge set N of positive and negative edges.S The score for edge reconstruction is calculated using the inner product between a pair of nodes. It calculates the score, h v Let 'v' be the representation of node v, '·' be the inner product, and 'h' be the value of node v. u The representation of node u uses the binary cross-entropy between predicted and true edges to calculate the loss function L for edge reconstruction. ER :

[0095]

[0096] |N S | represents the number of node pairs, BinaryCrossEntropy() represents the binary cross-entropy, e uv Let (u,v) represent the actual scores of nodes u and v, and (u,v) represent the edge connecting nodes u and v.

[0097] Pre-training through comparative learning

[0098] This embodiment aligns the representation spaces of text and graphs during training, and its learning objective is designed as a contrastive loss function. Specifically, given a set of text-subgraph pairs, this embodiment aims to maximize the similarity score of matching text-subgraph pairs while minimizing the score of mismatched text-subgraph pairs. For example, given a subgraph of a node, if the text information is a summary of that node, then the text-subgraph pair is matched; otherwise, it is not matched because the text information is irrelevant to that node. The similarity score is calculated using cosine similarity.

[0099] In contrastive learning environments, high-quality negative samples can help improve model performance. Therefore, in training batches, the text and subgraphs used are selected from nodes with the same label to make them difficult to distinguish.

[0100] Figure 3 This demonstrates a cue-based learning optimization framework. This embodiment can be applied to experimental environments with few samples. The pre-trained model, when faced with samples of new label types, can predict whether a node's subgraph matches the text description. This is achieved by comparing the node representations generated by the graph encoder with the classification weights generated by the text encoder. The text description can be used to specify the node category of interest, even if that category has few samples. Given a node v, the node representation learned by the graph encoder is H, and the weight vector generated by the text encoder is... Where K represents the number of categories. Each weight w iAll learning occurs from prompts, such as "a paper of [CLASS]domain," where "CLASS" can be a specific category name, such as "Information Retrieval," "database," or "data mining." To facilitate linking downstream tasks to the prediction, the prompt can also be designed as "The two nodes are [CLASS]," a binary identifier such as "connected" and "unconnected." Mathematically, the prediction probability can be calculated as...

[0101]

[0102] Where τ is the learned temperature hyperparameter, and <·,·> represent the similarity score. <w i H> represents the text weight vector w i The similarity score between nodes represents the vector H.

[0103] Continuous prompts

[0104] Traditional cue learning methods employ expert-designed manual cues. This embodiment replaces discrete text words with continuous vectors that can be learned end-to-end from the data. Specifically, the cue P input to the text encoder should be designed as follows:

[0105] P = [V1][V2]...[V M [CLASS], (10)

[0106] Where [CLASS] represents the node's category label, [V M `P` is a word vector with the same dimension as the word representations during training, and `M` is a hyperparameter representing the number of consecutive text vectors in the prompt. After inputting the consecutive prompts `P` into the text encoder `Text(·)`, the classification weight vectors representing the node concepts are obtained. Mathematically, the predicted probability is calculated as...

[0107]

[0108] Among them, each prompt P i The category markers in Text(P) are replaced by the word vector representation of the i-th category name. i ) indicates that a prompt will be made for P. i The vector obtained after being fed into the text encoder.

[0109] Residual connection

[0110] Considering the context nodes of a given node, such as the author node of a paper node, helps the text encoder become more accurate. Therefore, to further inform the pre-trained language model, a residual connection based on the text encoder and the graph encoder is used to leverage the context subgraph of a given node. First, the text representation of the category label and the node representation in the subgraph are input into a text-subgraph self-attention layer to help the text features find the most relevant context nodes for a given node. The output D of the text-subgraph contrastor is then obtained. e Then, the text features are updated through residual connections, as shown below.

[0111] Text(P)←Text(P)+λD e (12)

[0112] Here, λ is a learnable parameter used to control the degree of residual connectivity. λ is initialized to a small value of 10. -4 This is so that prior linguistic knowledge derived from text features can be preserved to the greatest extent possible.

[0113] To optimize the text vectors, training is performed to minimize the standard classification loss function based on cross-entropy. Gradients are backpropagated through the text encoder Text(·) to leverage the rich knowledge encoded in the parameters. The choice of continuous text vectors also allows for a full exploration of the word representation space, thereby improving the learning of task-relevant text.

[0114] This embodiment considers three real-world datasets: OAG, Yelp, and Reddit. OAG is an academic network with four types of nodes, selecting titles and abstracts as text, and classifying the corresponding paper nodes into five categories: (1) information retrieval, (2) databases, (3) data mining, (4) machine learning, and (5) natural language processing. Yelp is a venue check-in network with four types of nodes. The text description is a review of the restaurant, which is divided into five types: (1) Chinese, (2) fast food, (3) sushi, (4) French, and (5) Indian. Reddit is a dataset extracted from the online forum Reddit; the text description is a review of the posts, which are categorized into different communities. OAG and Yelp have only 5 labels for small sample classification, while Reddit has 42 labels, which verifies that the method of this invention can adapt to different real-world scenarios.

[0115] The dataset was divided into 80% training dataset, 10% validation dataset, and 10% test dataset. Table 1 summarizes the information of the dataset.

[0116] Table 1: Data set statistics.

[0117]

[0118] This embodiment selects several baseline models specifically for text graphs for comparison. TADW uses a matrix factorization framework to incorporate text features into the representation. CENE treats text as nodes to integrate textual and structural information. CANE learns text-aware node representations through a mutual attention mechanism that models node semantics. WANE incorporates text features into node representations by matching important words between text sequences of all node pairs. NEIFA proposes a deep neural architecture that can effectively fuse structural and textual information into a single representation. DetGP proposes a Gaussian process to dynamically model structural and textual information.

[0119] All representation vectors are fixed at 512 dimensions. For the text encoder, the vocabulary size is 49,152, with each text sequence fixed at 77, including [SOS] and [EOS] labels. Text vectors during optimization are initialized with a zero-mean Gaussian distribution with a standard deviation of 0.02. The number of text words is set to 8 during training. Training is performed using stochastic gradient descent with an initial learning rate of 0.002, decaying using a cosine annealing rule. The maximum number of training epochs is set to 200. To mitigate potentially explosive gradients in early training iterations, a warm-up technique is used to fix the learning rate at 1e-5 in the first training epoch. Three labels are used for training in OAG and Yelp, with the remaining labels used for testing. On the Reddit dataset, data with 31 labels is used for training, and the rest for testing. For the baseline model, the best parameter configuration reported in the original paper is used directly. Five samples (5-shot) are used for training in each category.

[0120] Experiments were conducted using an Intel(R) Xeon(R) Platinum 8268 CPU and a Tesla V100 to run pre-training and downstream tasks.

[0121] This embodiment evaluates the performance of the method and benchmark model of the present invention on the link prediction task. ACC and Macro-F1 scores are used as evaluation metrics (averaged over five trials).

[0122] Table 2 shows the experimental results of the link prediction task; the highest score is shown in bold.

[0123]

[0124] The method of this invention consistently outperforms the baseline on all datasets, demonstrating the effectiveness of the model. Specifically, the fine-tuning phase was adjusted in the experiments to enable the model to perform the link prediction task. Specifically, the link prediction task was transformed into a binary classification task, where labels determine whether pairs of nodes in a subgraph are connected.

[0125] This can be attributed to two main reasons. First, residual connections were used, encoding the contextual information of nodes into the representation vector of the labeled text. This contextual information helps in discovering the existence of links. Second, the cue learning setting helps to effectively mine text features, thereby achieving better performance in downstream tasks.

[0126] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

Claims

1. A method for predicting links in heterogeneous information networks based on a contrastive learning mechanism, characterized in that, Includes the following steps: Step 1: Use a text encoder to encode the text into a low-dimensional vector to generate a text representation; Step 2: Use a graph encoder to encode the structural features, heterogeneous features, and self-supervised information of the heterogeneous information network to obtain a graph representation; Step 3: Align the text representation and the graph representation through comparative learning during pre-training. Step 4: Introduce automatically generated learnable and continuous cue vectors, provide the labeled natural language statements to the text encoder, and compare the natural language statements with the structural and heterogeneous feature representations generated by the graph encoder to generate weights for classification, and fuse them to obtain a single representation; Step 5: Use the obtained single representation to perform link prediction in heterogeneous information networks; Step 2 specifically includes the following steps: Step 201: Sample the heterogeneous subgraph. For a given node, the subgraph surrounding the node needs to be sampled first. Step 202: An autoencoder is used to capture the structural information of the subgraph. Given the adjacency matrix A of the subgraph, it will first be processed by the encoder to generate a multi-layered latent representation. Then, the decoder reverses the above process to obtain the reconstructed output. Autoencoders aim to minimize the reconstruction error of the input and output, enabling nodes with similar structures to have similar representations, with a loss function L. structure The calculation formula is as follows: Where B is the penalty sparsity imposed on non-zero elements to mitigate sparsity problems, and ⊙ represents bitwise multiplication. Indicates a regularization operation; Step 203: Explore the heterogeneous features of the heterogeneous information network by grouping nodes of the same type together and applying Bi-LSTM to each group to model type-specific features, given a type T. j Node group Representation of node v The calculation is as follows: Where Bi-LSTM{v} represents applying Bi-LSTM to the type grouping of node v. Represents a node group Quantity; Then, an attention mechanism is applied to aggregate all groups of types to generate a representation h for a given node. v , Where δ represents the activation function, using LeakyReLU, u∈R d It is the weight parameter, u T This represents the transpose of u. Let {T} represent the node v, and {T} represent the set of types, α v,j Indicates attention weight; Step 204: Based on the self-supervised information pre-trained subgraph, two pre-training tasks are introduced: mask node modeling task and edge reconstruction task, to achieve node-level and edge-level graph exploration.

2. The heterogeneous information network link prediction method based on contrastive learning mechanism according to claim 1, characterized in that, The masked node modeling task involves sorting nodes according to their ranking, randomly selecting a preset proportion of nodes and replacing them with the [MASK] identifier, and then feeding the sorted nodes into the Transformer encoder. The representation generated by Bi-LSTM serves as the identifier, and the sorting information is used as the position vector. This is then processed by the hidden layers learned by the Transformer encoder. The data is fed into the feedforward layer to predict the target node, mathematically represented as: p v =softmax(W MNM z v ), Among them, z v This is the output of the feedforward layer. `Feedforward()` indicates the output from the feedforward layer, `softmax()` represents the activation function, and `W`... MNM ∈V v ×d is the weight shared with the input node representation matrix for classification, V v d is the number of nodes in the subgraph, d is the dimension of the hidden layer vector, and p is the number of nodes in the subgraph. v This is the predicted distribution of v across all nodes, using one-hot labels during training. and prediction Cross-entropy and loss function L MNM The calculation is as follows: Among them, y i and p i It is y i and p i The i-th component, y i p represents a set of labels i A set representing predicted probabilities; The edge reconstruction task involves sampling positive and negative edges in the subgraph. Positive edges are those that actually exist in the original subgraph, while negative edges do not. Given a merged set N of positive and negative edges... S The score for edge reconstruction is calculated using the inner product between a pair of nodes. It calculates the score, h v Let 'v' be the representation of node v, '·' be the inner product, and 'h' be the value of node v. u The representation of node u uses the binary cross-entropy between predicted and true edges to calculate the loss function L for edge reconstruction. ER : |N S | represents the number of node pairs, BinaryCrossEntropy() represents the binary cross-entropy, e uv Let (u,v) represent the actual scores of nodes u and v, and (u,v) represent the edge connecting nodes u and v.

3. The heterogeneous information network link prediction method based on contrastive learning mechanism according to claim 2, characterized in that, The sampling strategy for sampling the subgraph around a node is a random walk with restart. It iteratively traverses the neighborhood of a given node v and returns to the starting node v with a certain probability. In order to sample nodes with higher importance, the random walk strategy first reaches nodes with higher order. In order to make the graph encoder heterogeneous, the traversal is restricted to sampling all types of nodes.

4. The heterogeneous information network link prediction method based on contrastive learning mechanism according to claim 1, characterized in that, The contrastive learning described herein is used to align text representations and graph representations during training. The learning objective is designed as a contrastive loss function that, given a set of text-subgraph pairs, maximizes the similarity score of matching text-subgraph pairs while minimizing the score of non-matching text-subgraph pairs.

5. The heterogeneous information network link prediction method based on a contrastive learning mechanism according to claim 4, characterized in that, In the contrastive learning process, given a node v, the node representation learned by the graph encoder is H, and the weight vector generated by the text encoder is denoted as H. Where K represents the number of categories, and each weight w i All learning comes from prompts, and the predicted probability is calculated as follows: Where τ is the learned temperature hyperparameter, and <·,·> represent the similarity score. <w i H> represents the text weight vector w i The similarity score between nodes represents the vector H.

6. The method for predicting links in heterogeneous information networks based on a contrastive learning mechanism according to claim 2, characterized in that, The introduction of automatically generated, learnable, and continuous cue vectors in step 4 involves replacing discrete text words with continuous vectors learned end-to-end from the data. The cue P input to the text encoder is designed as follows: P=[V1][V2]...[V M ][CLASS], Where [CLASS] represents the node's category label, [V M [] is a word vector with the same dimension as the word representation in the training phase. M is a hyperparameter representing the number of consecutive text vectors in the prompt. After inputting the consecutive prompts P into the text encoder Text(·), the classification weight vectors representing the node concepts can be obtained. The predicted probability is calculated as follows: Among them, each prompt P i The category markers in Text(P) are replaced by the word vector representation of the i-th category name. i ) indicates that a prompt will be made for P. i The vector obtained after being fed into the text encoder.

7. The heterogeneous information network link prediction method based on contrastive learning mechanism according to claim 6, characterized in that, In step 4, to obtain a more accurate cue vector, a residual connection based on the text encoder and the graph encoder is used to utilize the context subgraph of a given node. The text representation of the category label and the node representation in the subgraph are input into the text-subgraph self-attention layer to help the text features find the most relevant context node for a given node. Obtain the output D of the text-subgraph comparator e Then, the text features are updated using residual connections. Text(P)←Text(P)+λD e Here, λ is a learnable parameter used to control the degree of residual connectivity.

8. The heterogeneous information network link prediction method based on contrastive learning mechanism according to claim 1, characterized in that, The text encoder uses the Sentence-BERT model to generate fixed-size text representations.

9. The heterogeneous information network link prediction method based on contrastive learning mechanism according to claim 8, characterized in that, Initialize λ to 10 -4 .

Images