Inductive link prediction method based on relational message passing and interaction information maximization

Abstract

The present application relates to the technical field of knowledge graph completion, and more particularly to a method for inductive knowledge graph completion based on mutual information maximization and relation message passing. The steps are as follows: K-hop closed entity subgraphs and one-hop non-closed entity subgraphs are extracted around the target triple, and then and are converted into closed relation subgraphs and non-closed relation subgraphs respectively; an ontology graph is constructed, the initial nodes of the ontology-enhanced are generated according to the training results of the ontology graph, and then the target relation in the closed relation subgraph is calculated through relation message passing, the target relation in the non-closed relation subgraph is calculated through one-hop neighbor aggregation, and finally the complete subgraph embedding is combined; the supervised learning loss is calculated according to the complete subgraph embedding, the contrastive learning loss is calculated by maximizing the mutual information, and the total loss is obtained to perform joint training. The more optimized local subgraph method studied by the present application can better solve the semantic problems in real inductive knowledge graph reasoning.

Description

Technical Field

[0001] This invention relates to the field of knowledge graph completion technology, and in particular to an inductive link prediction method based on relational message passing and maximizing interactive information. Background Technology

[0002] Knowledge graphs (KGs) organize real-world human knowledge in the form of fact triples (head entity, relation, tail entity), where entities are represented as nodes and relations as edges. In recent years, many large knowledge bases have emerged and are widely used in fields such as natural language processing, intelligent question answering, and recommender systems. However, due to the exponential growth of real-world data, KGs often suffer from noise and incompleteness, i.e., missing some links. Despite numerous successful studies in link prediction, it remains challenging in real-world scenarios. Real-world KGs are dynamically evolving, and traditional inductive link prediction relies on triples involving entities and relations observed during training. Therefore, when new entities and relations (i.e., invisible entities and relations) appear, the KG needs to be retrained, adding extra cost.

[0003] In recent years, researchers have increasingly focused on inductive link prediction, aiming to predict triples containing invisible entities and relations without retraining the knowledge graph. The main challenge in this field lies in effectively handling different types of invisible elements: in semi-inductive settings, invisible entities need to be connected to visible entities in the original knowledge graph; in fully inductive settings, it is necessary to predict invisible-invisible entity combinations in emerging knowledge graphs; and in truly inductive settings, the modeling and reasoning of invisible relations must be further addressed. Therefore, this invention provides an inductive link prediction method based on relational message passing and maximizing interaction information to solve the above problems. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention develops an inductive link prediction method based on relational message passing and maximizing interactive information. The main objective is to study a more optimized local subgraph method to better solve semantic problems in true inductive knowledge graph reasoning.

[0005] The technical solution to the problem solved by this invention is an inductive link prediction method based on relational message passing and maximizing interaction information, as detailed below:

[0006] S1. Subgraph Extraction and Transformation: Extracting K-jump closed entity subgraphs around the target triplet. and a one-hop non-closed entity subgraph Then and Convert each into a closed relational subgraph and non-closed relational subgraph ;

[0007] S2, Subgraph Message Passing: Construct the ontology graph, generate initial nodes for ontology enhancement based on the training results of the ontology graph, and then compute the closed relation subgraph through relation message passing. The target relationship in the graph is calculated by aggregating one-hop neighbors to compute the non-closed relation subgraph. The target relationships in the graph are then combined to form a complete subgraph embedding. ;

[0008] S3. Joint Training: Calculate supervised learning loss based on the complete subgraph embedding. The contrastive learning loss is calculated by maximizing the interaction information. The total losses are used for joint training.

[0009] S1 is as follows:

[0010] S1.1 Subgraph Extraction:

[0011] Around the target triplet Extract K-hop closed entity subgraphs respectively and a one-hop non-closed entity subgraph ,Will and Combined with joint inductive reasoning, in which, Indicates the head entity. Indicates the tail entity. Represents head entity Tail-end entity Relationships between entities and Represented as nodes, relationships It is represented as an edge;

[0012] S1.1.1 Extracting K-jump Closed Entity Subgraph :

[0013] Based on the head entity in the target triplet calculate -Leap-in neighbor According to the tail entity in the target triple Calculate out-degree neighbors ;

[0014] Find the head entity of -Leap-in neighbor Tail-end entity of -Breakout degree neighbor The intersection between them yields common neighbors. ;

[0015] Remove common neighbors Isolated nodes and nodes at distance from the target triple in the data. and Exceed Jumping nodes, get - Jump to closed entity subgraph , All nodes in the array are at maximum distance from the target triple node. Within the range;

[0016] in, - Jump to closed entity subgraph The time complexity is , Indicate the time complexity. Index representing the hop count, , , and They represent - Jump to closed entity subgraph The number of nodes, relations, and edges in the middle, and the embedding vector dimensions of nodes and relations are all [missing information]. ;

[0017] S1.1.2 Extracting a one-hop non-closed entity subgraph :

[0018] Based on the head entity in the target triplet calculate - Jump to the neighbor According to the tail entity in the target triple calculate - Jump to the neighbor ;

[0019] Find the head entity of - Jump to the neighbor Tail-end entity of - Jump to the neighbor The union of the nodes yields all neighboring nodes. ;

[0020] The head entity in the target triple Tail entity All neighboring nodes The one-hop non-closed entity subgraph is formed by the edges between the entities in the target triple and the entities corresponding to their neighboring nodes. Subgraph.

[0021] S1.2 Subgraph Transformation:

[0022] Subgraph transformation will extract the K-hop closed entity subgraph. and a one-hop non-closed entity subgraph Transform into a closed relational subgraph and non-closed relational subgraph Closed entity subgraph and unclosed entity subgraphs Using relationships as nodes, edges are added based on connection methods, and topological patterns are labeled. Through traversal and analysis, it is ensured that all relationship nodes and their connection methods are included. Finally, a complete closed relation subgraph is constructed with relationships as nodes and topological patterns as edges. and non-closed relational subgraph ;

[0023] The specific operation process is as follows:

[0024] From K-jump closed entity subgraph and a one-hop non-closed entity subgraph Each entity selects a relationship between itself to form a relationship pair. A relationship pair represents the association between two entities through one or more relationships and is used to describe the directionality and topological pattern of the relationship.

[0025] The relation pairs include six different topological patterns that capture the correlation between arbitrary relations, specifically head-to-tail, tail-to-tail, head-to-head, tail-to-head, parallel, and loop, with corresponding parameter representations of HT, TT, HH, TH, PARA, and LOOP, respectively.

[0026] Arbitrary relation pairs and The representation between them is or or or or or , Representing relation pairs and Connect in a head-to-tail manner, Representing relation pairs and Connected tail-to-tail. Representing relation pairs and Connect head-to-head. Representing relation pairs and Connected in a tail-to-head manner, Representing relation pairs and Connect in parallel. Representing relation pairs and Connected in a loop.

[0027] S2 is as follows:

[0028] S2.1 Construct the ontology graph and generate the initial nodes for ontology enhancement:

[0029] S2.1.1 Constructing the ontology graph:

[0030] (1) Determine the semantic vocabulary. Capture the semantic relationship of the relationship based on the semantic vocabulary. Prioritize the selection of vocabulary that supports reasoning and modeling the semantic relationship. Key terms of the semantic vocabulary include “rdfs:subPropertyOf” for describing the hierarchical relationship between attributes, “rdfs:subClassOf” for defining the hierarchical relationship between classes, “rdfs:domain” for specifying the head entity type of the relationship, and “rdfs:range” for specifying the tail entity type of the relationship.

[0031] (2) Exclude irrelevant terms. Irrelevant terms refer to terms that are not used to express semantic relationships. Irrelevant terms include basic semantic structures used to define sets, sequences and instance types.

[0032] (3) Extract relation information from the NELL-995 dataset and annotate the semantic attributes of each relation. and This represents any two relation information extracted from the dataset. One key term is selected from four key terms to define the two relations, resulting in an RDF triple, represented as: An ontology graph is constructed based on RDF triples composed of relational information and key terms extracted from the dataset. Each RDF triple represents a resource description framework triple, consisting of a subject, predicate, and object, with the subject corresponding to the relational information. Predicate corresponds to key terms, and object corresponds to key terms. ;

[0033] Among them, the NELL-995 dataset is a subset of knowledge graphs automatically extracted from the network for constructing semantic ontology and supporting knowledge reasoning and relation extraction, containing 995 categories and their relations;

[0034] (4) The reasoning tool OWL Reasoner performs semantic reasoning and logical verification on the ontology graph. OWL Reasoner is a tool that ensures the consistency of the structure and further optimizes the ontology graph.

[0035] S2.1.2, Generate the initial node for ontology enhancement:

[0036] Using the ontology graph as prior knowledge for initial nodes, the ontology graph is pre-trained using the knowledge graph embedding technique TransE to obtain representations of relations. Then, the relationships are embedded and projected onto the relationship subgraph through a two-layer fully connected network to generate the initial nodes for ontology enhancement. The calculation process is as follows:

[0037] ,

[0038] in, Indicates the initial node for ontology enhancement, index Indicates the initialization state; This represents the weight matrix of the first fully connected layer. Let represent the weight matrix of the second fully connected layer. This indicates that the ontology graph was obtained through pre-training. The node representation.

[0039] S2.2, Relationship Message Passing:

[0040] Calculating closed relation subgraphs using relational message passing The target relationship nodes are aggregated using a two-step iterative process. To switch neighbors, follow these steps:

[0041] (1) Take the target relation as the root node, select the one-hop neighbor of the target relation node, and then expand it layer by layer to the Kth layer, and use the relation-based graph pruning strategy for optimization;

[0042] The specific process is as follows: starting from the target relation node, gradually expand the neighborhood of the target relation node to... The layer is calculated, taking into account the direction of information transmission, and all in-degree neighbors of each node are sampled to finally obtain the target relation node. of Jump-in neighbor , Specifically, it is expressed as follows:

[0043] ,

[0044] ,

[0045] in, Target Relationship Node The Jump-in degree neighbor set and , This indicates the target relationship node that the current layer is processing. express Neighbor relationship nodes, express and The types of adjacent edges between them This represents the set of all edges in the relational subgraph. Represents a closed relation subgraph. Indicates the depth of message passing. ;

[0046] (2) Starting from the final layer, the message passing mechanism iteratively aggregates the information of neighboring nodes layer by layer to generate the final target relation node embedding;

[0047] Initially adopted The relational graph convolutional network framework assigns transformation matrices to different edge types, aggregates neighbor and self features layer by layer, and generates node representations containing relational semantics;

[0048] When operating on the data in the NELL-995 dataset, the initial nodes for ontology enhancement are generated using the method in step S2.1.2. Otherwise, a random initialization method is used to randomly sample the representations of initial nodes from a uniform distribution to provide preliminary random feature representations for the training process;

[0049] Message passing from In the beginning, The representations of neighboring nodes are aggregated within the jump, and the node embeddings are updated layer by layer to finally obtain the first node. Complete embedding of layer target relationship nodes;

[0050] Introducing a target-relationship-aware attention mechanism to capture neighboring nodes highly correlated with the target relationship, formally... The aggregation function of the GNN layer is defined as follows:

[0051] ,

[0052] in, Represents a node The directed neighborhood, Represents a non-linear activation function. Indicates the first Layer nodes The aggregation has a directed neighborhood embedding. Represents a node In edge type The following is a collection of neighbors. Indicates corresponding to the first Layer edge type The transformation matrix, Indicates the first Neighbors obtained through layer-by-layer iterative learning The expression, Indicates the first Layer in edge type Lower Relationship Node For target relation nodes The importance of express -1 level target relationship nodes Embedding;

[0053] Finally, the first Layer relationship nodes With the Neighborhood vectors of layer aggregation To update the first by splicing Potential layer nodes , It is expressed as follows:

[0054] .

[0055] S2.3, One-hop neighbor aggregation:

[0056] First, for the target relation nodes We sample one-hop neighbors and aggregate their features using an attention mechanism. The calculation process is as follows:

[0057] ,

[0058] in, Indicates target relationship A set of neighbors that jumps over time. Indicating neighborly relations Embedded, This represents the linear transformation matrix for each relation node. Represents a non-linear activation function, attention weights Used to measure neighbor relationships among nodes For target relation nodes The importance of;

[0059] S2.4, Complete Subgraph Embedding:

[0060] Features learned from non-closed relation subgraphs through one-hop neighbor aggregation Features learned through relational message passing in closed relational subgraphs The data is integrated using summation and concatenation to form the final representation of the relational subgraph. , means as follows:

[0061]

[0062] in, This represents the summation operation. This indicates a splicing operation.

[0063] S3 is as follows:

[0064] S3.1 Supervised Learning:

[0065] Based on the final representation of the relational subgraph Calculate the target triplet Likelihood score The specific calculations are as follows:

[0066] ,

[0067] in, Represents the relation weight matrix;

[0068] Then, through supervised learning loss... Optimize the likelihood score of the triples and replace the triples. The head or tail entity in the sample generates an equal number of negative and positive triplets, ensuring that the score of the positive triplet is significantly higher than that of the negative triplet. The calculation process is as follows:

[0069]

[0070] in, This indicates the number of all target triples in the input. and They represent the target triples respectively. After substitution, we obtain negative triples and positive triples. This represents the likelihood score for a negative triple. This represents the likelihood score for a positive triple. Indicates the boundary hyperparameters, This indicates taking the maximum value. Indicates the index of the triple. , ;

[0071] S3.2 Maximizing Interactive Information:

[0072] By maximizing the local and global interaction information (MI) to construct a global topological pattern between relationships, covering subgraphs and layer levels, the global representation of the graph is first obtained using a read function, specifically calculated as follows:

[0073] ,

[0074] in, Indicates the first Embedding of local relation subgraphs of positive sample triples This indicates the embedding of the entire relationship diagram. This represents the number of all positive sample triples in the input;

[0075] Then, a bilinear score function is introduced as a discriminator, and the Jensen-Shannon MI estimator is applied to estimate the interaction information between the relational subgraph embedding and the entire relational graph embedding, thereby maximizing their mutual information and determining whether they originate from the same graph. The Jensen-Shannon MI estimator is a method for estimating the mutual information (MI) between two random variables. The bilinear score function is as follows:

[0076] ,

[0077] in, This represents the activation function. The score matrix representing mutual information (MI) This indicates the similarity between the subgraph embedding and the entire graph embedding.

[0078] Through the corruption function Constructing a negative sampling relationship graph The specific calculations are as follows:

[0079] ,

[0080] in, Represents a closed relation graph. This represents the initial characteristics of the relation nodes in a closed relation graph. Indicates through the Initial features of damaged nodes obtained by randomly shuffling layers one by one. express The adjacency matrix;

[0081] Finally, through and Calculate the contrastive learning loss between positive and negative samples This enables the learning of graph embeddings, and the specific calculation is as follows:

[0082] ,

[0083] in, This indicates the number of negative triplet pairs. This indicates the number of positive triplet pairs. The embedding representation of the negative sample relation subgraph. Indicates the first The similarity between the negative sample relation subgraph embedding and the entire relation graph embedding;

[0084] S3.3 Joint Training Objectives:

[0085] The final loss is obtained by minimizing through joint training. Includes supervised learning loss and contrastive learning loss The calculation process is as follows:

[0086] ,

[0087] in, This represents the hyperparameter used to adjust the ratio of supervised learning loss to contrastive learning loss.

[0088] The effects described in the invention are merely those of the embodiments, and not all the effects of the invention. The above technical solutions have the following advantages or beneficial effects:

[0089] This invention proposes a novel model based on Relational Message Passing Network (RPMI). RPMI extracts closed entity subgraphs and one-hop open entity subgraphs around the target triples, merges topological patterns between relations, and transforms the entity graph into closed and open relation subgraphs. Furthermore, it enhances node features by injecting ontological patterns from the knowledge graph and completes reasoning by combining relational message passing and one-hop neighbor aggregation. In addition, to strengthen the global modeling of topological patterns between relations, RPMI maximizes the interaction information between subgraphs and the overall graph. Therefore, this invention performs excellently in fully inductive link prediction tasks, maintains high prediction accuracy in invisible entity scenarios, and achieves significant breakthroughs in true inductive link prediction tasks. It provides strong support for joint reasoning of relations, opens up new solutions and methods for reasoning problems in complex knowledge graph scenarios, and demonstrates outstanding performance and broad application prospects. Attached Figure Description

[0090] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used together with the embodiments of the invention to explain the invention and do not constitute a limitation thereof.

[0091] Figure 1 This is a structural diagram of the method of the present invention.

[0092] Figure 2 This is the result of the ablation experiment. Detailed Implementation

[0093] To clearly illustrate the technical features of this solution, the invention will be described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0094] Example 1

[0095] like Figure 1 As shown, an inductive link prediction method based on relational message passing and maximizing interaction information is presented below:

[0096] S1. Subgraph Extraction and Transformation: Extracting K-jump closed entity subgraphs around the target triplet. and a one-hop non-closed entity subgraph Then and Convert each into a closed relational subgraph and non-closed relational subgraph ;

[0097] S2, Subgraph Message Passing: Construct the ontology graph, generate initial nodes for ontology enhancement based on the training results of the ontology graph, and then compute the closed relation subgraph through relation message passing. The target relationship in the graph is calculated by aggregating one-hop neighbors to compute the non-closed relation subgraph. The target relationships in the graph are then combined to form a complete subgraph embedding. ;

[0098] S3. Joint Training: Calculate supervised learning loss based on the complete subgraph embedding. The contrastive learning loss is calculated by maximizing the interaction information. The total losses are used for joint training.

[0099] S1 is as follows:

[0100] S1.1 Subgraph Extraction:

[0101] Around the target triplet Extract K-hop closed entity subgraphs respectively and a one-hop non-closed entity subgraph ,Will and Combined with joint inductive reasoning, in which, Indicates the head entity. Indicates the tail entity. Represents head entity Tail-end entity Relationships between entities and Represented as nodes, relationships It is represented as an edge;

[0102] S1.1.1 Extracting K-jump Closed Entity Subgraph :

[0103] Based on the head entity in the target triplet calculate -Leap-in neighbor According to the tail entity in the target triple Calculate out-degree neighbors ;

[0104] Find the head entity of -Leap-in neighbor Tail-end entity of -Breakout degree neighbor The intersection between them yields common neighbors. ;

[0105] Remove common neighbors Isolated nodes and nodes at distance from the target triple in the data. and Exceed Jumping nodes, get - Jump to closed entity subgraph , All nodes in the array are at maximum distance from the target triple node. Within the range;

[0106] in, - Jump to closed entity subgraph The time complexity is , Indicate the time complexity. Index representing the hop count, , , and They represent - Jump to closed entity subgraph The number of nodes, relations, and edges in the middle, and the embedding vector dimensions of nodes and relations are all [missing information]. ;

[0107] S1.1.2 Extracting a one-hop non-closed entity subgraph :

[0108] Based on the head entity in the target triplet calculate - Jump to the neighbor According to the tail entity in the target triple calculate - Jump to the neighbor ;

[0109] Find the head entity of - Jump to the neighbor Tail-end entity of - Jump to the neighbor The union of the nodes yields all neighboring nodes. ;

[0110] The head entity in the target triple Tail entity All neighboring nodes The one-hop non-closed entity subgraph is formed by the edges between the entities in the target triple and the entities corresponding to their neighboring nodes. Subgraph.

[0111] S1.2 Subgraph Transformation:

[0112] Subgraph transformation will extract the K-hop closed entity subgraph. and a one-hop non-closed entity subgraph Transform into a closed relational subgraph and non-closed relational subgraph Closed entity subgraph and unclosed entity subgraphs Using relationships as nodes, edges are added based on connection methods, and topological patterns are labeled. Through traversal and analysis, it is ensured that all relationship nodes and their connection methods are included. Finally, a complete closed relation subgraph is constructed with relationships as nodes and topological patterns as edges. and non-closed relational subgraph ;

[0113] The specific operation process is as follows:

[0114] From K-jump closed entity subgraph and a one-hop non-closed entity subgraph Each entity selects a relationship between itself to form a relationship pair. A relationship pair represents the association between two entities through one or more relationships and is used to describe the directionality and topological pattern of the relationship.

[0115] The relation pairs include six different topological patterns that capture the correlation between arbitrary relations, specifically head-to-tail, tail-to-tail, head-to-head, tail-to-head, parallel, and loop, with corresponding parameter representations of HT, TT, HH, TH, PARA, and LOOP, respectively.

[0116] Arbitrary relation pairs and The representation between them is or or or or or , Representing relation pairs and Connect in a head-to-tail manner, Representing relation pairs and Connected tail-to-tail. Representing relation pairs and Connect head-to-head. Representing relation pairs and Connected in a tail-to-head manner, Representing relation pairs and Connect in parallel. Representing relation pairs and Connected in a loop.

[0117] S2 is as follows:

[0118] S2.1 Construct the ontology graph and generate the initial nodes for ontology enhancement:

[0119] S2.1.1 Constructing the ontology graph:

[0120] (1) Determine the semantic vocabulary. Capture the semantic relationship of the relationship based on the semantic vocabulary. Prioritize the selection of vocabulary that supports reasoning and modeling the semantic relationship. Key terms of the semantic vocabulary include “rdfs:subPropertyOf” for describing the hierarchical relationship between attributes, “rdfs:subClassOf” for defining the hierarchical relationship between classes, “rdfs:domain” for specifying the head entity type of the relationship, and “rdfs:range” for specifying the tail entity type of the relationship.

[0121] (2) Exclude irrelevant terms. Irrelevant terms refer to terms that are not used to express semantic relationships. Irrelevant terms include basic semantic structures used to define sets, sequences and instance types.

[0122] (3) Extract relation information from the NELL-995 dataset and annotate the semantic attributes of each relation. and This represents any two relation information extracted from the dataset. One key term is selected from four key terms to define the two relations, resulting in an RDF triple, represented as: An ontology graph is constructed based on RDF triples composed of relational information and key terms extracted from the dataset. Each RDF triple represents a resource description framework triple, consisting of a subject, predicate, and object, with the subject corresponding to the relational information. Predicate corresponds to key terms, and object corresponds to key terms. ;

[0123] Among them, the NELL-995 dataset is a subset of knowledge graphs automatically extracted from the network for constructing semantic ontology and supporting knowledge reasoning and relation extraction, containing 995 categories and their relations;

[0124] (4) The reasoning tool OWL Reasoner performs semantic reasoning and logical verification on the ontology graph. OWL Reasoner is a tool that ensures the consistency of the structure and further optimizes the ontology graph.

[0125] S2.1.2, Generate the initial node for ontology enhancement:

[0126] Using the ontology graph as prior knowledge for initial nodes, the ontology graph is pre-trained using the knowledge graph embedding technique TransE to obtain representations of relations. Then, the relationships are embedded and projected onto the relationship subgraph through a two-layer fully connected network to generate the initial nodes for ontology enhancement. The calculation process is as follows:

[0127] ,

[0128] in, Indicates the initial node for ontology enhancement, index Indicates the initialization state; This represents the weight matrix of the first fully connected layer. Let represent the weight matrix of the second fully connected layer. This indicates that the ontology graph was obtained through pre-training. The node representation.

[0129] S2.2, Relationship Message Passing:

[0130] Calculating closed relation subgraphs using relational message passing The target relationship nodes are aggregated using a two-step iterative process. To switch neighbors, follow these steps:

[0131] (1) Take the target relation as the root node, select the one-hop neighbor of the target relation node, and then expand it layer by layer to the Kth layer, and use the relation-based graph pruning strategy for optimization;

[0132] The specific process is as follows: starting from the target relation node, gradually expand the neighborhood of the target relation node to... The layer is calculated, taking into account the direction of information transmission, and all in-degree neighbors of each node are sampled to finally obtain the target relation node. of Jump-in neighbor , Specifically, it is expressed as follows:

[0133] ,

[0134] ,

[0135] in, Target Relationship Node The Jump-in degree neighbor set and , This indicates the target relationship node that the current layer is processing. express Neighbor relationship nodes, express and The types of adjacent edges between them This represents the set of all edges in the relational subgraph. Represents a closed relation subgraph. Indicates the depth of message passing. ;

[0136] (2) Starting from the final layer, the message passing mechanism iteratively aggregates the information of neighboring nodes layer by layer to generate the final target relation node embedding;

[0137] Initially adopted The relational graph convolutional network framework assigns transformation matrices to different edge types, aggregates neighbor and self features layer by layer, and generates node representations containing relational semantics;

[0138] When operating on the data in the NELL-995 dataset, the initial nodes for ontology enhancement are generated using the method in step S2.1.2. Otherwise, a random initialization method is used to randomly sample the representations of initial nodes from a uniform distribution to provide preliminary random feature representations for the training process;

[0139] Message passing from In the beginning, The representations of neighboring nodes are aggregated within the jump, and the node embeddings are updated layer by layer to finally obtain the first node. Complete embedding of layer target relationship nodes;

[0140] Introducing a target-relationship-aware attention mechanism to capture neighboring nodes highly correlated with the target relationship, formally... The aggregation function of the GNN layer is defined as follows:

[0141] ,

[0142] in, Represents a node The directed neighborhood, Represents a non-linear activation function. Indicates the first Layer nodes The aggregation has a directed neighborhood embedding. Represents a node In edge type The following is a collection of neighbors. Indicates corresponding to the first Layer edge type The transformation matrix, Indicates the first Neighbors obtained through layer-by-layer iterative learning The expression, Indicates the first Layer in edge type Lower Relationship Node For target relation nodes The importance of express -1 level target relationship nodes Embedding;

[0143] Finally, the first Layer relationship nodes With the Neighborhood vectors of layer aggregation To update the first by splicing Potential layer nodes , It is expressed as follows:

[0144] .

[0145] S2.3, One-hop neighbor aggregation:

[0146] First, for the target relation nodes We sample one-hop neighbors and aggregate their features using an attention mechanism. The calculation process is as follows:

[0147] ,

[0148] in, Indicates target relationship A set of neighbors that jumps over time. Indicating neighborly relations Embedded, This represents the linear transformation matrix for each relation node. Represents a non-linear activation function, attention weights Used to measure neighbor relationships among nodes For target relation nodes The importance of;

[0149] S2.4, Complete Subgraph Embedding:

[0150] Features learned from non-closed relation subgraphs through one-hop neighbor aggregation Features learned through relational message passing in closed relational subgraphs The data is integrated using summation and concatenation to form the final representation of the relational subgraph. , means as follows:

[0151]

[0152] in, This represents the summation operation. This indicates a splicing operation.

[0153] S3 is as follows:

[0154] S3.1 Supervised Learning:

[0155] Based on the final representation of the relational subgraph Calculate the target triplet Likelihood score The specific calculations are as follows:

[0156] ,

[0157] in, Represents the relation weight matrix;

[0158] Then, through supervised learning loss... Optimize the likelihood score of the triples and replace the triples. The head or tail entity in the sample generates an equal number of negative and positive triplets, ensuring that the score of the positive triplet is significantly higher than that of the negative triplet. The calculation process is as follows:

[0159]

[0160] in, This indicates the number of all target triples in the input. and They represent the target triples respectively. After substitution, we obtain negative triples and positive triples. This represents the likelihood score for a negative triple. This represents the likelihood score for a positive triple. Indicates the boundary hyperparameters, This indicates taking the maximum value. Indicates the index of the triple. , ;

[0161] S3.2 Maximizing Interactive Information:

[0162] By maximizing the local and global interaction information (MI) to construct a global topological pattern between relationships, covering subgraphs and layer levels, the global representation of the graph is first obtained using a read function, specifically calculated as follows:

[0163] ,

[0164] in, Indicates the first Embedding of local relation subgraphs of positive sample triples This indicates the embedding of the entire relationship diagram. This represents the number of all positive sample triples in the input;

[0165] Then, a bilinear score function is introduced as a discriminator, and the Jensen-Shannon MI estimator is applied to estimate the interaction information between the relational subgraph embedding and the entire relational graph embedding, thereby maximizing their mutual information and determining whether they originate from the same graph. The Jensen-Shannon MI estimator is a method for estimating the mutual information (MI) between two random variables. The bilinear score function is as follows:

[0166] ,

[0167] in, This represents the activation function. The score matrix representing mutual information (MI) This indicates the similarity between the subgraph embedding and the entire graph embedding.

[0168] Through the corruption function Constructing a negative sampling relationship graph The specific calculations are as follows:

[0169] ,

[0170] in, Represents a closed relation graph. This represents the initial characteristics of the relation nodes in a closed relation graph. Indicates through the Initial features of damaged nodes obtained by randomly shuffling layers one by one. express The adjacency matrix;

[0171] Finally, through and Calculate the contrastive learning loss between positive and negative samples This enables the learning of graph embeddings, and the specific calculation is as follows:

[0172] ,

[0173] in, This indicates the number of negative triplet pairs. This indicates the number of positive triplet pairs. The embedding representation of the negative sample relation subgraph. Indicates the first The similarity between the negative sample relation subgraph embedding and the entire relation graph embedding;

[0174] S3.3 Joint Training Objectives:

[0175] The final loss is obtained by minimizing through joint training. Includes supervised learning loss and contrastive learning loss The calculation process is as follows:

[0176] ,

[0177] in, This represents the hyperparameter used to adjust the ratio of supervised learning loss to contrastive learning loss.

[0178] Example 2

[0179] This invention uses two sets of datasets to evaluate the proposed method. The first set consists of datasets WN18RR.v1 / v2 and FB15k-237.v1 / v2, which are derived from the datasets proposed by the GraIL model. This dataset is the first dataset designed specifically for fully inductive link prediction scenarios. The characteristic of this dataset is that entities in the test set do not appear in the training set, but the relationships already exist in the test set. It is mainly used to evaluate the model's inductive reasoning ability for unseen entities, as shown in Table 1.

[0180] Table 1. Statistics of datasets for total inductive link prediction of invisible-invisible entities.

[0181]

[0182] The second set of datasets, NELL-995.v1.v3 and NELL-995.v2.v3, are datasets proposed by the RMPI model. Each dataset contains two versions: one containing semi-invisible relations and the other containing fully invisible relations. This dataset is designed for true inductive link prediction scenarios. The test set contains unseen entities and unseen relations to test the model's comprehensive inductive ability in scenarios with unseen entities and unseen relations. The dataset is shown in Table 2.

[0183] Table 2 shows the statistics of datasets used for true inductive link prediction of invisible-invisible entities and invisible relationships.

[0184]

[0185] To ensure fair comparison, triplet classification and entity prediction tasks were used for evaluation. Triplet classification used the AUC-PR metric, which represents the area under the precision-recall curve and measures the classification model's ability to distinguish between positive and negative samples; a higher value indicates better performance. Entity prediction used the MRR and Hits@N metrics, where MRR represents the average reciprocal rank of the correct answer in the candidate list, and Hits@N represents the proportion of correct answers appearing in the top N candidate positions; higher values ​​indicate better performance. To predict missing head or tail entities, 50 negative samples were randomly sampled for each positive triplet. Higher AUC-PR, MRR, and Hits@N indicate better link prediction performance. Five experiments were conducted in this invention, and the average results were taken.

[0186] This invention implements the model using PyTorch and employs the Adam optimizer. Each triple samples a 2-hop closed subgraph and a 1-hop open subgraph, and applies two layers of message passing on the closed subgraph. Hyperparameters such as the learning rate are manually adjusted during training. In {0.1, 0.01, 0.001, 0.0001}, the feature dimension is... In {16, 32, 64, 128}, the edge drop rate is... In {0.1, 0.3, 0.5, 0.8}, the mutual information loss coefficient In {1, 3, 5, 7}, the optimal LP configuration is summarized as follows: 0.001, 32, =0.5, =1;

[0187] To evaluate RPMI's performance in fully inductive LP, this invention was compared with five benchmark models: GraIL, TACT-Base, TACT, CoMPILE, and RMPI. These models originate from inductive learning scenarios and drive performance optimization for relation prediction models. The GraIL model is the first to propose a method that eliminates global graph dependence through subgraph extraction, node labeling, and GNN scoring, making it suitable for inductive learning. The TACT model, based on GraIL, introduces relation topological relevance and combines relation and graph structure modules to enhance graph structure reasoning capabilities. The TACT-Base model, a simplified version of TACT, uses only the relation module, providing a baseline for comparison. The CoMPILE model extends GraIL by introducing a node-edge communication mechanism, enhancing feature learning capabilities through node-edge information interaction. The RMPI model proposes an innovative relation message passing network and designs the RMPI-NE mechanism, specifically designed for true inductive LP, suitable for handling unseen entities and relations.

[0188] Because these two techniques have limited applicability across all datasets, this invention proposes several RPMI variants. First, there are two initialization methods for unseen relations in the test graph: i) RMPI: random initialization, ii) RMPI+: pattern-enhanced initialization. Second, the target relation-aware neighborhood attention (TA) mechanism used in one-hop neighborhood aggregation of open subgraphs is not applicable to true inductive settings that simultaneously contain unseen entities and unseen relations, but only to fully inductive settings that contain only unseen entities. Therefore, this invention tests several variants: RPMI+ (without TA) for true inductive settings, and RPMI-TA (with TA) for fully inductive settings.

[0189] For the true inductive setting, semantic vectors learned from ontology patterns are used to initialize the representation of unseen relations in the test graph (limited to the NELL-related dataset). In addition, two different fusion functions based on summation (SUM) and concatenation (CONC) are used, and the results are reported. The results are shown in Table 3, with the best score shown in bold. As can be seen from the experimental results of ontology-enhanced unseen relations in Table 3, the relation association module of TACT-base only aggregates one layer of neighborhood of the target relation, resulting in poor performance. RMPI, on the other hand, outperforms TACT-base through direct iterative message passing between relations. The proposed RPMI+ further models the topological patterns between relations by maximizing interaction information (MI) globally, outperforming all other state-of-the-art models in both semi-unseen and fully unseen relation scenarios, except for the semi-unseen relation MRR of NELL-995.v2.v3.

[0190] Table 3 Results on True Inductive Link Prediction

[0191]

[0192] For the fully inductive setting, two different fusion functions based on summation (SUM) and concatenation (CONC) were used. The performance of the model in fully inductive LP was evaluated using only the MRR and Hits@10 metrics. The results are shown in Table 4, with the best scores shown in bold. It can be seen that compared with the baseline model that only uses local subgraphs for inference, the proposed RPMI-TA has the ability to fuse local (subgraph) and global (graph) information, and both metrics are significantly better than the baseline model on most datasets. However, the MRR on the WN18RR.v2 dataset and the Hits@10 on the FB15k-237.v1 dataset are slightly lower, which may be because subgraph inference alone provides sufficient information, and adding MI maximization to learn global structural information has a negative impact.

[0193] Table 4 shows the results for predicting total inductive linkage.

[0194]

[0195] This invention selected the NELL-995.v1.v3 fully invisible relation and the WN18RR.v1 dataset, and conducted ablation experiments on RPMI+(CONC) and RPMI-TA(CONC) respectively. Each component was removed one by one to analyze its effect, as shown in Figure 2. The experimental results for Hits@10 are reported. These components include: 1) ontology pattern injection of the knowledge graph (denoted as w / o OS), 2) one-hop neighborhood aggregation (denoted as w / o NE), 3) mutual information maximization (denoted as w / o MI), and 4) a target relation-aware neighborhood attention mechanism (denoted as w / o TA). Figure 2 As can be seen, each component has a significant impact on model performance. Removing key components such as ontology pattern injection or one-hop neighborhood aggregation will significantly reduce Hits@10, indicating that these components are crucial in improving link prediction performance.

[0196] Although the specific embodiments of the invention have been described above in conjunction with the accompanying drawings, this is not intended to limit the scope of protection of the invention. Based on the technical solutions of the invention, various modifications or variations that can be made by those skilled in the art without creative effort are still within the scope of protection of the invention.

Claims

1. An inductive link prediction method based on relational message passing and maximizing interaction information, characterized in that, Includes the following steps: S1. Subgraph Extraction and Transformation: Extracting K-jump closed entity subgraphs around the target triplet. and a one-hop non-closed entity subgraph Then and Convert each into a closed relational subgraph and non-closed relational subgraph ; S1 is as follows: S1.1 Subgraph Extraction: Around the target triplet Extract K-hop closed entity subgraphs respectively and a one-hop non-closed entity subgraph ,Will and Combined with joint inductive reasoning, in which, Indicates the head entity. Indicates the tail entity. Represents head entity Tail-end entity Relationships between entities and Represented as nodes, relationships It is represented as an edge; S1.1.1 Extracting K-jump Closed Entity Subgraph : Based on the head entity in the target triplet calculate -Leap-in neighbor According to the tail entity in the target triple Calculate out-degree neighbors ; Find the head entity of -Leap-in neighbor Tail-end entity of -Breakout degree neighbor The intersection between them yields common neighbors. ; Remove common neighbors Isolated nodes and nodes at distance from the target triple in the data. and Exceed Jumping nodes, get - Jump to closed entity subgraph , All nodes in the array are at maximum distance from the target triple node. Within the range; in, - Jump to closed entity subgraph The time complexity is , Indicate the time complexity. Index representing the hop count, , , and They represent - Jump to closed entity subgraph The number of nodes, relations, and edges in the middle, and the embedding vector dimensions of nodes and relations are all [missing information]. ; S1.1.2 Extracting a one-hop non-closed entity subgraph : Based on the head entity in the target triplet calculate - Jump to the neighbor According to the tail entity in the target triple calculate - Jump to the neighbor ; Find the head entity of - Jump to the neighbor Tail-end entity of - Jump to the neighbor The union of the nodes yields all neighboring nodes. ; The head entity in the target triple Tail entity All neighboring nodes The one-hop non-closed entity subgraph is formed by the edges between the entities in the target triple and the entities corresponding to their neighboring nodes. Subgraph; S2, Subgraph Message Passing: Construct the ontology graph, generate initial nodes for ontology enhancement based on the training results of the ontology graph, and then compute the closed relation subgraph through relation message passing. The target relationship in the graph is calculated by aggregating one-hop neighbors to compute the non-closed relation subgraph. The target relationships in the graph are then combined to form a complete subgraph embedding. ; S2 is as follows: S2.1 Construct the ontology graph and generate the initial nodes for ontology enhancement: S2.1.1 Constructing the ontology graph: (1) Determine the semantic vocabulary. Capture the semantic relationship of the relationship based on the semantic vocabulary. Prioritize the selection of vocabulary that supports reasoning and modeling the semantic relationship. Key terms of the semantic vocabulary include "rdfs:subPropertyOf" for describing the hierarchical relationship between attributes, "rdfs:subClassOf" for defining the hierarchical relationship between classes, "rdfs:domain" for specifying the head entity type of the relationship, and "rdfs:range" for specifying the tail entity type of the relationship. (2) Exclude irrelevant terms. Irrelevant terms refer to terms that are not used to express semantic relationships. Irrelevant terms include basic semantic structures used to define sets, sequences and instance types. (3) Extract relation information from the NELL-995 dataset and annotate the semantic attributes of each relation. and This represents any two relation information extracted from the dataset. One key term is selected from four key terms to define the two relations, resulting in an RDF triple, represented as: An ontology graph is constructed based on RDF triples composed of relational information and key terms extracted from the dataset. Each RDF triple represents a resource description framework triple, consisting of a subject, predicate, and object, with the subject corresponding to the relational information. Predicate corresponds to key terms, and object corresponds to key terms. ; Among them, the NELL-995 dataset is a subset of knowledge graphs automatically extracted from the network for constructing semantic ontology and supporting knowledge reasoning and relation extraction, containing 995 categories and their relations; (4) The reasoning tool OWL Reasoner performs semantic reasoning and logical verification on the ontology graph. OWL Reasoner is a tool that ensures the consistency of the structure and further optimizes the ontology graph. S2.1.2, Generate the initial node for ontology enhancement: Using the ontology graph as prior knowledge for initial nodes, the ontology graph is pre-trained using the knowledge graph embedding technique TransE to obtain representations of relations. Then, the relationships are embedded and projected onto the relationship subgraph through a two-layer fully connected network to generate the initial nodes for ontology enhancement. The calculation process is as follows: ， in, Indicates the initial node for ontology enhancement, index Indicates the initialization state; This represents the weight matrix of the first fully connected layer. Let represent the weight matrix of the second fully connected layer. This indicates that the ontology graph was obtained through pre-training. The node representation; S3. Joint Training: Calculate supervised learning loss based on the complete subgraph embedding. The contrastive learning loss is calculated by maximizing the interaction information. To obtain the total loss for joint training; S3 is as follows: S3.1 Supervised Learning: Based on the final representation of the relational subgraph Calculate the target triplet Likelihood score The specific calculations are as follows: ， in, Represents the relation weight matrix; Then, through supervised learning loss... Optimize the likelihood score of the triples and replace the triples. The head or tail entity in the sample generates an equal number of negative and positive triplets, ensuring that the score of the positive triplet is significantly higher than that of the negative triplet. The calculation process is as follows: in, This indicates the number of all target triples in the input. and They represent the target triples respectively. After substitution, we obtain negative triples and positive triples. This represents the likelihood score for a negative triple. This represents the likelihood score for a positive triple. Indicates the boundary hyperparameters, This indicates taking the maximum value. Indicates the index of the triple. , ; S3.2 Maximizing Interactive Information: By maximizing the local and global interaction information (MI) to construct a global topological pattern between relationships, covering subgraphs and layer levels, the global representation of the graph is first obtained using a read function, specifically calculated as follows: ， in, Indicates the first Embedding of local relation subgraphs of positive sample triples This indicates the embedding of the entire relationship diagram. This represents the number of all positive sample triples in the input; Then, a bilinear score function is introduced as a discriminator, and the Jensen-Shannon MI estimator is applied to estimate the interaction information between the relational subgraph embedding and the entire relational graph embedding, thereby maximizing their mutual information and determining whether they originate from the same graph. The Jensen-Shannon MI estimator is a method for estimating the mutual information (MI) between two random variables. The bilinear score function is as follows: ， in, This represents the activation function. The score matrix representing mutual information (MI) This indicates the similarity between the subgraph embedding and the entire graph embedding. Through the corruption function Constructing a negative sampling relationship graph The specific calculations are as follows: ， in, Represents a closed relation graph. This represents the initial characteristics of the relation nodes in a closed relation graph. Indicates through the The initial features of damaged nodes are obtained by randomly shuffling them layer by layer. express The adjacency matrix; Finally, through and Calculate the contrastive learning loss between positive and negative samples This enables the learning of graph embeddings, and the specific calculation is as follows: ， in, This indicates the number of negative triplets. Indicates the number of positive triplet pairs. The embedding representation of the negative sample relation subgraph. Indicates the first The similarity between the negative sample relation subgraph embedding and the entire relation graph embedding; S3.3 Joint Training Objectives: The final loss is obtained by minimizing through joint training. Includes supervised learning loss and contrastive learning loss The calculation process is as follows: ， in, This represents the hyperparameter used to adjust the ratio of supervised learning loss to contrastive learning loss.

2. The inductive link prediction method based on relational message passing and maximizing interaction information according to claim 1, characterized in that, S1 is as follows: S1.2 Subgraph Transformation: Subgraph transformation will extract the K-hop closed entity subgraph. and a one-hop non-closed entity subgraph Transform into a closed relational subgraph and non-closed relational subgraph Closed entity subgraph and unclosed entity subgraphs Using relationships as nodes, edges are added based on connection methods, and topological patterns are labeled. Through traversal and analysis, it is ensured that all relationship nodes and their connection methods are included. Finally, a complete closed relation subgraph is constructed with relationships as nodes and topological patterns as edges. and non-closed relational subgraph ; The specific operation process is as follows: From K-jump closed entity subgraph and a one-hop non-closed entity subgraph Each entity selects a relationship between itself to form a relationship pair. A relationship pair represents the association between two entities through one or more relationships and is used to describe the directionality and topological pattern of the relationship. The relation pairs include six different topological patterns that capture the correlation between arbitrary relations, specifically head-to-tail, tail-to-tail, head-to-head, tail-to-head, parallel, and loop, with corresponding parameter representations of HT, TT, HH, TH, PARA, and LOOP, respectively. Arbitrary relation pairs and The representation between them is or or or or or , Representing relation pairs and Connect in a head-to-tail manner, Representing relation pairs and Connected tail-to-tail. Representing relation pairs and Connect head-to-head. Representing relation pairs and Connected in a tail-to-head manner, Representing relation pairs and Connect in parallel. Representing relation pairs and Connected in a loop.

3. The inductive link prediction method based on relational message passing and maximizing interaction information according to claim 2, S2 is specifically as follows: S2.2, Relationship Message Passing: Calculating closed relation subgraphs using relational message passing The target relationship nodes are aggregated using a two-step iterative process. To switch neighbors, follow these steps: (1) Take the target relation as the root node, select the one-hop neighbor of the target relation node, and then expand it layer by layer to the Kth layer, and use the relation-based graph pruning strategy for optimization; The specific process is as follows: starting from the target relation node, gradually expand the neighborhood of the target relation node to... The layer is calculated, taking into account the direction of information transmission, and all in-degree neighbors of each node are sampled to finally obtain the target relation node. of Jump-in neighbor , Specifically, it is expressed as follows: ， ， in, Target Relationship Node The Jump-in degree neighbor set and , This indicates the target relationship node that the current layer is processing. express Neighbor relationship nodes, express and The types of adjacent edges between them This represents the set of all edges in the relational subgraph. Represents a closed relation subgraph. Indicates the depth of message passing. ; (2) Starting from the final layer, the message passing mechanism iteratively aggregates the information of neighboring nodes layer by layer to generate the final target relation node embedding; Initially adopted The relational graph convolutional network framework assigns transformation matrices to different edge types, aggregates neighbor and self features layer by layer, and generates node representations containing relational semantics; When operating on the data in the NELL-995 dataset, the initial nodes for ontology enhancement are generated using the method in step S2.1.

2. Otherwise, a random initialization method is used to randomly sample the representations of initial nodes from a uniform distribution to provide preliminary random feature representations for the training process; Message passing from In the beginning, The representations of neighboring nodes are aggregated within the jump, and the node embeddings are updated layer by layer to finally obtain the first node. Complete embedding of layer target relationship nodes; Introducing a target-relationship-aware attention mechanism to capture neighboring nodes highly correlated with the target relationship, formally... The GNN aggregation function for the layer is defined as follows: ， in, Represents a node The directed neighborhood, Represents a non-linear activation function. Indicates the first Layer nodes The aggregation has a directed neighborhood embedding. Represents a node In edge type The following is a collection of neighbors. Indicates corresponding to the first Layer edge type The transformation matrix, Indicates the first Neighbors obtained through layer-by-layer iterative learning The expression, Indicates the first Layer in edge type Lower Relationship Node For target relation nodes The importance of express -1 level target relationship nodes Embedding; Finally, the first Layer relationship nodes With the Neighborhood vectors of layer aggregation To update the first by splicing Potential layer nodes , It is expressed as follows: 。 4. The inductive link prediction method based on relational message passing and maximizing interaction information according to claim 3, S2 is specifically as follows: S2.3, One-hop neighbor aggregation: First, for the target relation nodes We sample one-hop neighbors and aggregate their features using an attention mechanism. The calculation process is as follows: ， in, Indicates target relationship A set of neighbors that jumps over time. Indicating neighborly relations Embedded, This represents the linear transformation matrix for each relation node. Represents a non-linear activation function, attention weights Used to measure neighbor relationships among nodes For target relation nodes The importance of; S2.4, Complete Subgraph Embedding: Features learned from non-closed relation subgraphs through one-hop neighbor aggregation Features learned through relational message passing in closed relational subgraphs The data is integrated using summation and concatenation to form the final representation of the relational subgraph. , means as follows: in, This represents the summation operation. This indicates a splicing operation.

Images