Induction link prediction method based on relation message passing and interaction information maximization

Abstract

The invention relates to the technical field of knowledge graph completion, in particular to an inductive knowledge graph completion method for relation message passing based on mutual information maximization. The method comprises the following steps: extracting a K-hop closed entity subgraph # imgabs0 # and a one-hop non-closed entity subgraph # imgabs1 # around a target triple, and then respectively converting the # imgabs2 # and the # imgabs3 # into a closed relation subgraph # imgabs4 # and a non-closed relation subgraph # imgabs5 #; the method comprises the following steps: constructing an ontology graph, generating an ontology enhanced initial node according to a training result of the ontology graph, then calculating a target relationship in a closed relationship sub-graph # imgabs6 # through relationship message transmission, calculating a target relationship in an unclosed relationship sub-graph # imgabs7 # through one-hop neighbor aggregation, and finally combining into a complete sub-graph and embedding the complete sub-graph # imgabs8 #; and calculating supervised learning loss # imgabs9 # according to complete subgraph embedding, calculating contrast learning loss # imgabs10 # by maximizing mutual information, and obtaining total loss for joint training. The more optimized local subgraph method researched by the method can better solve the semantic problem in real induction knowledge graph reasoning.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge graph completion, and in particular to an inductive link prediction method based on relational message passing and mutual information maximization. Background Art

[0002] Knowledge graphs (KGs) organize real-world human knowledge in the form of fact triples (head entity, relationship, tail entity), where entities are represented as nodes and relationships are represented as edges. In recent years, many large knowledge bases have been widely used in natural language processing, intelligent question answering, and recommendation systems. However, due to the exponential growth of real-world data, KGs often have noise and incompleteness, that is, some links are missing. Although there have been many successful studies on link prediction, it is still challenging in real-world scenarios. The real-world KGs evolve dynamically. The entities and relationships involved in the triplets predicted by traditional direct link prediction are all observed during training. Therefore, when new entities and new relationships appear (that is, invisible entities and invisible relationships), the KG needs to be retrained, which will increase additional costs; In recent years, researchers have gradually focused on inductive link prediction, aiming to predict triples containing invisible entities and relations without retraining the knowledge graph. The challenge in this field is mainly reflected in how to effectively handle different types of invisible elements: in the semi-inductive setting, invisible entities need to be connected to visible entities in the original knowledge graph; in the full inductive setting, it is necessary to predict invisible-invisible entity combinations in the emerging knowledge graph; and in the true inductive setting, it is necessary to further solve the modeling and reasoning problems of invisible relations. Therefore, the present invention provides an inductive link prediction method based on relational message passing and mutual information maximization to solve the above problems. Summary of the invention

[0003] In view of the shortcomings of the prior art, the present invention develops an inductive link prediction method based on relational message passing and mutual information maximization. The main purpose is to study a more optimized local subgraph method to better solve the semantic problems in truly inductive knowledge graph reasoning.

[0004] The technical solution of the present invention to solve the technical problem is an inductive link prediction method based on relational message passing and mutual information maximization, which is as follows: S1. Subgraph extraction and transformation: extracting K-hop closed entity subgraphs around target triples and one-hop unclosed entity subgraph , then and Convert to closed relation subgraphs respectively and unclosed relation subgraph ; S2, subgraph message passing: construct the ontology graph, generate the initial node of ontology enhancement according to the training results of the ontology graph, and then calculate the closed relation subgraph through relation message passing The target relationship in the unclosed relationship subgraph is calculated by one-hop neighbor aggregation The target relationship in is finally combined into a complete subgraph embedding ; S3. Joint training: Compute supervised learning loss based on complete subgraph embeddings , by maximizing the mutual information to calculate the contrastive learning loss , and get the total loss for joint training.

[0005] S1 is as follows: S1.1 Sub-graph extraction: Around the target triple Extract K-hop closed entity subgraphs separately and one-hop unclosed entity subgraph ,Will and Combined with joint inductive reasoning, where Represents the head entity, Represents the tail entity, Represents the header entity and tail entity The relationship between entities and Represented as nodes, relationships are represented as edges; S1.1.1 Extracting K-hop closed entity subgraphs : According to the head entity in the target triple calculate - Jump into the neighbor , according to the tail entity in the target triple Calculate degree neighbors ; Find the head entity of - Jump into the neighbor and tail entity of -Bounce Neighbors The intersection between them gives the common neighbors ; Remove common neighbors Isolated nodes and nodes in the distance triplet and Exceed Jump to the node and get - Skip closed entity subgraph , The distance between all nodes and the target triplet node is within the maximum distance within the scope; in, - Skip closed entity subgraph The time complexity is , It represents the time complexity, The index indicating the number of hops. , , and Respectively - Skip closed entity subgraph The number of nodes, relationships, and edges in the , and the embedding vector dimensions of nodes and relationships are ; S1.1.2. Extracting a one-hop non-closed entity subgraph : According to the head entity in the target triple calculate - Neighborhood Hopping , according to the tail entity in the target triple calculate - Neighborhood Hopping ; Find the head entity of - Neighborhood Hopping and tail entity of - Neighborhood Hopping The union between them gets all neighbor nodes ; The head entity in the target triple , tail entity , all neighbor nodes , and the edges between the entities in the target triple and the entities corresponding to the neighbor nodes, together forming a one-hop non-closed entity subgraph Sub-graph.

[0006] S1.2, sub-image conversion: Subgraph transformation extracts K-hop closed entity subgraphs and one-hop unclosed entity subgraph Convert to closed relation subgraph and unclosed relation subgraph , the closed entity subgraph and unclosed entity subgraphs The relationship is used as a node, and edges are added according to the connection mode and the topological mode is marked. Through traversal and analysis, it is ensured that all relationship nodes and their connection modes are included. Finally, a complete closed relationship subgraph is constructed with relationships as nodes and topological modes as edges. and unclosed relation subgraph ; The specific operation process is as follows: From K-hop closed entity subgraph and one-hop unclosed entity subgraph The relationship between each selected entity forms a relationship pair. The relationship pair represents the association between two entities formed by one or more relationships, and is used to describe the directionality and topological pattern of the relationship. The relation pairs include six different topological modes that capture the association between arbitrary relations, including head-tail, tail-tail, head-head, tail-head, parallel and loop, and the corresponding parameter representations are HT, TT, HH, TH, PARA and LOOP respectively; Any relationship pair and The expression between or or or or or , Represents a relationship and Connect in a head-to-tail manner. Represents a relationship and Connect in a tail-to-tail manner. Represents a relationship and Connect in head-to-head mode. Represents a relationship and Connect in a tail-head manner. Represents a relationship and Connect in parallel, Represents a relationship and Connect in a loop.

[0007] S2 is as follows: S2.1. Construct the ontology graph and generate the initial nodes of ontology enhancement: S2.1.1. Constructing ontology graph: (1) Determine the semantic vocabulary. The semantic vocabulary captures the semantic relationship of the relationship, and gives priority to the vocabulary that supports reasoning and modeling relational semantics. The key terms of the semantic vocabulary include "rdfs:subPropertyOf" for describing the hierarchical relationship between properties, "rdfs:subClassOf" for defining the hierarchical relationship between classes, "rdfs:domain" for specifying the head entity type of the relationship, and "rdfs:range" for specifying the tail entity type of the relationship. (2) Exclude irrelevant terms. Irrelevant terms are terms that are not used to express semantic relationships. Irrelevant terms include basic semantic structures used to define sets, sequences, and instance types. (3) Extract relation information from the NELL-995 dataset and annotate the semantic attributes of each relation. and Represents any two relationship information extracted from the data set. Select one key term from the four key terms to define the two relationships and obtain an RDF triple, which is expressed as , the ontology graph is constructed based on the RDF triples composed of relational information and key terms extracted from the data set. The RDF triples represent the resource description framework triples, which are composed of subject, predicate and object. , the predicate corresponds to the key term, the object corresponds to the relationship ; Among them, the NELL-995 dataset is a subset of knowledge graphs automatically extracted from the network for building semantic ontology and supporting knowledge reasoning and relationship extraction, containing 995 categories and their relationships; (4) The reasoning tool OWL Reasoner performs semantic reasoning and logical verification on the ontology graph. OWL Reasoner is a tool to ensure structural consistency and further optimize the ontology graph. S2.1.2. Generate the initial node of ontology enhancement: The ontology graph is used as the prior knowledge of the initial node, and the ontology graph is pre-trained through the knowledge graph embedding technology TransE to obtain the representation of the relationship , and then project the relationship embedding onto the relationship subgraph through a two-layer fully connected network to generate the initial node of the ontology enhancement. The calculation process is as follows: , in, Indicates the initial node of the ontology enhancement, the corner mark Indicates the initialization state; represents the weight matrix of the first layer of the fully connected network, represents the weight matrix of the second layer of the fully connected network, Represents the pre-trained Node representation of .

[0008] S2.2, Relationship Message Passing: Compute closed relation subgraphs via relation message passing The target relationship node in the target relationship node is specifically a two-step iterative aggregation target relationship node. To jump to a neighbor, the steps are as follows: (1) Taking the target relationship as the root node, select the one-hop neighbor of the target relationship node, then expand it layer by layer to the Kth layer, and use the relationship-based graph pruning strategy for optimization; The specific process is: starting from the target relationship node, gradually expand the neighborhood of the target relationship node to layer, and consider the direction of information transmission, sample all in-degree neighbors for each node, and finally obtain the target relationship node of Hop-in Neighbor , The specific expressions are as follows: , , in, Target relationship node No. Jump into the neighbor set, and , Indicates the target relationship node that the current layer is processing. express Neighbor relationship nodes, express and The type of adjacent edges between represents the set of all edges in the relation subgraph, represents a closed relation subgraph, Indicates the depth of message passing, ; (2) Starting from the final layer, the message passing mechanism iteratively aggregates the information of neighboring nodes layer by layer to generate the final target relation node embedding; Initially adopted The relational graph convolutional network framework assigns transformation matrices to different edge types, aggregates neighbor and self features layer by layer, and generates node representations that contain relational semantics; When operating on the data in the NELL-995 dataset, the method in step S2.1.2 is used to generate the initial node of ontology enhancement ,Otherwise, a random initialization method is used to randomly sample the representation of the initial node from a uniform distribution to provide a preliminary random feature representation for the training process; Messaging from Start at Aggregate the representation of neighbor nodes within the hop, update the node embedding layer by layer, and finally obtain the The complete embedding of the layer target relation nodes; The target relationship-aware attention mechanism is introduced to capture the neighboring nodes that are highly related to the target relationship. The GNN aggregation function of the layer is defined as follows: , in, Representation Node The directed neighborhood of represents a nonlinear activation function, Indicates Layer Node The aggregation of the directed neighborhood embedding, Representation Node On edge type The neighbor set below, Indicates that the corresponding Layer Edge Type The transformation matrix, Indicated in Neighbors learned by layer iteration The expression, Indicates Layer on edge type Lower relationship node For target relationship nodes The importance of express -1 layer target relationship node Embedding Finally, the Layer Relationship Node With Neighborhood vector of layer aggregation Splice to update Potential of layer nodes , It is expressed as follows: .

[0009] S2.3, One-hop neighbor aggregation: First, for the target relationship node The one-hop neighbors are sampled and the one-hop neighbors’ features are aggregated using the attention mechanism. , the calculation process is as follows: , in, Represents the target relationship The set of one-hop neighbors of Indicates neighbor relationship The embedding Represents the linear transformation matrix of each relationship node, represents the nonlinear activation function, attention weight Used to measure neighbor relationship nodes For target relationship nodes The importance of S2.4. Complete subgraph embedding: Features learned in unclosed relation subgraphs are aggregated via one-hop neighbor aggregation Features learned through relational message passing in closed relational subgraphs Integrate, use summation and concatenation to form the final representation of the relationship subgraph , which is expressed as follows: in, represents the sum operation, Represents a concatenation operation.

[0010] S3 is as follows: S3.1. Supervised Learning: According to the final representation of the relationship subgraph Calculate the target triple The likelihood score of , the specific calculation is as follows: , in, represents the relationship weight matrix; Then the supervised learning loss Optimize the likelihood score of the triples and replace the triples The head entity or tail entity in generates an equal number of negative sample triplets and positive samples, and ensures that the score of the positive sample triplets is significantly higher than that of the negative sample triplets. The calculation process is as follows: in, represents the number of all target triples input, and Respectively represent the target triples After replacement, we get negative triples and positive triples. represents the likelihood score of negative triples, represents the likelihood score of a positive triplet, represents the boundary hyperparameter, Indicates taking the maximum value, represents the index of the triple, , ; S3.2, mutual information maximization: By maximizing the local and global interaction information MI, we construct a global topological pattern between relationships, covering the subgraph and layer levels. First, we use the read function to obtain the global representation of the graph. The specific calculation is as follows: , in, Indicates The embedding of the local relation subgraph of the positive triples, represents the embedding of the entire relationship graph, Represents the number of all positive sample triplets input; Then, a bilinear score function is introduced as a discriminator, and the Jensen-Shannon MI estimator is applied to estimate the mutual information between the relationship subgraph embedding and the entire relationship graph embedding, so as to maximize the mutual information between the two and determine whether they come from the same graph. The Jensen-Shannon MI estimator is a method for estimating the mutual information MI between two random variables. The bilinear score function is as follows: , in, represents the activation function, represents the score matrix of mutual information MI, Represents the similarity between the embedding of the relation subgraph and the embedding of the entire relation graph; Through the corruption function Constructing the Negative Sampling Relationship Graph , the specific calculation is as follows: , in, represents a closed relationship graph, represents the initial features of the relationship nodes in the closed relationship graph, Indicates that through The initial features of the damaged nodes are obtained by random perturbation layer by layer. express The adjacency matrix of Finally, through and Compute contrastive learning loss between positive and negative samples , and then realize the learning of graph embedding. The specific calculation is as follows: , in, represents the number of negative sample triplets, represents the number of positive sample triplets, represents the embedding representation of the negative sample relation subgraph, Indicates The similarity between the embedding of the negative sample relation subgraph and the embedding of the entire relation graph; S3.3 Joint training objectives: The final loss is minimized by joint training. Contains supervised learning loss and contrastive learning loss , the calculation process is as follows: , in, represents the hyperparameter used to adjust the ratio of supervised learning loss and contrastive learning loss.

[0011] The effects provided in the content of the invention are only the effects of the embodiments, rather than all the effects of the invention. The above technical solution has the following advantages or beneficial effects: The method in the present invention proposes a new model based on a relational message passing network, denoted as RPMI. Through RPMI, closed entity subgraphs and one-hop unclosed entity subgraphs are extracted around the target triples, and topological patterns are merged between relations to convert the entity graph into closed relation subgraphs and unclosed relation subgraphs. Node features are enhanced by injecting ontological patterns of knowledge graphs, and reasoning is completed by combining relational message passing and one-hop neighbor aggregation. In addition, to strengthen the global modeling of topological patterns between relations, RPMI maximizes the interactive information between the subgraph and the overall graph. Therefore, the present invention can perform well in the full induction link prediction task, maintain a high prediction accuracy in invisible entity scenarios, and achieve important breakthroughs in the true induction link prediction task, providing strong support for the joint reasoning of relations, opening up new solutions and methods for reasoning problems in complex knowledge graph scenarios, and demonstrating excellent performance and broad application prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0012] The accompanying drawings are used to provide further understanding of the present invention and constitute a part of the specification. They are used to explain the present invention together with the embodiments of the present invention and do not constitute a limitation of the present invention.

[0013] Figure 1 It is a structural diagram of the method of the present invention.

[0014] Figure 2 The ablation experiment results. DETAILED DESCRIPTION

[0015] In order to clearly illustrate the technical features of the present invention, the present invention is described in detail below through specific implementation methods and in conjunction with the accompanying drawings.

[0016] Example 1 like Figure 1 As shown in Figure 1, an inductive link prediction method based on relational message passing and mutual information maximization is as follows: S1. Subgraph extraction and transformation: extracting K-hop closed entity subgraphs around target triples and one-hop unclosed entity subgraph , then and Convert to closed relation subgraphs respectively and unclosed relation subgraph ; S2, subgraph message passing: construct the ontology graph, generate the initial node of ontology enhancement according to the training results of the ontology graph, and then calculate the closed relation subgraph through relation message passing The target relationship in the unclosed relationship subgraph is calculated by one-hop neighbor aggregation The target relationship in is finally combined into a complete subgraph embedding ; S3. Joint training: Compute supervised learning loss based on complete subgraph embeddings , by maximizing the mutual information to calculate the contrastive learning loss , and get the total loss for joint training.

[0017] S1 is as follows: S1.1 Sub-graph extraction: Around the target triple Extract K-hop closed entity subgraphs separately and one-hop unclosed entity subgraph ,Will and Combined with joint inductive reasoning, where Represents the head entity, Represents the tail entity, Represents the header entity and tail entity The relationship between entities and Represented as nodes, relationships are represented as edges; S1.1.1 Extracting K-hop closed entity subgraphs : According to the head entity in the target triple calculate - Jump into the neighbor , according to the tail entity in the target triple Calculate degree neighbors ; Find the head entity of - Jump into the neighbor and tail entity of -Bounce Neighbors The intersection between them gives the common neighbors ; Remove common neighbors Isolated nodes and nodes in the distance triplet and Exceed Jump to the node and get - Skip closed entity subgraph , The distance between all nodes and the target triplet node is within the maximum distance within the scope; in, - Skip closed entity subgraph The time complexity is , It represents the time complexity, The index indicating the number of hops. , , and Respectively - Skip closed entity subgraph The number of nodes, relationships, and edges in the , and the embedding vector dimensions of nodes and relationships are ; S1.1.2. Extracting a one-hop non-closed entity subgraph : According to the head entity in the target triple calculate - Neighborhood Hopping , according to the tail entity in the target triple calculate - Neighborhood Hopping ; Find the head entity of - Neighborhood Hopping and tail entity of - Neighborhood Hopping The union between them gets all neighbor nodes ; The head entity in the target triple , tail entity , all neighbor nodes , and the edges between the entities in the target triple and the entities corresponding to the neighbor nodes, together forming a one-hop non-closed entity subgraph Sub-graph.

[0018] S1.2, sub-image conversion: Subgraph transformation extracts K-hop closed entity subgraphs and one-hop unclosed entity subgraph Convert to closed relation subgraph and unclosed relation subgraph , the closed entity subgraph and unclosed entity subgraphs The relationship is used as a node, and edges are added according to the connection mode and the topological mode is marked. Through traversal and analysis, it is ensured that all relationship nodes and their connection modes are included. Finally, a complete closed relationship subgraph is constructed with relationships as nodes and topological modes as edges. and unclosed relation subgraph ; The specific operation process is as follows: From K-hop closed entity subgraph and one-hop unclosed entity subgraph The relationship between each selected entity forms a relationship pair. The relationship pair represents the association between two entities formed by one or more relationships, and is used to describe the directionality and topological pattern of the relationship. The relation pairs include six different topological modes that capture the association between arbitrary relations, including head-tail, tail-tail, head-head, tail-head, parallel and loop, and the corresponding parameter representations are HT, TT, HH, TH, PARA and LOOP respectively; Any relationship pair and The expression between or or or or or , Represents a relationship and Connect in a head-to-tail manner. Represents a relationship and Connect in a tail-to-tail manner. Represents a relationship and Connect in head-to-head mode. Represents a relationship and Connect in a tail-head manner. Represents a relationship and Connect in parallel, Represents a relationship and Connect in a loop.

[0019] S2 is as follows: S2.1. Construct the ontology graph and generate the initial nodes of ontology enhancement: S2.1.1. Constructing ontology graph: (1) Determine the semantic vocabulary. The semantic vocabulary captures the semantic relationship of the relationship, and gives priority to the vocabulary that supports reasoning and modeling relational semantics. The key terms of the semantic vocabulary include "rdfs:subPropertyOf" for describing the hierarchical relationship between properties, "rdfs:subClassOf" for defining the hierarchical relationship between classes, "rdfs:domain" for specifying the head entity type of the relationship, and "rdfs:range" for specifying the tail entity type of the relationship. (2) Exclude irrelevant terms. Irrelevant terms are terms that are not used to express semantic relationships. Irrelevant terms include basic semantic structures used to define sets, sequences, and instance types. (3) Extract relation information from the NELL-995 dataset and annotate the semantic attributes of each relation. and Represents any two relationship information extracted from the data set. Select one key term from the four key terms to define the two relationships and obtain an RDF triple, which is expressed as , the ontology graph is constructed based on the RDF triples composed of relational information and key terms extracted from the data set. The RDF triples represent the resource description framework triples, which are composed of subject, predicate and object. , the predicate corresponds to the key term, the object corresponds to the relationship ; Among them, the NELL-995 dataset is a subset of knowledge graphs automatically extracted from the network for building semantic ontology and supporting knowledge reasoning and relationship extraction, containing 995 categories and their relationships; (4) The reasoning tool OWL Reasoner performs semantic reasoning and logical verification on the ontology graph. OWL Reasoner is a tool to ensure structural consistency and further optimize the ontology graph. S2.1.2. Generate the initial node of ontology enhancement: The ontology graph is used as the prior knowledge of the initial node, and the ontology graph is pre-trained through the knowledge graph embedding technology TransE to obtain the representation of the relationship , and then project the relationship embedding onto the relationship subgraph through a two-layer fully connected network to generate the initial node of the ontology enhancement. The calculation process is as follows: , in, Indicates the initial node of the ontology enhancement, the corner mark Indicates the initialization state; represents the weight matrix of the first layer of the fully connected network, represents the weight matrix of the second layer of the fully connected network, Represents the pre-trained Node representation of .

[0020] S2.2, Relationship Message Passing: Compute closed relation subgraphs via relation message passing The target relationship node in the target relationship node is specifically a two-step iterative aggregation target relationship node. To jump to a neighbor, the steps are as follows: (1) Taking the target relationship as the root node, select the one-hop neighbor of the target relationship node, then expand it layer by layer to the Kth layer, and use the relationship-based graph pruning strategy for optimization; The specific process is: starting from the target relationship node, gradually expand the neighborhood of the target relationship node to layer, and consider the direction of information transmission, sample all in-degree neighbors for each node, and finally obtain the target relationship node of Hop-in Neighbor , The specific expressions are as follows: , , in, Target relationship node No. Jump into the neighbor set, and , Indicates the target relationship node that the current layer is processing. express Neighbor relationship nodes, express and The type of adjacent edges between represents the set of all edges in the relation subgraph, represents a closed relation subgraph, Indicates the depth of message passing, ; (2) Starting from the final layer, the message passing mechanism iteratively aggregates the information of neighboring nodes layer by layer to generate the final target relation node embedding; Initially adopted The relational graph convolutional network framework assigns transformation matrices to different edge types, aggregates neighbor and self features layer by layer, and generates node representations that contain relational semantics; When operating on the data in the NELL-995 dataset, the method in step S2.1.2 is used to generate the initial node of ontology enhancement ,Otherwise, a random initialization method is used to randomly sample the representation of the initial node from a uniform distribution to provide a preliminary random feature representation for the training process; Messaging from Start at Aggregate the representation of neighbor nodes within the hop, update the node embedding layer by layer, and finally obtain the The complete embedding of the layer target relation nodes; The target relationship-aware attention mechanism is introduced to capture the neighboring nodes that are highly related to the target relationship. The GNN aggregation function of the layer is defined as follows: , in, Representation Node The directed neighborhood of represents a nonlinear activation function, Indicates Layer Node The aggregation of the directed neighborhood embedding, Representation Node On edge type The neighbor set below, Indicates that the corresponding Layer Edge Type The transformation matrix, Indicated in Neighbors learned by layer iteration The expression, Indicates Layer on edge type Lower relationship node For target relationship nodes The importance of express -1 layer target relationship node Embedding Finally, the Layer Relationship Node With Neighborhood vector of layer aggregation Splice to update Potential of layer nodes , It is expressed as follows: .

[0021] S2.3, One-hop neighbor aggregation: First, for the target relationship node The one-hop neighbors are sampled and the one-hop neighbors’ features are aggregated using the attention mechanism. , the calculation process is as follows: , in, Represents the target relationship The set of one-hop neighbors of Indicates neighbor relationship The embedding Represents the linear transformation matrix of each relationship node, represents the nonlinear activation function, attention weight Used to measure neighbor relationship nodes For target relationship nodes The importance of S2.4. Complete subgraph embedding: Features learned in unclosed relation subgraphs are aggregated via one-hop neighbor aggregation Features learned through relational message passing in closed relational subgraphs Integrate, use summation and concatenation to form the final representation of the relationship subgraph , which is expressed as follows: in, represents the sum operation, Represents a concatenation operation.

[0022] S3 is as follows: S3.1. Supervised Learning: According to the final representation of the relationship subgraph Calculate the target triple The likelihood score of , the specific calculation is as follows: , in, represents the relationship weight matrix; Then the supervised learning loss Optimize the likelihood score of the triples and replace the triples The head entity or tail entity in generates an equal number of negative sample triplets and positive samples, and ensures that the score of the positive sample triplets is significantly higher than that of the negative sample triplets. The calculation process is as follows: in, represents the number of all target triples input, and Respectively represent the target triples After replacement, we get negative triples and positive triples. represents the likelihood score of negative triples, represents the likelihood score of a positive triplet, represents the boundary hyperparameter, Indicates taking the maximum value, represents the index of the triple, , ; S3.2, mutual information maximization: By maximizing local and global interaction information MI, we construct a global topological pattern between relationships, covering subgraph and layer levels. First, we use the read function to obtain the global representation of the graph. The specific calculation is as follows: , in, Indicates The embedding of the local relation subgraph of the positive triples, represents the embedding of the entire relationship graph, Represents the number of all positive sample triplets input; Then, a bilinear score function is introduced as a discriminator, and the Jensen-Shannon MI estimator is applied to estimate the mutual information between the relationship subgraph embedding and the entire relationship graph embedding, so as to maximize the mutual information between the two and determine whether they come from the same graph. The Jensen-Shannon MI estimator is a method for estimating the mutual information MI between two random variables. The bilinear score function is as follows: , in, represents the activation function, represents the score matrix of mutual information MI, Represents the similarity between the embedding of the relation subgraph and the embedding of the entire relation graph; Through the corruption function Constructing the Negative Sampling Relationship Graph , the specific calculation is as follows: , in, represents a closed relationship graph, represents the initial features of the relationship nodes in the closed relationship graph, Indicates that through The initial features of the damaged nodes are obtained by random perturbation layer by layer. express The adjacency matrix of Finally, through and Compute contrastive learning loss between positive and negative samples , and then realize the learning of graph embedding. The specific calculation is as follows: , in, represents the number of negative sample triplets, represents the number of positive sample triplets, represents the embedding representation of the negative sample relation subgraph, Indicates The similarity between the embedding of the negative sample relation subgraph and the embedding of the entire relation graph; S3.3 Joint training objectives: The final loss is minimized by joint training. Contains supervised learning loss and contrastive learning loss , the calculation process is as follows: , in, represents the hyperparameter used to adjust the ratio of supervised learning loss and contrastive learning loss.

[0023] Example 2 The present invention uses two sets of data sets to evaluate the method proposed in the present invention. The first set is the data sets WN18RR.v1 / v2 and FB15k-237.v1 / v2, which are derived from the data sets proposed by the GraIL model. It is the first data set designed for the fully inductive link prediction scenario. The characteristic of this data set is that the entities in the test set do not appear in the training set, but the relationships in the test set already exist. It is mainly used to evaluate the model's inductive reasoning ability for unseen entities, as shown in Table 1.

[0024] Table 1. Statistics of datasets for fully inductive link prediction for unseen-unseen entities The second set of data sets are NELL-995.v1.v3 and NELL-995.v2.v3, which are derived from the data sets proposed by the RMPI model. Each data set contains two versions: one contains semi-invisible relations and the other contains fully invisible relations. This data set is designed for true inductive link prediction scenarios. The test set contains unseen entities and unseen relations, which is used to test the comprehensive inductive ability of the model in unseen entity and unseen relation scenarios. The data set is shown in Table 2.

[0025] Table 2. Statistics of datasets for true inductive link prediction for unseen-unseen entities and unseen relations For fair comparison, triple classification and entity prediction tasks are used for evaluation. The triple classification uses the AUC-PR indicator. AUC-PR represents the area under the precision and recall curves, which is used to measure the ability of the classification model to distinguish between positive and negative samples. The higher the value, the better the effect. Entity prediction uses the MRR and Hits@N indicators, where MRR represents the average reciprocal ranking of the correct answer in the candidate list, and Hits@N represents the proportion of the correct answer appearing in the top N of the candidate list. The higher the value, the better the effect. In order to predict the missing head or tail entity, 50 negative samples are randomly sampled for each positive sample triple. The higher the AUC-PR, MRR and Hits@N, the better the link prediction performance. Five experiments were conducted in this invention and the average results were taken. The present invention implements the model through PyTorch and uses the Adam optimizer. Each triplet samples a 2-hop closed subgraph and a 1-hop open subgraph, and applies two layers of message passing on the closed subgraph. The hyperparameters are manually adjusted during training: learning rate In {0.1, 0.01, 0.001, 0.0001}, the feature dimension In {16, 32, 64, 128}, the edge drop rate At {0.1, 0.3, 0.5, 0.8}, the mutual information loss coefficient In {1, 3, 5, 7}, the optimal LP configuration is summarized as 0.001, 32, =0.5, =1; To evaluate the performance of RPMI in full inductive LP, the present invention is compared with five benchmark models: GraIL model, TACT-Base model, TACT model, CoMPILE model and RMPI model. These models are derived from inductive learning scenarios and promote the performance optimization of relational prediction models. The GraIL model first proposed to get rid of global graph dependence through subgraph extraction, node labeling and GNN scoring methods, which is suitable for inductive learning; the TACT model introduces relational topological correlation based on GraIL, combines relational modules with graph structure modules, and enhances graph structure reasoning capabilities; the TACT-Base model is a simplified version of TACT, which only uses relational modules to provide a comparison baseline; the CoMPILE model extends GraIL and introduces a node-edge communication mechanism to enhance the feature learning ability of node and edge information interaction; the RMPI model proposes an innovative relational message passing network and designs the RMPI-NE mechanism, which is designed for true inductive LP and is suitable for processing unseen entities and relationships.

[0026] Due to the limited applicability of these two techniques on all datasets, we propose multiple RPMI variants. First, there are two ways to initialize the unseen relations in the test graph: i) RMPI: random initialization, ii) RMPI+: pattern-enhanced initialization. Second, the target relation-aware neighborhood attention mechanism (TA) used in the one-hop neighborhood aggregation of open subgraphs is not applicable to the true induction setting that contains both unseen entities and unseen relations, but only to the full induction setting that only contains unseen entities. Therefore, we test multiple variants: RPMI+ (without TA) for the true induction setting, and RPMI-TA (with TA) for the full induction setting. For the true induction setting, the semantic vectors learned from the ontology patterns are used to initialize the representation of the unseen relations in the test graph (limited to NELL related datasets). In addition, two different fusion functions based on summation (SUM) and concatenation (CONC) are used, and their respective results are reported. The results are shown in Table 3, and the best score is shown in bold. From the experimental results of ontology-enhanced unseen relations in Table 3, it can be seen that the relation association module of TACT-base only aggregates one layer of neighborhood of the target relation, resulting in poor performance, while RMPI outperforms TACT-base through direct iterative message passing between relations; the proposed RPMI+ further globally models the topological patterns between relations by maximizing mutual information (MI), and outperforms all other SOTA models in the scenarios of semi-unseen and fully unseen relations, except for the semi-unseen relation MRR of NELL-995.v2.v3.

[0027] Table 3 Results on true inductive link prediction For the full induction setting, two different fusion functions based on summation (SUM) and concatenation (CONC) were used. Only MRR and Hits@10 indicators were used to evaluate the performance of the model in the full induction LP. The results are shown in Table 4, and the best score is shown in bold. It can be seen that compared with the baseline model that only uses local subgraphs for reasoning, the proposed RPMI-TA has the ability to fuse local (subgraph) and global (graph) information. Both indicators are significantly better than the baseline model on most datasets; however, the MRR on the WN18RR.v2 dataset and the Hits@10 on the FB15k-237.v1 dataset are slightly lower, which may be because relying solely on subgraph reasoning provides sufficient information, and adding MI maximization to learn global structural information has a negative impact.

[0028] Table 4 Results on fully inductive link prediction This paper selects the fully invisible relations of NELL-995.v1.v3 and WN18RR.v1 datasets, and conducts ablation experiments on RPMI+(CONC) and RPMI-TA(CONC), respectively, removing each component one by one to analyze its role. As shown in Figure 2, the experimental results of Hits@10 are reported. These components include: 1) ontology mode injection of knowledge graph (denoted as w / oOS), 2) one-hop neighborhood aggregation (denoted as w / o NE), 3) mutual information maximization (denoted as w / o MI), and 4) target relationship-aware neighborhood attention mechanism (denoted as w / o TA). Figure 2 It can be seen that each component has an important impact on the model performance. Removing key components such as ontology pattern injection or one-hop neighborhood aggregation will significantly reduce Hits@10, indicating that these components are crucial in improving link prediction performance.

[0029] Although the above describes the specific implementation mode of the invention in conjunction with the drawings, it is not intended to limit the scope of protection of the invention. Based on the technical solution of the present invention, various modifications or variations that can be made by those skilled in the art without creative work are still within the scope of protection of the present invention.

Claims

1. An inductive link prediction method based on relational message passing and mutual information maximization, characterized in that, The following steps are involved: S1. Subgraph extraction and transformation: extracting K-hop closed entity subgraphs around target triples and one-hop unclosed entity subgraph , then and Convert to closed relation subgraphs respectively and unclosed relation subgraph ; S2, subgraph message passing: construct the ontology graph, generate the initial node of ontology enhancement according to the training results of the ontology graph, and then calculate the closed relation subgraph through relation message passing The target relationship in the unclosed relationship subgraph is calculated by one-hop neighbor aggregation The target relationship in is finally combined into a complete subgraph embedding ; S3. Joint training: Compute supervised learning loss based on complete subgraph embeddings , by maximizing the mutual information to calculate the contrastive learning loss , and get the total loss for joint training.

2. The inductive link prediction method based on relational message passing and mutual information maximization according to claim 1 is characterized in that: S1 is as follows: S1.1 Sub-graph extraction: Around the target triple Extract K-hop closed entity subgraphs separately and one-hop unclosed entity subgraph ,Will and Combined with joint inductive reasoning, where Represents the head entity, Represents the tail entity, Represents the header entity and tail entity The relationship between entities and Represented as nodes, relations are represented as edges; S1.1.1 Extracting K-hop closed entity subgraphs : According to the head entity in the target triple calculate - Jump into the neighbor , according to the tail entity in the target triple Calculate degree neighbors ; Find the head entity of - Jump into the neighbor and tail entity of -Bounce Neighbors The intersection between them gives the common neighbors ; Remove common neighbors Isolated nodes and nodes in the distance triplet and Exceed Jump to the node and get - Skip closed entity subgraph , The distance between all nodes and the target triplet node is within the maximum distance within the scope; in, - Skip closed entity subgraph The time complexity is , It represents the time complexity, An index indicating the number of hops. , , and Respectively - Skip closed entity subgraph The number of nodes, relationships, and edges in the , and the embedding vector dimensions of nodes and relationships are ; S1.1.

2. Extracting a one-hop non-closed entity subgraph : According to the head entity in the target triple calculate - Neighborhood Hopping , according to the tail entity in the target triple calculate - Neighborhood Hopping ; Find the head entity of - Neighborhood Hopping and tail entity of - Neighborhood Hopping The union between them gets all neighbor nodes ; The head entity in the target triple , tail entity , all neighbor nodes , and the edges between the entities in the target triple and the entities corresponding to the neighbor nodes, together forming a one-hop non-closed entity subgraph Sub-graph.

3. The inductive link prediction method based on relational message passing and mutual information maximization according to claim 2 is characterized in that: S1 is as follows: S1.2, sub-image conversion: Subgraph transformation extracts K-hop closed entity subgraphs and one-hop unclosed entity subgraph Convert to closed relation subgraph and unclosed relation subgraph , the closed entity subgraph and unclosed entity subgraphs The relationship is used as a node, and edges are added according to the connection mode and the topological mode is marked. Through traversal and analysis, it is ensured that all relationship nodes and their connection modes are included. Finally, a complete closed relationship subgraph is constructed with relationships as nodes and topological modes as edges. and unclosed relation subgraph ; The specific operation process is as follows: From K-hop closed entity subgraph and one-hop unclosed entity subgraph The relationship between each selected entity forms a relationship pair. The relationship pair represents the association between two entities formed by one or more relationships, and is used to describe the directionality and topological pattern of the relationship. The relation pairs include six different topological modes that capture the association between arbitrary relations, including head-tail, tail-tail, head-head, tail-head, parallel and loop, and the corresponding parameter representations are HT, TT, HH, TH, PARA and LOOP respectively; Any relationship pair and The expression between or or or or or , Represents a relationship and Connect in a head-to-tail manner. Represents a relationship and Connect in a tail-to-tail manner. Represents a relationship and Connect in a head-to-head manner. Represents a relationship and Connect in a tail-head manner. Represents a relationship and Connect in parallel, Represents a relationship and Connect in a loop.

4. According to the inductive link prediction method based on relational message passing and mutual information maximization according to claim 3, S2 is specifically as follows: S2.

1. Construct the ontology graph and generate the initial nodes of ontology enhancement: S2.1.

1. Constructing ontology graph: (1) Determine the semantic vocabulary. The semantic vocabulary captures the semantic relationship of the relationship, and gives priority to the vocabulary that supports reasoning and modeling relational semantics. The key terms of the semantic vocabulary include "rdfs:subPropertyOf" for describing the hierarchical relationship between properties, "rdfs:subClassOf" for defining the hierarchical relationship between classes, "rdfs:domain" for specifying the head entity type of the relationship, and "rdfs:range" for specifying the tail entity type of the relationship. (2) Exclude irrelevant terms. Irrelevant terms are terms that are not used to express semantic relationships. Irrelevant terms include basic semantic structures used to define sets, sequences, and instance types. (3) Extract relation information from the NELL-995 dataset and annotate the semantic attributes of each relation. and Represents any two relationship information extracted from the data set. Select one key term from the four key terms to define the two relationships and obtain an RDF triple, which is expressed as , the ontology graph is constructed based on the RDF triples composed of relational information and key terms extracted from the data set. The RDF triples represent the resource description framework triples, which are composed of subject, predicate and object. , the predicate corresponds to the key term, the object corresponds to the relationship ; in, The NELL-995 dataset is a subset of knowledge graphs automatically extracted from the Internet for building semantic ontology and supporting knowledge reasoning and relationship extraction. It contains 995 categories and their relationships. (4) The reasoning tool OWL Reasoner performs semantic reasoning and logical verification on the ontology graph. OWL Reasoner is a tool to ensure structural consistency and further optimize the ontology graph. S2.1.

2. Generate the initial node of ontology enhancement: The ontology graph is used as the prior knowledge of the initial node, and the ontology graph is pre-trained through the knowledge graph embedding technology TransE to obtain the representation of the relationship , and then project the relationship embedding onto the relationship subgraph through a two-layer fully connected network to generate the initial node of the ontology enhancement. The calculation process is as follows: ， in, Indicates the initial node of the ontology enhancement, the corner mark Indicates the initialization state; represents the weight matrix of the first layer of the fully connected network, represents the weight matrix of the second layer of the fully connected network, Represents the pre-trained Node representation of .

5. According to the inductive link prediction method based on relational message passing and mutual information maximization according to claim 4, S2 is specifically as follows: S2.2, Relationship Message Passing: Compute closed relation subgraphs via relation message passing The target relationship node in the target relationship node is specifically a two-step iterative aggregation target relationship node. To jump to a neighbor, the steps are as follows: (1) Taking the target relationship as the root node, select the one-hop neighbor of the target relationship node, then expand it layer by layer to the Kth layer, and use the relationship-based graph pruning strategy for optimization; The specific process is: starting from the target relationship node, gradually expand the neighborhood of the target relationship node to layer, and consider the direction of information transmission, sample all in-degree neighbors for each node, and finally obtain the target relationship node of Hop-in Neighbor , The specific expressions are as follows: ， ， in, Target relationship node No. Jump into the neighbor set, and , Indicates the target relationship node that the current layer is processing. express Neighbor relationship nodes, express and The type of adjacent edges between represents the set of all edges in the relation subgraph, represents a closed relation subgraph, Indicates the depth of message passing, ; (2) Starting from the final layer, the message passing mechanism iteratively aggregates the information of neighboring nodes layer by layer to generate the final target relation node embedding; Initially adopted The relational graph convolutional network framework assigns transformation matrices to different edge types, aggregates neighbor and self features layer by layer, and generates node representations that contain relational semantics; When operating on the data in the NELL-995 dataset, the method in step S2.1.2 is used to generate the initial node of ontology enhancement ,Otherwise, a random initialization method is used to randomly sample the representation of the initial node from a uniform distribution to provide a preliminary random feature representation for the training process; Messaging from Start at Aggregate the representation of neighbor nodes within the hop, update the node embedding layer by layer, and finally obtain the The complete embedding of the layer target relation nodes; The target relationship-aware attention mechanism is introduced to capture the neighboring nodes that are highly related to the target relationship. The GNN aggregation function of the layer is defined as follows: ， in, Representation Node The directed neighborhood of represents a nonlinear activation function, Indicates Layer Node The aggregation of the directed neighborhood embedding, Representation Node On edge type The neighbor set below, Indicates that the corresponding Layer Edge Type The transformation matrix, Indicated in Neighbors learned by layer iteration The expression, Indicates Layer on edge type Lower relationship node For target relationship nodes The importance of express -1 layer target relationship node Embedding Finally, the Layer Relationship Node With Neighborhood vector of layer aggregation Perform splicing to update the Potential of layer nodes , It is expressed as follows: 。 6. According to the inductive link prediction method based on relational message passing and mutual information maximization according to claim 5, S2 is specifically as follows: S2.3, One-hop neighbor aggregation: First, for the target relationship node The one-hop neighbors are sampled and the one-hop neighbors’ features are aggregated using the attention mechanism. , the calculation process is as follows: ， in, Represents the target relationship The set of one-hop neighbors of Indicates neighbor relationship Embedded, Represents the linear transformation matrix of each relationship node, represents the nonlinear activation function, attention weight Used to measure neighbor relationship nodes For target relationship nodes The importance of S2.

4. Complete subgraph embedding: Features learned in unclosed relation subgraphs are aggregated via one-hop neighbor aggregation Features learned through relational message passing in closed relational subgraphs Integrate, use summation and concatenation to form the final representation of the relationship subgraph , which is expressed as follows: in, represents the sum operation, Represents a concatenation operation.

7. According to the inductive link prediction method based on relational message passing and mutual information maximization according to claim 6, S3 is specifically as follows: S3.

1. Supervised Learning: According to the final representation of the relationship subgraph Calculate the target triple The likelihood score , the specific calculation is as follows: ， in, represents the relationship weight matrix; Then the supervised learning loss Optimize the likelihood score of the triples and replace the triples The head entity or tail entity in generates an equal number of negative sample triplets and positive samples, and ensures that the score of the positive sample triplets is significantly higher than that of the negative sample triplets. The calculation process is as follows: in, represents the number of all target triples input, and Respectively represent the target triples After replacement, we get negative triples and positive triples. represents the likelihood score of negative triples, represents the likelihood score of a positive triplet, represents the boundary hyperparameter, Indicates taking the maximum value, represents the index of the triple, , ; S3.2, mutual information maximization: By maximizing the local and global interaction information MI, we construct a global topological pattern between relationships, covering the subgraph and layer levels. First, we use the read function to obtain the global representation of the graph. The specific calculation is as follows: ， in, Indicates The embedding of the local relation subgraph of the positive triples, represents the embedding of the entire relationship graph, Represents the number of all positive sample triplets input; Then, a bilinear score function is introduced as a discriminator, and the Jensen-Shannon MI estimator is applied to estimate the mutual information between the relationship subgraph embedding and the entire relationship graph embedding, so as to maximize the mutual information between the two and determine whether they come from the same graph. The Jensen-Shannon MI estimator is a method for estimating the mutual information MI between two random variables. The bilinear score function is as follows: ， in, represents the activation function, represents the score matrix of mutual information MI, Represents the similarity between the embedding of the relation subgraph and the embedding of the entire relation graph; Through the corruption function Constructing the Negative Sampling Relationship Graph , the specific calculation is as follows: ， in, represents a closed relationship graph, represents the initial features of the relationship nodes in the closed relationship graph, Indicates that through The initial features of the damaged nodes are obtained by random perturbation layer by layer. express The adjacency matrix of Finally, through and Compute contrastive learning loss between positive and negative samples , and then realize the learning of graph embedding. The specific calculation is as follows: ， in, represents the number of negative sample triplets, represents the number of positive sample triplets, represents the embedding representation of the negative sample relation subgraph, Indicates The similarity between the embedding of the negative sample relation subgraph and the embedding of the entire relation graph; S3.3 Joint training objectives: The final loss is minimized by joint training. Contains supervised learning loss and contrastive learning loss , the calculation process is as follows: ， in, represents the hyperparameter used to adjust the ratio of supervised learning loss and contrastive learning loss.

Images