Knowledge graph completion method based on type constraint and relation memory enhanced network

Abstract

A knowledge graph completion method based on type constraints and relational memory enhancement networks includes the following steps: constructing entity types and initializing features; constructing a dual-view collaborative representation learning framework; adaptive node information fusion and updating; and model relation decoding and training optimization. This invention integrates pre-generated entity type information into the node message passing stage by constructing entity types and initializing features, enabling the model to prune the candidate space using prior semantics, effectively reducing erroneous predictions caused by type mismatch. By constructing a dual-view framework, it collaboratively learns the fact propagation perspective and the relational interaction enhancement perspective. Furthermore, by leveraging multi-source information fusion and adaptive node aggregation mechanisms, it significantly improves the representation capabilities of entities and relations, achieving global interaction and knowledge transfer across relations, and enhancing the model's representation capabilities and prediction accuracy for complex structures.

Description

Technical Field

[0001] This invention relates to a knowledge graph completion method based on type constraints and relational memory enhancement networks, belonging to the fields of artificial intelligence and knowledge graph technology. Background Technology

[0002] Knowledge graph completion (KGC) aims to infer missing information in a knowledge graph to enhance the functionality of knowledge-driven applications. In recent years, graph neural networks (GNNs) have become the mainstream technological paradigm for solving KGC tasks due to their powerful ability to aggregate local neighborhood information. However, existing GNN-based methods still have significant limitations when processing highly complex heterogeneous knowledge graphs, mainly manifested in the following ways:

[0003] (1) Existing methods typically treat relationships as passive labels connecting entities, severely neglecting the higher-order structural interactions between relationships. Although some subsequent studies have begun to focus on relationship modeling and attempted to construct perspective graphs with both entities and relationships as nodes, these designs often limit interactions to binary associations of "entity-relationship" mediated by entities. Under this mechanism, the interactions between relationships are diluted, and the model is unable to learn the direct associations and higher-order dependencies between relationships;

[0004] (2) Existing models generally ignore the rich semantic constraints that explicit entity types can provide during feature learning. Since datasets often lack complete type annotations, existing methods usually rely solely on topological connections or external semantic information to construct node embeddings, thus ignoring the enormous potential of type-aware constraints. Furthermore, existing methods for constructing relationship graphs often strictly rely on entity co-occurrence. Since long-tail relationships (low-frequency relationships) rarely share entities with other relationships, they inevitably degenerate into isolated nodes in sparse interaction graphs, failing to effectively acquire neighbor context features, resulting in a severe lack of representational power. Summary of the Invention

[0005] The purpose of this invention is to provide a knowledge graph completion method based on type constraints and relational memory augmentation networks. This method can realize global interaction and knowledge transfer across relations, and improve the model's ability to represent complex structures and its prediction accuracy.

[0006] To achieve the above objectives, this invention provides a knowledge graph completion method based on type constraints and relational memory enhancement networks, comprising the following steps:

[0007] S1. Construct entity types and initialize features;

[0008] S2. Construct a dual-view collaborative representation learning framework;

[0009] S3, Adaptive node information fusion and updating;

[0010] S4. Model Relationship Decoding and Training Optimization.

[0011] Furthermore, the specific process of S1 is as follows:

[0012] S1.1 Collect three types of features of entities in the knowledge graph: entity name, entity relational context structure, and entity long text description;

[0013] S1.2, For entities The semantic embeddings of three types of feature text are extracted using the pre-trained language model BERT, and then concatenated and normalized to obtain a composite entity representation, as shown in the formula:

[0014] ;

[0015] ;

[0016] ;

[0017] ;

[0018] In the formula, This represents the corresponding BERT embedding vector;

[0019] S1.3. Use the HDBSCAN clustering algorithm to automatically cluster the composite entity representation, generate entity type labels, and convert them into entity type embeddings. The formula is as follows:

[0020] ;

[0021] ;

[0022] In the formula, Represents a set of entities. This represents the clustering results. Indicates the type embedding of the entity;

[0023] S1.4. The entity type embedding, the text semantic embedding extracted by BERT, and the original embedding are concatenated as the node initialization vector. The original embedding refers to the unique identifier feature vector assigned to each entity node in the knowledge graph. This vector is obtained by random initialization in the early stage of model training and is updated and optimized during the backpropagation process of the model. It is used to capture the independent identity information of the entity in the graph.

[0024] Furthermore, the specific process of S2 is as follows:

[0025] S2.1 Construct a fact propagation view containing only entity nodes based on the original triples, used to capture explicit semantic interactions and structural dependencies between entity pairs:

[0026] The fact propagation view is constructed based on the original set of triples G={(h,r,t)}, and contains only entities as graph nodes. If there is a relation r between two entities h and t, then an edge is established in the graph.

[0027] In the fact propagation view, relation-aware message aggregation is employed, and a combination operator is used to fuse entity vectors and relation vectors as a moderating factor to collect neighbor features of the target node. The following are representative combination operators, and the one with the best performance was selected through experiments:

[0028] ;

[0029] ;

[0030] ;

[0031] ;

[0032] In the formula, For combination operators, and represents entity and relation vectors respectively, and sub, mult, corr and cross represent subtraction, multiplication, cyclic correlation and cross interaction modifiers respectively;

[0033] S2.2 Construct a heterogeneous, relationship-enhanced view that breaks the limitation of "relationships only serving as edge labels," that is, construct a heterogeneous graph in which entities and relationships coexist, so as to explicitly model higher-order interactions between relationships;

[0034] Enhanced relational interaction views construct a scalable heterogeneous graph. , where the node set It is the union of the entity set and the relation set; in this view, entities and relations have equal status and can exchange information with each other; Represents the set of edges; the connection strategy between nodes is based on the structural co-occurrence constraint: that is, when two relations... and When they appear around the same entity 'e', ​​an edge is added between the two relations;

[0035] S2.3. Describe the structural features of each relation by constructing relation topological features: For a given relation, the topological feature vector is defined as the joint probability distribution of the entity types it connects to:

[0036] ;

[0037] Where k represents the number of entity types generated by clustering in S1.2. Representing the relationship The vector distribution of the head and tail entities;

[0038] Calculate the global similarity matrix between relations based on relational topological features. And calculate the relationship and Correlation between :

[0039] ;

[0040] ;

[0041] in, The eigenvectors representing all relations are... The topological feature matrix assembled from the combination is normalized to obtain... ; , Representing the relationship and The query vector and key vector, It is a learnable temperature parameter. These represent learnable weight parameters used to dynamically adjust the mode prior. Impact; Global Similarity Matrix The element in the i-th row and j-th column is , used to represent relations and Global pattern similarity across specific entity types;

[0042] Simultaneously, a memory-enhancing attention mechanism is introduced, treating all relations as a queryable memory bank to achieve semantic transfer across relations:

[0043] ;

[0044] ;

[0045] ;

[0046] in, This represents the enhanced embedding of the i-th relation node; It is the retrieval weight, representing the relevance between relation i and j; It is aggregated information obtained by weighted summation of entries in the memory based on retrieval weights; It is a dynamic gating vector; It is a learnable weight matrix.

[0047] Furthermore, the specific process of S3 is as follows:

[0048] S3.1 An adaptive node information aggregation network is constructed in each view, introducing learnable node-level scalar attention weights to adaptively and dynamically adjust the fusion ratio of neighbor context information and node intrinsic information; in the feature update of the l-th layer, the neighbor aggregation information is calculated as follows:

[0049] ;

[0050] in, For combination operators, It is a linear transformation matrix; Indicates adjacent nodes. This represents the relationship between the target node and its neighboring nodes; Represents the adjacency set of the target node o;

[0051] The node retains its own computation as follows:

[0052] ;

[0053] The final update is obtained through a node-level attention mechanism, using the following formula:

[0054] ;

[0055] ;

[0056] in, This represents information aggregated from the neighborhood. These are the retained items representing the node's own characteristics; and These are the fusion weights for the retained terms of neighborhood aggregation information and node-specific features, respectively. This is the attention weight matrix;

[0057] S3.2. The entity features output from the two views are proportionally fused using an adaptive gating mechanism, as shown in the formula:

[0058] ;

[0059] ;

[0060] in, Indicates the first The final entity feature representation of the layer, The output obtained from the solid view. The output obtained from the relational view; This represents the gating value, used to control the fusion ratio of information from two views; This is the entity view weight matrix; This is the weight matrix of the relational view; This is a bias term.

[0061] Furthermore, the specific process of S4 is as follows:

[0062] S4.1. Input the fused entity embedding and relation embedding into the relation decoder, and calculate the probability score of the triplet's validity using a scoring function. This process is expressed as:

[0063] ;

[0064] in, This represents the scoring function of the knowledge graph relation decoder, used to measure the performance of the head entity. ,relation Tail-end entity The reasonableness score of the constructed triples; this invention uses TransE, DistMult, and ConvE knowledge graph completion models as decoders, for a given triple The corresponding scoring functions are expressed as follows:

[0065] ;

[0066] ;

[0067] ;

[0068] in, This represents the convolution operation. It is a set of filters. It is a weight matrix;

[0069] S4.2. Train and optimize the model using the cross-entropy loss function with a label smoothing strategy:

[0070] In a knowledge graph, when the input triples When the triplet is valid, the model predicts its valid probability p to be close to 1; conversely, if the triplet is invalid, the model predicts its probability to be 0. Based on this objective, the loss function is defined as:

[0071] ;

[0072] ;

[0073] in: ;

[0074] In the formula, Represents a triplet Whether it is correct or not, its value is {0,1}; Indicates the effective probability; This represents the total number of tail nodes to be predicted; The set of triples representing correct information; The set of triples representing error messages; to improve the robustness and generalization ability of the model, a label smoothing strategy is used, under which... It becomes:

[0075] ;

[0076] in, This represents the smoothing factor.

[0077] This invention constructs entity types and initializes features, integrating pre-generated entity type information into the node message passing stage. This allows the model to prune the candidate space using prior semantics, effectively reducing erroneous predictions caused by type mismatch. By constructing a dual-view framework, it collaboratively learns the fact propagation perspective and the relationship interaction enhancement perspective. Furthermore, by leveraging multi-source information fusion and adaptive node aggregation mechanisms, it significantly improves the representation capabilities of entities and relationships. Through the relationship interaction enhancement view, relationships are elevated from edge labels to graph carrier nodes. Based on a memory-enhanced attention mechanism guided by topological profiling, it achieves global-level relationship interaction that goes beyond simple entity co-occurrence, enhancing the model's ability to capture complex multi-mapping relationships. This invention realizes global interaction and knowledge transfer across relationships, improving the model's representation capabilities and prediction accuracy for complex structures. Attached Figure Description

[0078] Figure 1 This is a flowchart of the present invention;

[0079] Figure 2 This is a structural diagram of the present invention;

[0080] Figure 3 This is a schematic diagram of the adaptive node information aggregation mechanism of the present invention. Detailed Implementation

[0081] The invention will now be further described with reference to the accompanying drawings.

[0082] like Figure 1 As shown, a knowledge graph completion method based on type constraints and relational memory augmentation networks includes the following steps:

[0083] S1. Construct entity types and initialize features;

[0084] S2. Construct a dual-view collaborative representation learning framework;

[0085] S3, Adaptive node information fusion and updating;

[0086] S4. Model Relationship Decoding and Training Optimization.

[0087] like Figure 2 As shown, the specific process of S1 is as follows:

[0088] S1.1 Collect three types of features of entities in the knowledge graph: entity name, entity relational context structure, and entity long text description;

[0089] S1.2, For entities The semantic embeddings of three types of feature text are extracted using the pre-trained language model BERT, and then concatenated and normalized to obtain a composite entity representation, as shown in the formula:

[0090] ;

[0091] ;

[0092] ;

[0093] ;

[0094] In the formula, This represents the corresponding BERT embedding vector;

[0095] S1.3. Use the HDBSCAN clustering algorithm to automatically cluster the composite entity representation, generate entity type labels, and convert them into entity type embeddings. The formula is as follows:

[0096] ;

[0097] ;

[0098] In the formula, Represents a set of entities. This represents the clustering results. Indicates the type embedding of the entity;

[0099] S1.4. The entity type embedding, the text semantic embedding extracted by BERT, and the original embedding are concatenated as the node initialization vector. The original embedding refers to the unique identifier feature vector assigned to each entity node in the knowledge graph. This vector is obtained by random initialization in the early stage of model training and is updated and optimized during the backpropagation process of the model. It is used to capture the independent identity information of the entity in the graph.

[0100] As a preferred embodiment, the specific process of S2 is as follows:

[0101] S2.1 Construct a fact propagation view containing only entity nodes based on the original triples, used to capture explicit semantic interactions and structural dependencies between entity pairs:

[0102] The fact propagation view is constructed based on the original set of triples G={(h,r,t)}, and contains only entities as graph nodes. If there is a relation r between two entities h and t, then an edge is established in the graph.

[0103] In the fact propagation view, relation-aware message aggregation is employed, and a combination operator is used to fuse entity vectors and relation vectors as a moderating factor to collect neighbor features of the target node. The following are representative combination operators, and the one with the best performance was selected through experiments:

[0104] ;

[0105] ;

[0106] ;

[0107] ;

[0108] In the formula, For combination operators, and represents entity and relation vectors respectively, and sub, mult, corr and cross represent subtraction, multiplication, cyclic correlation and cross interaction modifiers respectively;

[0109] S2.2 Construct a heterogeneous, relationship-enhanced view that breaks the limitation of "relationships only serving as edge labels," that is, construct a heterogeneous graph in which entities and relationships coexist, so as to explicitly model higher-order interactions between relationships;

[0110] Enhanced relational interaction views construct a scalable heterogeneous graph. , where the node set It is the union of the entity set and the relation set; in this view, entities and relations have equal status and can exchange information with each other; Represents the set of edges; the connection strategy between nodes is based on the structural co-occurrence constraint: that is, when two relations... and When they appear around the same entity 'e', ​​an edge is added between the two relations;

[0111] S2.3. Describe the structural features of each relation by constructing relation topological features: For a given relation, the topological feature vector is defined as the joint probability distribution of the entity types it connects to:

[0112] ;

[0113] Where k represents the number of entity types generated by clustering in S1.2. Representing the relationship The vector distribution of the head and tail entities;

[0114] Calculate the global similarity matrix between relations based on relational topological features. And calculate the relationship and Correlation between :

[0115] ;

[0116] ;

[0117] in, The eigenvectors representing all relations are... The topological feature matrix assembled from the combination is normalized to obtain... ; , Representing the relationship and The query vector and key vector, It is a learnable temperature parameter. These represent learnable weight parameters used to dynamically adjust the mode prior. Impact; Global Similarity Matrix The element in the i-th row and j-th column is , used to represent relations and Global pattern similarity across specific entity types;

[0118] Simultaneously, a memory-enhancing attention mechanism is introduced, treating all relations as a queryable memory bank to achieve semantic transfer across relations:

[0119] ;

[0120] ;

[0121] ;

[0122] in, This represents the enhanced embedding of the i-th relation node; It is the retrieval weight, representing the relevance between relation i and j; It is aggregated information obtained by weighted summation of entries in the memory based on retrieval weights; It is a dynamic gating vector; It is a learnable weight matrix.

[0123] like Figure 3 As shown, the specific process of S3 is as follows:

[0124] S3.1 An adaptive node information aggregation network is constructed in each view, introducing learnable node-level scalar attention weights to adaptively and dynamically adjust the fusion ratio of neighbor context information and node intrinsic information; in the feature update of the l-th layer, the neighbor aggregation information is calculated as follows:

[0125] ;

[0126] in, For combination operators, It is a linear transformation matrix; Indicates adjacent nodes. This represents the relationship between the target node and its neighboring nodes; Represents the adjacency set of the target node o;

[0127] The node retains its own computation as follows:

[0128] ;

[0129] The final update is obtained through a node-level attention mechanism, using the following formula:

[0130] ;

[0131] ;

[0132] in, This represents information aggregated from the neighborhood. These are the retained items representing the node's own characteristics; and These are the fusion weights for the retained terms of neighborhood aggregation information and node-specific features, respectively. This is the attention weight matrix;

[0133] S3.2. The entity features output from the two views are proportionally fused using an adaptive gating mechanism, as shown in the formula:

[0134] ;

[0135] ;

[0136] in, Indicates the first The final entity feature representation of the layer, The output obtained from the solid view. The output obtained from the relational view; This represents the gating value, used to control the fusion ratio of information from two views; This is the entity view weight matrix; This is the weight matrix of the relational view; This is a bias term.

[0137] As a preferred embodiment, the specific process of S4 is as follows:

[0138] S4.1. Input the fused entity embedding and relation embedding into the relation decoder, and calculate the probability score of the triplet's validity using a scoring function. This process is represented as follows:

[0139] ;

[0140] in, This represents the scoring function of the knowledge graph relation decoder, used to measure the performance of the head entity. ,relation Tail-end entity The reasonableness score of the constructed triples; this invention uses TransE, DistMult, and ConvE knowledge graph completion models as decoders, for a given triple The corresponding scoring functions are expressed as follows:

[0141] ;

[0142] ;

[0143] ;

[0144] in, This represents the convolution operation. It is a set of filters. It is a weight matrix;

[0145] S4.2. Train and optimize the model using the cross-entropy loss function with a label smoothing strategy:

[0146] In a knowledge graph, when the input triples When the triplet is valid, the model predicts its valid probability p to be close to 1; conversely, if the triplet is invalid, the model predicts its probability to be 0. Based on this objective, the loss function is defined as:

[0147] ;

[0148] ;

[0149] in: ;

[0150] In the formula, Represents a triplet Whether it is correct or not, its value is {0,1}; Indicates the effective probability; This represents the total number of tail nodes to be predicted; The set of triples representing correct information; The set of triples representing error messages; to improve the robustness and generalization ability of the model, a label smoothing strategy is used, under which... It becomes:

[0151] ;

[0152] in, This represents the smoothing factor, which is set to 0.1 in this embodiment.

[0153] This invention achieves extremely high prediction accuracy on the UMLS medical test set, with a Mean Reciprocal Rank (MRR) of 0.936, Hits@1 of 0.881, Hits@3 of 0.991, and Hits@10 of 0.997. Compared with existing baseline models, this method has significant advantages: its MRR score surpasses the second-best performing model, CompGCN (with an MRR of 0.918), by 1.96%, and it improves the Hits@1 core metric by 2.8% compared to the KBGAT model.

Claims

1. A knowledge graph completion method based on type constraint and relation memory enhanced network, characterized in that, Includes the following steps: S1. Construct entity types and initialize features; Its construction process includes: S1.1, collecting three types of features of entities in the knowledge graph: entity name, entity relational context structure, and entity long text description; S2. Construct a dual-view collaborative representation learning framework; S3, Adaptive node information fusion and updating; S4. Model Relationship Decoding and Training Optimization; The specific process of S2 is as follows: S2.1 Construct a fact propagation view containing only entity nodes based on the original triples, used to capture explicit semantic interactions and structural dependencies between entity pairs: The fact propagation view is constructed based on the original set of triples G={(h,r,t)}, and contains only entities as graph nodes. If there is a relation r between two entities h and t, then an edge is established in the graph. In the fact propagation view, relation-aware message aggregation is employed, and a combination operator is used to fuse entity vectors and relation vectors as a moderating factor to collect neighbor features of the target node; the following are representative combination operators: ； ； ； ； wherein is a combination operator, and denote entity and relation vectors, respectively, and sub, mult, corr, and cross denote subtraction, multiplication, cyclic correlation, and cross-interaction adjustment factors, respectively. S2.2 Construct a heterogeneous, relationship-enhanced view of relationships, that is, construct a heterogeneous graph in which entities and relationships coexist, so as to explicitly model the high-order interactions between relationships; Enhanced relational interaction views construct a scalable heterogeneous graph. , where the node set It is the union of the entity set and the relation set. In this view, entities and relations have equal status and can exchange information with each other. Represents the set of edges; the connection strategy between nodes is based on the structural co-occurrence constraint: that is, when two relations... and When they appear around the same entity 'e', ​​an edge is added between the two relations; S2.

3. Describe the structural features of each relation by constructing relation topological features: For a given relation, the topological feature vector is defined as the joint probability distribution of the entity types it connects to: ； Where k represents the number of entity types generated by clustering in S1.

2. Representing the relationship The vector distribution of the head and tail entities; Calculate the global similarity matrix between relations based on relational topological features. And calculate the relationship and Correlation between : ； ； in, The eigenvectors representing all relations are... The topological feature matrix assembled from the combination is normalized to obtain... ; , Representing the relationship and The query vector and key vector, It is a learnable temperature parameter. These represent learnable weight parameters used to dynamically adjust the mode prior. Impact; Global Similarity Matrix The element in the i-th row and j-th column is , used to represent relations and Global pattern similarity across specific entity types; Simultaneously, a memory-enhancing attention mechanism is introduced, treating all relations as a queryable memory bank to achieve semantic transfer across relations: ； ； ； in, This represents the enhanced embedding of the i-th relation node; This is the retrieval weight, representing the relevance between relation i and j; It is aggregated information obtained by weighted summation of entries in the memory based on retrieval weights; It is a dynamic gating vector; It is a learnable weight matrix.

2. The knowledge graph completion method based on type constraints and relational memory augmentation networks according to claim 1, characterized in that, The specific process of S1 also includes: S1.2, For entities The semantic embeddings of three types of feature text are extracted using the pre-trained language model BERT, and then concatenated and normalized to obtain a composite entity representation, as shown in the formula: ； ； ； ； In the formula, This represents the corresponding BERT embedding vector; S1.

3. Use the HDBSCAN clustering algorithm to automatically cluster the composite entity representation, generate entity type labels, and convert them into entity type embeddings. The formula is as follows: ； ； In the formula, Represents a set of entities. This represents the clustering results. Indicates the type embedding of the entity; S1.

4. The entity type embedding, the text semantic embedding extracted by BERT, and the original embedding are concatenated as the node initialization vector. The original embedding refers to the unique identifier feature vector assigned to each entity node in the knowledge graph. This vector is obtained by random initialization in the early stage of model training and is updated and optimized during the backpropagation process of the model. It is used to capture the independent identity information of the entity in the graph.

3. The knowledge graph completion method based on type constraints and relational memory augmentation networks according to claim 2, characterized in that, The specific process of S3 is as follows: S3.1 An adaptive node information aggregation network is constructed in each view, introducing learnable node-level scalar attention weights to adaptively and dynamically adjust the fusion ratio of neighbor context information and node intrinsic information; in the feature update of the l-th layer, the neighbor aggregation information is calculated as follows: ； in, For combination operators, It is a linear transformation matrix; Indicates adjacent nodes, This represents the relationship between the target node and its neighboring nodes; Represents the adjacency set of the target node o; The node retains its own computation as follows: ； The final update is obtained through a node-level attention mechanism, using the following formula: ； ； in, This represents information aggregated from the neighborhood. These are the retained items representing the node's own characteristics; and These are the fusion weights for neighborhood aggregation information and node-specific feature retention items, respectively; This is the attention weight matrix; S3.

2. The entity features output from the two views are proportionally fused using an adaptive gating mechanism, as shown in the formula: ； ； in, Indicates the first The final entity feature representation of the layer, The output obtained from the solid view. The output obtained from the relational view; This represents the gating value, used to control the fusion ratio of information from two views; This is the entity view weight matrix; This is the weight matrix of the relational view; This is a bias term.

4. The knowledge graph completion method based on type constraints and relational memory augmentation networks according to claim 3, characterized in that, The specific process of S4 is as follows: S4.

1. Input the fused entity embedding and relation embedding into the relation decoder, and calculate the probability score of the triplet's validity using a scoring function. This process is expressed as: ； in, This represents the scoring function of the knowledge graph relation decoder, used to measure the performance of the head entity. ,relation Tail-end entity The reasonableness score of the constructed triples; using TransE, DistMult, and ConvE knowledge graph completion models as decoders, for a given triple... The corresponding scoring functions are expressed as follows: ； ； ； in, This represents the convolution operation. It is a set of filters. It is a weight matrix; S4.

2. Train and optimize the model using the cross-entropy loss function with a label smoothing strategy: In a knowledge graph, when the input triples When the triplet is valid, the model predicts its valid probability p to be close to 1; conversely, if the triplet is invalid, the model predicts its probability to be 0. Based on this objective, the loss function is defined as: ； ； in: ; In the formula, Represents a triplet Whether it is correct or not, its value is {0,1}; Indicates the effective probability; This represents the total number of tail nodes to be predicted; The set of triples representing correct information; The set of triples representing error messages; to improve the robustness and generalization ability of the model, a label smoothing strategy is used, under which... It becomes: ； in, This represents the smoothing factor.

Images