KGE-TF Method for Improving Knowledge Graph Embedding by Leveraging Type Fusion

Abstract

The present invention proposes a KGE-TF method for improving knowledge graph embedding by type fusion, which uses an entity type aggregator and a relational context type aggregator to incorporate type information into the entity embedding of the knowledge graph and the relational embedding of the knowledge graph; the method further includes a KGE-TF framework based on the KGE-TF model, and the framework forms a general knowledge representation learning framework by stipulating a unified loss function. The framework takes type information as a low-dimensional vector of any dimension, maps the type information into the entity embedding space and the relational embedding space for fusion with it, enriches the embedding representations of entities and relationships, and alleviates the data sparsity problem; the type information is additional fusion information and does not require each entity to have type information; the present invention can enrich the embedding representations of entities and relationships, alleviate the data sparsity problem, and the present invention also stipulates a unified loss function to form a general knowledge representation learning framework, so as to significantly improve the performance of the original model in the knowledge completion task.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge representation and reasoning under a knowledge graph, and in particular to the KGE-TF method for improving knowledge graph embedding by using type fusion. Background Art

[0002] The knowledge graph combines graph theory and natural language processing methods to provide an efficient implementation for the intelligent organization and representation of Freebase knowledge. The knowledge graph completion task is to represent entities and relationships in the knowledge graph as vectors in a low-dimensional space, and predict the missing head entity or tail entity in a triple, that is, given an incomplete triple or, it is required to complete the entity at the "?".

[0003] Introducing other auxiliary information into the knowledge graph is beneficial for entities and relationships to learn richer and more accurate embedding representations, such as type information, text information, image information, etc. Among them, compared with text information and image information, type information has the characteristics of simple processing, small storage space occupation, high information density, intuitive expression, etc. More importantly, type information can also directly reflect the attributes of objects and can conveniently provide a constraint function in the representation learning process. The graph neural network GNNs can make full use of neighborhood features to enrich the representation of the central entity.

[0004] Currently, most models considering type information use type information as an additional projection matrix or other constraints to enhance the embedding of entities by the projection matrix or fusing types, ignoring the role of type information in relationships. In fact, a relationship is a link connecting two entities, and the type constraint function mainly acts on the relationship. The relationship has an important discrimination ability for the types of the head entity and the tail entity. For example, given the relationship "sing", it is easy to conclude that the head entity of this relationship should include types such as "person" and "singer", rather than other types such as "plant" and "movie"; correspondingly, the tail entity should include types such as "work" and "song". A relationship with the information that "the head entity includes a singer and the tail entity includes a song" is also very easy to conclude that the relationship is "sing" from the perspective of human reasoning. It can also be understood that: as a connecting bridge, a relationship itself does not have any meaning, but the common features of all the head and tail entities connected to it endow the relationship with characteristics and constraint properties. Therefore, integrating the type embedding of the head entity and the type embedding of the tail entity into the relationship embedding can not only play a role in enriching the relationship embedding, but also play a constraint role. For example, given a predicted triple, since the relationship already has the type information of the head and tail entities, it can be biased towards entities whose entity types include "work", "song", etc. during the prediction process, thereby improving the accuracy of knowledge graph completion.

[0005] After the above analysis, the present invention believes that the problems to be solved by the previous combined type of knowledge representation learning model are as follows: (1) Regarding type information as an independent matrix or vector feature is vulnerable to data sparsity. (2) Only considering the influence of type information on entities and ignoring the influence of type information on relationships. (3) Being overly dependent on the dataset, requiring each entity to have prior knowledge such as type information and type hierarchy information, severely restricting the application scenarios of the model. (4) Most of them are single models and do not have the ability to cooperate with various current knowledge representation learning models.

[0006] Therefore, the present invention proposes a knowledge representation learning framework of Improved Knowledge Graph Embedding Using Type Fusion (KGE-TF) to address the problems and deficiencies mentioned above. Instead of using type information as a projection matrix for mapping entities or other non-vector features, KGE-TF uses type information as a low-dimensional vector of any dimension and maps the type information into the entity embedding space and the relationship embedding space to fuse with them, enriching the embedding representations of entities and relationships and alleviating the data sparsity problem. Since types are used as additional fusion information, it is not necessary for each entity to have a type, which expands the scope of application of the model. Finally, the present invention stipulates a unified loss function to form a general knowledge representation learning framework to significantly improve the performance of the original model in the knowledge completion task. Summary of the Invention

[0007] The present invention proposes a KGE-TF method for improving knowledge graph embedding using type fusion, which can use type information as a low-dimensional vector of any dimension and map the type information into the entity embedding space and the relationship embedding space to fuse with them, enriching the embedding representations of entities and relationships and alleviating the data sparsity problem. The present invention also stipulates a unified loss function to form a general knowledge representation learning framework to significantly improve the performance of the original model in the knowledge completion task.

[0008] The present invention adopts the following technical solutions.

[0009] The KGE-TF method for improving knowledge graph embedding using type fusion is used to introduce other auxiliary information in the knowledge graph to facilitate the entities and relationships to learn richer and more accurate embedding representations. The method uses an entity type aggregator and a relationship context type aggregator to incorporate type information into the entity embedding of the knowledge graph and the relationship embedding of the knowledge graph.

[0010] The method further includes a knowledge representation learning framework that uses type fusion to improve knowledge graph embedding, which is the KGE-TF framework based on the KGE-TF model. The framework forms a general knowledge representation learning framework by agreeing on a unified loss function. The framework treats type information as a low-dimensional vector in any dimension and maps the type information into the entity embedding space and the relationship embedding space to fuse with them, enriching the embedding representations of entities and relationships and alleviating the data sparsity problem. The type information is additional fusion information and does not require each entity to have type information.

[0011] The knowledge graph is represented as G = {E, R, T, C}, where E represents the set of entities in the knowledge graph, R represents the set of relationships in the knowledge graph, C represents the set of types in the knowledge graph, and T represents the set of triples in the knowledge graph. A triple is represented as (h, r, t) ∈ T, where h ∈ E represents the head entity, r ∈ R represents the relationship, and t ∈ E represents the tail entity.

[0012] When type information is introduced, each entity e ∈ E has its own set of types C e ∈ C, and each type is represented as c ∈ C.

[0013] The KGE-TF framework includes a type fusion module, a basic model, and a unified binary cross-entropy function. The type fusion module includes an entity type aggregator ETA and a relationship context type aggregator RCTA, which are used to incorporate type information into entity embeddings and relationship embeddings respectively to enhance the embedding representations of entities and relationships.

[0014] The basic model is a knowledge representation learning model that has been proposed.

[0015] The binary cross-entropy function provides a unified training strategy by agreeing on a loss function.

[0016] When training the KGE-TF model, the original knowledge graph embedding passes through the entity type aggregator and the relationship context type aggregator to obtain the entity embedding and relationship embedding fused with types respectively, and the final embedding is used as the input of the basic model for model training.

[0017] The design of the KGE-TF model includes an entity type aggregator, a relationship context type aggregator, and framework capability design.

[0018] In the knowledge graph, each entity has several types, and the direct combination of each type of information directly reflects the abstract information of the entity.

[0019] The entity type aggregator uses the average aggregation method to aggregate the type information of the entity to equally take care of each type of information of the entity, that is, given an entity e in the knowledge graph and the set of types C of e e={c1, c2,..., c k ,..., c n}, where n represents the number of entity e's types. Aggregate the types of e to obtain the type aggregation embedding c e , and the formula is as follows:

[0020]

[0021] where |C e | is the number of entity e's types;

[0022] The type information, as an abstract information, contains less information than entities and relationships. Therefore, the type is expressed as a vector with a lower dimension than entities and relationships, that is

[0023] The type and the entity are not in the same vector space. By designing a projection matrix project the aggregated type information into the vector space where the entity is located, and further extract the type-related features so as to fuse the type information into the entity embedding. The entity type information after projection is expressed as The type dimension can be freely set without being affected by the embedding dimensions of entities and relationships. The formula is

[0024]

[0025] Since e c is only the type aggregation feature of entity e. To enhance the embedding representation of entity e, incorporate e c into the embedding of e; Since it is intuitive to fuse the two kinds of information by addition and can largely retain the original features of the entity, therefore, fuse the original embedding of the entity and the type aggregation feature by directly adding them. The final representation of entity e after fusing the type is e', and the formula is as follows.

[0026] e' = e + e c Formula 3.

[0027] In a knowledge graph, the connection of a relationship with the head entity and the tail entity is called the relationship context; The relationship plays a connecting role in a certain triple. The head entity sets of the triples related to this relationship mostly have similar types and commonalities, and the same is true for the tail entity sets, and there are significant differences in the types of the head and tail entities; The relationship context type aggregator divides the relationship context type information into relationship-head type information and relationship-tail type information, and obtains the type aggregation feature of the relationship by aggregating them separately and combining them at the end; Given a certain relationship r in the knowledge graph, let the set of all head entities related to relationship r be denoted as E rh={e1,...,e k ,...,e n}, the set of all tail entities related to the relationship r is E rt ={e1,...,e k ,...,e m}, where n and m respectively represent the number of different head entities and tail entities connected to the relationship r; further, the set of all head entity types related to the relationship r is denoted as C rh ={c|c∈C e |e∈E rh}, the set of all tail entity types related to the relationship r is denoted as C rt ={c|c∈C e |e∈E rt}; for the relationship, the relationship-head type information and the relationship-tail type information are also aggregated respectively in an average aggregation manner, denoted as c rh and c rt respectively, as shown in Formula Four and Formula Five below;

[0028]

[0029]

[0030] At this time, it is necessary to effectively combine the relationship-head type and the relationship-tail type embeddings together as the relationship type information embedding, and five operations φ(r ch ,r ct ) can be selected for execution;

[0031] Additive fusion: φ add (r ch ,r ct ) = r ch +r ch ;

[0032] Additive mapping fusion: φ add_w (r ch ,r ct ) = W add (r ch +r ch ), where is a weight matrix designed to further extract the features after adding the head and tail types of the relationship.

[0033] Multiplicative fusion: , where represents element-wise multiplication.

[0034] Concatenation fusion: φ cat (r ch ,r ct ) = [rch ; r ct , where [·;·] represents the vector concatenation operation. Under this operation, it is necessary to make d r = 1 / 2d e .

[0035] Concatenation mapping fusion: φ cat_w (r ch , r ct ) = W cat [r ch ; r ct , is a projection matrix that maps the concatenated vector back to the entity embedding space;

[0036] After effectively fusing the relation - head type and relation - tail type through the φ operation, the aggregated type information of the relation is obtained, which is expressed by the formula:

[0037] r c = φ(r ch , r ct ) Formula VIII;

[0038] Finally, in order to enhance the embedding representation of the relation r, the type embedding r c of the relation r is incorporated into the original embedding r; the relation is obtained by addition, and the formula for the final representation of the relation r after fusing the type is as follows.

[0039] r' = r + r c Formula IX.

[0040] The framework ability design includes basic model adaptation, and also includes a unified training strategy and loss function; the training strategy uses a 1 - N scoring strategy. For each (h, r) in the batch, all entities in the knowledge graph are used as candidate tail entities t for scoring simultaneously;

[0041] The loss function uses the binary cross - entropy function BCE, taking the label and the probability of a certain sample as input to calculate the loss. The BCE loss function is as follows

[0042]

[0043] where i represents the i - th triple; y i represents the label of the i - th triple, with a value of 0 or 1, 1 for positive examples and 0 for negative examples; 0 < p i < 1 represents the probability of the i - th triple being valid, and N represents the number of all triples participating in the scoring; the goal of this loss function is to maximize the probability of positive - example triples being valid and minimize the probability of negative - example triples being valid;

[0044] The content adapted for the base model is as follows: the input of the BCE loss function is the probability of a certain sample being valid, while the input of the margin-based loss is the gap between the actual calculation result and the calculation result agreed upon by the scoring function; in order to enable it to be successfully transplanted into the KGE-TF framework and uniformly use the BCE loss function for model training optimization, it is necessary to make a simple modification to the scoring function without changing the original design concept of the model. The purpose of modifying the scoring function is to make the output of the scoring function be the probability of the validity of the triple;

[0045] After unifying the loss function to the BCE loss function and abandoning the negative sampling strategy, one form of the formula of the modified scoring function is

[0046] f(h,r,t) = σ(γ - ||h + r - t|| L1 / .L2 ) Formula XIII;

[0047] γ is still the margin parameter, and σ is the sigmoid activation function;

[0048] After being activated by σ, the output of the scoring function is a number between 0 and 1, which is used to represent the probability of the triple being valid. Therefore, it can be conveniently applied to the BCE loss function;

[0049] Another form of the formula of the modified scoring function is

[0050] f(h,r,t) = σ(h Τ diag(M r )t) Formula XVI.

[0051] The method is used for the FB15k and FB15k-237-T datasets of the knowledge graph; FB15k is a knowledge graph dataset constructed based on the Freebase dataset. Freebase is a multi-domain knowledge graph dataset that collects a large amount of entity, attribute, and relationship information; the FB15k-237 dataset is a more challenging dataset obtained from the FB15k dataset by removing many inverse relationships.

[0052] The method uses link prediction to perform the knowledge graph completion task, that is: given the head / tail entity and relationship, predict the corresponding tail / head entity, and achieve the purpose of knowledge graph completion by predicting a missing element in the triple.

[0053] In recent years, knowledge graph embedding combined with type information has gradually become the focus of research in the industry. Aiming at the problems existing in the existing methods considering type information, such as being easily affected by the data sparsity problem, ignoring the impact of type information on relation embedding, limited data application scenarios, and inability to form a framework, etc., the present invention proposes a method for improving knowledge graph embedding using type fusion (KGE-TF): by using the entity type aggregator and relation context type aggregator proposed in the invention, type information is incorporated into entity embedding and relation embedding. The present invention also provides a framework function, which can apply this method to various existing knowledge graph embedding models, significantly improving the performance of the original model.

[0054] The present invention proposes a knowledge representation learning framework for improving knowledge graph embedding using type fusion (Improved Knowledge Graph Embedding Using Type Fusion, KGE-TF), aiming to solve the problems and deficiencies mentioned above; the KGE-TF of the present invention does not use type information as a projection matrix for mapping entities or other non-vector features, but uses type information as a low-dimensional vector of any dimension, and maps the type information into the entity embedding space and relation embedding space to fuse with them, enriching the embedding representations of entities and relations and alleviating the data sparsity problem. Since type is used as additional fusion information, it is not necessary for each entity to have a type, which expands the scope of application of the model; finally, the present invention stipulates a unified loss function to form a general knowledge representation learning framework, so as to significantly improve the performance of the original model in the knowledge completion task. Specifically: 1. A framework method KGE-TF for improving knowledge graph embedding using type fusion is proposed.

[0055] 2. The algorithm proposed by the present invention proposes a type fusion mechanism, namely an entity type aggregator and a relation context type aggregator, which reasonably fuse entity types in an additional way into the embedding representations of entities and relations, improving the performance of the model.

[0056] 3. In order to facilitate the adaptation of various models to the KGE-TF framework, the present invention provides the idea and key points for modifying the scoring function.

[0057] The advantages of the present invention also lie in:

[0058] 1. Most of the existing knowledge completion models only use structural information or consider implicit types, which are easily affected by data sparsity and affect the embedding representation of entities. The present invention introduces explicit type information, making the model have richer information and achieving higher knowledge completion accuracy.

[0059] 2. Most of the existing models that introduce type information use types as the projection matrix of entities or as a non-vector representation for measuring type overlap, which fails to enrich entity embeddings and relationship embeddings and ignores the role of type information in relationship embeddings. The present invention designs an entity type aggregator and a relationship context type aggregator to enrich knowledge graph embeddings and alleviate the data sparsity problem by integrating type information into entities and relationships. In addition, since the present invention does not require each entity to have type information, it can be more easily applied to various data scenarios than previous work.

[0060] 3. Most of the existing models are proposed as single methods and have problems with difficult adaptation. The present invention designs a framework ability to apply various basic models into KGE-TF, greatly enhancing the performance of the basic models and having wide applicability. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The present invention will be further described in detail below with reference to the drawings and specific embodiments:

[0062] FIG. Figure 1 is a schematic diagram of the overall framework of KGE-TF of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0063] As shown in the figure, the KGE-TF method for improving knowledge graph embeddings using type fusion is used to introduce other auxiliary information in the knowledge graph to facilitate the learning of richer and more accurate embedding representations for entities and relationships. The method uses an entity type aggregator and a relationship context type aggregator to integrate type information into the entity embeddings of the knowledge graph and the relationship embeddings of the knowledge graph;

[0064] The method further includes a knowledge representation learning framework for improving knowledge graph embeddings using type fusion, which is a KGE-TF framework based on the KGE-TF model. The framework forms a general knowledge representation learning framework by stipulating a unified loss function. The framework takes type information as a low-dimensional vector of any dimension and maps the type information into the entity embedding space and the relationship embedding space to fuse with them, enriching the embedding representations of entities and relationships and alleviating the data sparsity problem; the type information is additional fusion information and does not require each entity to have type information.

[0065] The knowledge graph is represented as G = {E, R, T, C}, where E represents the set of entities in the knowledge graph, R represents the set of relationships in the knowledge graph, C represents the set of types in the knowledge graph, and T represents the set of triples in the knowledge graph. A triple is represented as (h, r, t) ∈ T, where h ∈ E represents the head entity, r ∈ R represents the relationship, and t ∈ E represents the tail entity;

[0066] When type information is introduced, each entity e ∈ E has its own type set C e∈C, each type is represented as c ∈ C.

[0067] The KGE-TF framework includes a type fusion module, a base model, and a unified binary cross-entropy function; the type fusion module includes an entity type aggregator ETA and a relation context type aggregator RCTA, which are used to incorporate type information into entity embeddings and relation embeddings respectively to enhance the embedding representations of entities and relations.

[0068] The base model is a knowledge representation learning model that has been proposed;

[0069] The binary cross-entropy function provides a unified training strategy by specifying a loss function;

[0070] When training the KGE-TF model, the original knowledge graph embeddings are passed through the entity type aggregator and the relation context type aggregator to obtain type-fused entity embeddings and relation embeddings respectively, and the final embeddings are used as the input to the base model for model training.

[0071] The design of the KGE-TF model includes an entity type aggregator, a relation context type aggregator, and framework capability design;

[0072] In the knowledge graph, each entity has several types, and the direct combination of each type of information directly reflects the abstract information of the entity.

[0073] The entity type aggregator uses an average aggregation method to aggregate the type information of an entity to equally consider each type of information of the entity, that is, given an entity e in the knowledge graph and the set of types C e ={c1, c2,..., c k ,..., c n} of entity e, where n represents the number of types of entity e. Aggregate the types of e to obtain the type-aggregated embedding c e of entity e, and the formula is as follows:

[0074]

[0075] where |C e | is the number of types of entity e;

[0076] As an abstract information, the type information contains less information than entities and relations. Therefore, the type is expressed as a vector with a lower dimension than entities and relations, that is,

[0077] The type and the entity are not in the same vector space. By designing a projection matrix Project the aggregated type information into the vector space where the entity is located, and further extract type-related features to fuse the type information into the entity embedding. The entity type information after projection is expressed as The type dimension can be freely set without being affected by the embedding dimensions of entities and relationships. The formula is

[0078] e c = W e c e Formula Two;

[0079] Since e c is only the type aggregation feature of entity e, to enhance the embedding representation of entity e, incorporate e c into the embedding of e; since it is intuitive to fuse the two types of information by addition and can largely retain the original features of the entity, therefore, the original embedding of the entity and the type aggregation feature are fused by direct addition. The final representation of entity e after fusing the type is e', and the formula is as follows.

[0080] e' = e + e c Formula Three.

[0081] In a knowledge graph, the connection of a relationship with the head entity and the tail entity is called the relationship context; the relationship plays a connecting role in a certain triple. The set of head entities of the triples related to this relationship mostly has similar types and commonalities, and the same is true for the set of tail entities, and there are significant differences in the types of the head and tail entities; the relationship context type aggregator divides the relationship context type information into relationship-head type information and relationship-tail type information, and obtains the type aggregation feature of the relationship by aggregating them separately and combining them at the end; given a certain relationship r in the knowledge graph, let the set of all head entities related to relationship r be denoted as E rh = {e1,..., e k ,..., e n}, and the set of all tail entities related to relationship r is E rt = {e1,..., e k ,..., e m}, where n and m respectively represent the numbers of different head entities and tail entities connected to relationship r; further, the set of all head entity types related to relationship r is denoted as C rh = {c|c ∈ C e |e ∈ E rh}, and the set of all tail entity types related to relationship r is denoted as C rt = {c|c ∈ C e |e ∈ E rt}; For relationships, the average aggregation method is also used to aggregate relationship-head type information and relationship-tail type information respectively, denoted as c rh and c rt , as shown in Formula 4 and Formula 5 below;

[0082]

[0083]

[0084] At this time, it is necessary to effectively combine the embeddings of the relationship-head type and the relationship-tail type as the relationship type information embedding. The following five operations φ(r ch , r ct ) can be executed;

[0085] Additive fusion: φ add (r ch , r ct ) = r ch + r ch ;

[0086] Additive mapping fusion: φ add_w (r ch , r ct ) = W add (r ch + r ch ), where is a weight matrix designed to further extract the features after adding the head and tail types of the relationship.

[0087] Multiplicative fusion: , where represents element-wise multiplication.

[0088] Concatenation fusion: φ cat (r ch , r ct ) = [r ch ; r ct , where [·; ·] represents the vector concatenation operation. Under this operation, it is necessary to make d r = 1 / 2d e .

[0089] Concatenation mapping fusion: φ cat_w (r ch , r ct ) = W cat [r ch ; r ct , is a projection matrix that maps the concatenated vector back to the entity embedding space;

[0090] After effectively fusing the relationship - head type and relationship - tail type through the φ operation, the aggregated type information of the relationship is obtained, which is expressed by the formula:

[0091] r c = φ(r ch , r ct ) Formula VIII;

[0092] Finally, in order to enhance the embedding representation of the relationship r, the type embedding r c of the relationship is incorporated into the original embedding r; the relationship is obtained by adding, and the formula for the final representation after fusing the type of the relationship r is as follows.

[0093] r' = r + r c Formula IX.

[0094] The framework capability design includes basic model adaptation, and also includes a unified training strategy and loss function; the training strategy uses a 1 - N scoring strategy, and for each (h, r) in the batch, all entities in the knowledge graph are used as candidate tail entities t for scoring at the same time;

[0095] The loss function adopts the binary cross - entropy function BCE, taking the label and the probability of occurrence of a certain sample as inputs to calculate the loss. The BCE loss function is as follows

[0096]

[0097] where i represents the i - th triple; y i represents the label of the i - th triple, with a value of 0 or 1, 1 for positive examples and 0 for negative examples; 0 < p i < 1 represents the probability of occurrence of the i - th triple, and N represents the number of all triples participating in the scoring; the goal of this loss function is to maximize the probability of occurrence of positive - example triples and minimize the probability of occurrence of negative - example triples;

[0098] The content of the basic model adaptation is: the input of the BCE loss function is the probability of occurrence of a certain sample, while the input of the margin - based loss is the gap between the actual calculation result and the calculation result agreed by the scoring function; in order to enable it to be successfully transplanted into the KGE - TF framework, the BCE loss function is uniformly used for model training optimization. Without changing the original design concept of the model, the scoring function needs to be simply modified. The purpose of modifying the scoring function is to make the output of the scoring function be the probability of occurrence of this triple;

[0099] After unifying the loss function as the BCE loss function and abandoning the negative sampling strategy, one of the formulas for the modified scoring function is

[0100] f(h, r, t) = σ(γ - ||h + r - t||L1 / .L2 ) Formula XIII;

[0101] γ is still the margin parameter, and σ is the sigmoid activation function;

[0102] After being activated by σ, the output of the scoring function is a number between 0 and 1, which is used to represent the probability of the triple being valid. Therefore, it can be conveniently applied to the BCE loss function;

[0103] Another form of the formula of the modified scoring function is

[0104] f(h, r, t) = σ(h Τ diag(M r )t) Formula XVI.

[0105] The method is used for the FB15k and FB15k-237-T datasets of the knowledge graph; FB15k is a knowledge graph dataset constructed based on the Freebase dataset. Freebase is a multi-domain knowledge graph dataset that collects a large amount of entity, attribute, and relationship information; the FB15k-237 dataset is a more challenging dataset obtained from the FB15k dataset by removing many inverse relationships.

[0106] The method uses link prediction to perform the knowledge graph completion task, that is: given the head / tail entity and relationship, predict the corresponding tail / head entity, and achieve the purpose of knowledge graph completion by predicting a missing element in the triple.

Claims

1. The KGE-TF method for improving knowledge graph embedding using type fusion, which is used to introduce other auxiliary information in the knowledge graph to facilitate the learning of more rich and accurate embedding representations of entities and relationships, is characterized in that: The method utilizes an entity type aggregator and a relation context type aggregator to incorporate type information into the entity embeddings and relation embeddings of the knowledge graph; The method further includes a knowledge representation learning framework that uses type fusion to improve the knowledge graph embedding, which is the KGE-TF framework based on the KGE-TF model. The framework forms a general knowledge representation learning framework by stipulating a unified loss function. The framework takes type information as a low-dimensional vector in any dimension and maps the type information into the entity embedding space and the relation embedding space to fuse with them, enriching the embedding representations of entities and relations and alleviating the data sparsity problem; the type information is additional fusion information and does not require each entity to have type information; The knowledge graph is represented as G = {E, R, T, C}, where E represents the set of entities in the knowledge graph, R represents the set of relations in the knowledge graph, C represents the set of types in the knowledge graph, and T represents the set of triples in the knowledge graph; a triple is represented as (h, r, t) ∈ T, where h ∈ E represents the head entity, r ∈ R represents the relation, and t ∈ E represents the tail entity; When the relation of the relation embedding in the knowledge graph is "sing", the head entities include types such as "person" and "singer"; When introducing type information, each entity e ∈ E has its own set of types C e ∈ C, and each type is represented as c ∈ C; The KGE-TF framework includes a type fusion module, a basic model, and a unified binary cross-entropy function; The type fusion module includes an entity type aggregator ETA and a relation context type aggregator RCTA, which are respectively used to incorporate type information into entity embeddings and relation embeddings to enhance the embedding representations of entities and relations; The basic model is a previously proposed knowledge representation learning model; The binary cross-entropy function provides a unified training strategy by stipulating a loss function; When training the KGE-TF model, the original knowledge graph embeddings are passed through the entity type aggregator and the relation context type aggregator to obtain entity embeddings and relation embeddings fused with types respectively, and the final embeddings are used as the input of the basic model for model training; The design of the KGE-TF model includes an entity type aggregator, a relation context type aggregator, and framework capacity design; In the knowledge graph, each entity has several types, and the direct combination of each type of information directly reflects the abstract information of the entity; The entity type aggregator adopts the average aggregation method to aggregate the type information of entities, so as to equally take into account each type information of the entity, that is, given an entity e in the knowledge graph and the type set C of e e ={c1, c2,..., c k ,..., c n}, where n represents the number of types of entity e; aggregate the types of e to obtain the type aggregation embedding c e , and the formula is as follows: Among them |C e | is the number of types of entity e; As an abstract information, the type information contains less information than entities and relationships. Therefore, the type is expressed as a vector with a lower dimension than entities and relationships, i.e., d c <d e ; The type and the entity are not in the same vector space. By designing a projection matrix project the aggregated type information into the vector space where the entity is located, and further extract type-related features to fuse the type information into the entity embedding; the entity type information after projection is expressed as The dimension of the type can be freely set without being affected by the embedding dimensions of the entity and the relationship. The formula is e c = W e c e Formula 2; Since e c is merely the type aggregation feature of entity e, to enhance the embedding representation of entity e, e c is incorporated into the embedding of e; since it is intuitive to fuse the two kinds of information by addition and can largely preserve the original features of the entity, therefore, the original embedding of the entity and the type aggregation feature are fused by direct addition, and the final representation of entity e after fusing the type is e', and the formula is shown as follows; e' = e + e c Formula III; The connection of a relation with the head entity and the tail entity is called the relation context; The relation context type aggregator divides the relation context type information into relation-head type information and relation-tail type information, and obtains the type aggregation feature of the relation by aggregating them separately and then combining them at the end; Given a certain relation r in a knowledge graph, let the set of all head entities related to relation r be denoted as E rh ={e1,...,e k ,...,e n}, and the set of all tail entities related to relation r is E rt ={e1,...,e k ,...,e m}, where n and m respectively represent the number of different head entities and tail entities connected to relation r; further, the set of all head entity types related to relation r is denoted as C rh ={c|c∈C e |e∈E rh}, and the set of all tail entity types related to relation r is denoted as C rt ={c|c∈C e |e∈E rt}; For relations, the relation-head type information and relation-tail type information are also aggregated respectively in an average aggregation manner, denoted as c rh and c rt respectively, as shown in Formula Four and Formula Five below; At this time, it is necessary to effectively combine the embeddings of the relation-head type and the relation-tail type as the relation type information embedding, and select any one of the following five operations φ(r ch , r ct ) to execute; Additive fusion: φ add (r ch ,r ct ) = r ch + r ch ; Additive mapping fusion: φ add_w (r ch , r ct ) = W add (r ch + r ch ), where W add ∈ d r × d r is a weight matrix designed to further extract the features after adding the head and tail types of the relationship; Multiply Fusion: φ mul (r ch ,r ct ) = r ch οr ct , where ο represents the dot product of elements; Concatenation fusion: φ cat (r ch ,r ct ) = [r ch ; r ct , where [·; ·] represents the vector concatenation operation. Under this operation, it is necessary to make d r = 1 / 2d e ; Stitching mapping fusion: φ cat_w (r ch ,r ct ) = W cat [r ch ;r ct , is a projection matrix that maps the stitched vector back into the entity embedding space; After effectively fusing the relation-head type and the relation-tail type through the φ operation, the aggregated type information of the relation is obtained, which is expressed by the formula: r c = φ(r ch , r ct ) Formula VIII; Finally, to enhance the embedding representation of relation r, the type embedding r of relation r c is incorporated into the original embedding r; the formula for obtaining the final representation of relation r after fusing the type through addition is as follows; r' = r + r c Formula Nine; The framework capacity design includes basic model adaptation, and also includes a unified training strategy and loss function; The training strategy uses a 1-N scoring strategy. For each (h, r) in the batch, all entities in the knowledge graph are used as candidate tail entities t for scoring at the same time; The loss function uses the binary cross-entropy function BCE, which takes the label and the probability of a certain sample as input to calculate the loss. The BCE loss function is as follows where i represents the i-th triple; y i represents the label of the i-th triple, with a value of 0 or 1, where the positive example is 1 and the negative example is 0; 0 < p i < 1 represents the probability of the i-th triple being valid, and N represents the number of all triples participating in the scoring; the goal of this loss function is to maximize the probability of positive example triples being valid and minimize the probability of negative example triples being valid; The content adapted for the base model is as follows: the input of the BCE loss function is the probability of a certain sample being valid, while the input of the margin-based loss is the difference between the actual calculation result and the calculation result agreed upon by the scoring function; in order to enable its smooth transplantation into the KGE-TF framework and uniformly use the BCE loss function for model training optimization, it is necessary to make a simple modification to the scoring function without changing the original design concept of the model. The purpose of modifying the scoring function is to make the output of the scoring function be the probability of the validity of the triple; After unifying the loss function to the BCE loss function and abandoning the negative sampling strategy, one form of the formula of the modified scoring function is f(h,r,t) = σ(γ - ||h + r - t|| L1 / L2 ) Formula XIII; γ is still the margin parameter, and σ is the sigmoid activation function; After being activated by σ, the output of the scoring function is a number between 0 and 1, which is used to represent the probability of the triple being valid. Therefore, it can be conveniently applied to the BCE loss function; Another form of the formula of the modified scoring function is f(h,r,t) = σ(h Τ diag(M r )t) Formula XVI.

2. The KGE-TF method for improving knowledge graph embedding by using type fusion according to claim 1, wherein: The method described above is used for the FB15k and FB15k-237-T datasets of the knowledge graph; FB15k is a knowledge graph dataset constructed based on the Freebase dataset. Freebase is a multi-domain knowledge graph dataset that collects a large amount of entity, attribute, and relationship information; The FB15k-237 dataset is a more challenging dataset obtained from the FB15k dataset by removing many inverse relationships.

3. The KGE-TF method for improving knowledge graph embedding by using type fusion according to claim 1, characterized in that: The method described above uses link prediction to perform the knowledge graph completion task, that is: given the head / tail entity and the relationship, predict the corresponding tail / head entity, and achieve the purpose of knowledge graph completion by predicting a missing element in the triple.

Images