Context-aware knowledge completion method and system based on unsupervised type constraints

Abstract

The present invention relates to a context-aware knowledge completion method and system based on unsupervised type constraints. The method constructs a context-aware model based on unsupervised type constraints. First, by setting the time granularity, the data set is preprocessed to make the data balanced in time distribution; secondly, in the quadruple structure module, the entity context information is aggregated through the neighbor encoder to enhance the embedding representation of the entity; in the quadruple type module, the quadruple is embedded in the real number space, the quadruple type module obtains the entity representation of the type constraint in an unsupervised environment, and further constrains the entity embedding representation through the entity potential type information to improve the model completion capability; finally, the score of the quadruple structure module and the score of the quadruple type module are aggregated according to the set weight to obtain the final score of the quadruple. The method and system improve the quality of entity embedding and improve the performance of the model in the knowledge completion task.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graphs, and specifically relates to a context-aware knowledge completion method and system based on unsupervised type constraints. Background Art

[0002] Knowledge representation learning is an effective and reliable knowledge completion technology. In recent years, representation learning methods for knowledge graphs have emerged in an endless stream. ComplEx embeds triples into complex space, which can model symmetric / antisymmetric and reflexive relationship models, and has achieved certain results in static knowledge graph completion tasks. AutoETER embeds quadruple into real space, automatically learns the type embedding of entities, and enriches the general characteristics of entities. In addition, it can also infer and model all three types of symmetric / antisymmetric, reflexive and combined relationship modes, as well as complex one-to-many, many-to-one and many-to-many relationships. CompGCN uses GCN to aggregate the neighbor information of entities and enhances the embedding representation of entities. The model also solves the problem of over-parameterization by setting basis vectors. The above models are all built on static knowledge graphs and perform well on static knowledge graph completion tasks, but because they do not consider time information, they perform poorly on temporal knowledge graph completion tasks.

[0003] In recent years, many works have tried to extend static knowledge graph completion models to temporal knowledge graphs. For example, HyTE defines a temporal hyperplane for each timestamp and projects entities and relations into the temporal hyperplane, then applies TransE to the projected embeddings to obtain the scores of the quadruple. et al. divided the timestamp into a token sequence and input it into LSTM together with the relationship to obtain the relationship representation at different times. The obtained relationship representation can be applied to multiple models. Jain et al. applied this method to ComplEx and obtained the TA-ComplEx model. Inspired by diachronic words, Goel et al. integrated time information into entity embedding and proposed the DE-SimplE model. DE-SimplE believes that there may be some time-varying features and some fixed features in entity embedding, so a hyperparameter γ∈[0,1] is set to control the percentage of time features, and significant results have been achieved in the task of temporal knowledge graph completion. TeRo defines the time evolution of entity embedding as the rotation of the entity from the initial time to the current time in the complex space. The model combines the advantages of RotatE and can model complex relationships between entities (such as reflexive relationships). In addition, TeRo uses time granularity to merge some timestamps, which can alleviate the problem of uneven distribution of data sets in time. Inspired by the fourth-order tensor decomposition, Lacroix et al. embedded the quaternion into the complex space and proposed the TNT-ComplEx model, which obtains the score of the quaternion through the inner product operation of the quaternion. The score function is defined as TIMEPLEX proposed by Jain et al. defines three types of time constraints based on the unique properties of time: the repeatability of relations, the order between relations, and the time interval between relations. Among them, the repeatability of relations means that many relations will not appear repeatedly for a specific entity (for example, a person is born only once), and some relations repeat in a fixed period (for example, the Olympics appear every four years). The order between relations means that for a given entity, one relation precedes another. For example, PersonBornYear should be before PersonDiedYear of a given entity. The time interval between relations means that for a given entity, the time difference between two relations is distributed around an average value, for example, the average value of PersonDiedYear minus PersonBornYear is about 70. TIMEPLEX uses Gaussian distribution to model these three time constraints without additional time constraint input, and shows good performance in the task of temporal knowledge graph completion. Although the above temporal knowledge graph completion models have achieved certain results in the task of temporal knowledge graph completion, they all ignore the potential information implied by the entity, such as type information and neighbor information, and have certain limitations in the task of temporal knowledge graph completion.

[0004] In the real world, uneven distribution of data in different time periods is a common phenomenon. Similarly, in existing time knowledge graphs, there is also a serious problem of data imbalance, such as YAGO11k and Wikidata12k (see below).Figure 1 As shown in the figure, the distribution of these two datasets in each time period is long-tailed, and the data imbalance problem is very serious. If trained directly, the model training is prone to underfitting in the time period with a large amount of data, and the prediction of such instances will become very difficult; in the time period with only a small number of instances, the model training is prone to overfitting, so the model training cannot reach a good balance point, resulting in poor results in the model completion task. In addition, YAGO11k and Wikidata12k also have a large number of timestamps missing, which is also a factor that reduces the model completion performance.

[0005] TeRo can alleviate the problem of uneven data distribution to a certain extent by setting the time granularity to merge timestamps with low frequency into one time. However, TeRo only uses the year information in the timestamp, which can easily lead to unreasonable merging. For example, for entity e, at timestamp t 1 =2005-01-05, t 2 =2005-12-12 and t 3 = 2006-02-01, we expect it to be 2 The embedding representation at t 3 The embedding representation is closer to t 1 The embedding representation should be quite different when t is used. For this case, TeRo’s method of using only year information is not satisfactory, because TeRo will 1 and t 2 Assuming the same time point (i.e. 2005), the entity at t 1 and t 2 In addition, some timestamps in YAGO11k contain complete year, month, and day information. Discarding the month and day information may lead to the loss of important information.

[0006] In the knowledge graph, in addition to the entity's own structural information, it also contains rich potential information, such as the entity's context information (neighborhood information) and type information. The entity's neighbor information is important information in the knowledge graph. The rational use of neighbor information can improve the quality of entity embedding and thus improve model performance. Figure 2 As shown in Figure 1, when predicting (A, Make a visit, ?, 1987), according to the neighbor information of entity A, A should have a greater probability of visiting B and less likely to visit C. Existing models that incorporate neighbor information include CompGCN, RE-Net, CyGNet, etc., which have shown good performance in completion tasks. Intuitively, the importance of entity neighbors should be inversely proportional to the temporal distance, such as Figure 2(A, Make a visit, C, 1902) should play a small role in predicting (A, Make a visit, ?, 1987). However, CompGCN is built on a static knowledge graph. When aggregating neighbor information, it assigns a uniform weight to (A, Make a visit, C, 1902) and (A, Threaten, C, 1987), causing the model to get the wrong answer. Although RE-Net and CyGNet are built on temporal knowledge graphs, they aim to infer unknown future knowledge from past knowledge, so I will not go into details.

[0007] The type information of an entity defines the category of the entity, which can significantly improve the entity embedding representation and improve the prediction accuracy. For an entity "Apple", if we only rely on the structural information of the entity itself, we cannot judge whether "Apple" is a fruit or Apple. But if the type information of the entity indicates that it is a fruit, then we can clearly know that "Apple" refers to apples. Some existing models that use entity type information, such as TKRL and TransT, require explicit entity type input, which has certain limitations for the completion task of knowledge graphs that do not explicitly provide entity type information. AutoETER can automatically encode entity type information, but it is built on a static knowledge graph and ignores the dynamic nature of facts. The type information of an entity should be affected by both relations and time. For example, for the entity "Lu Xun", when the connected relationship is "writing", his entity type is "writer". But Lu Xun gave up medicine and turned to literature in 1906, so his entity type could not be "writer" before 1906, that is, before 1906, the four-tuple formed by Lu Xun and the relationship "writing" was wrong. Summary of the invention

[0008] The purpose of the present invention is to provide a context-aware knowledge completion method and system based on unsupervised type constraints, which improves the quality of entity embedding and enhances the performance of the model in knowledge completion tasks.

[0009] To achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a context-aware knowledge completion method based on unsupervised type constraints, constructing a context-aware model based on unsupervised type constraints including an adaptive time granularity aggregation module, a quadruple structure module and a quadruple type module. First, by setting the time granularity, the data set is preprocessed to make the data balanced in time distribution; secondly, in the quadruple structure module, the entity context information is aggregated through a neighbor encoder to enhance the embedded representation of the entity; in the quadruple type module, the quadruple is embedded into the real number space, and the quadruple type module obtains a type-constrained entity representation in an unsupervised environment, further constrains the entity embedding representation through the entity's potential type information, and improves the model completion capability; finally, the score of the quadruple structure module and the score of the quadruple type module are aggregated according to the set weight to obtain the final score of the quadruple.

[0010] Furthermore, the context-aware model sets a hyperparameter: time granularity thre. Through the time granularity, timestamps with low occurrence frequency are merged into one timestamp, while timestamps with high occurrence frequency form separate timestamps, so that the data is as balanced as possible in time distribution; the workflow of the adaptive time granularity aggregation module is as follows:

[0011] First, calculate the number of quadruplets num[t], t at each timestamp in the data set; second, sort the timestamps in chronological order; then, merge the timestamps according to the time granularity thre. When the following conditions are met, t 1 , t 2 ,……,t i are merged into the same time τ:

[0012]

[0013] After the above steps, the quaternion (s, r, o, t) becomes (s, r, o, τ), (s, r, o, T = [t b ,t e ]) becomes (s,r,o,T=[τ b ,τ e ]); for (s,r,o,T=[τ b ,τ e ]), and map it into a four-tuple in the form of time points through enumeration operations.

[0014] Furthermore, the four-tuple (s, r, o, τ) processed by the adaptive time granularity aggregation module is input into the four-tuple structure module. After the initialization embedding module, the initial vector representation e of the head entity s, tail entity o, relation r and time τ in the complex space is obtained. s , e o , r, In order to capture the neighborhood information of the entity, e s , e o , r, τ is input into the neighbor encoder to obtain the enhanced representation e after the head and tail entity aggregated neighbors s , e o ; Then the four-tuple structure score f is obtained through the four-tuple scoring module stru .

[0015] Furthermore, the neighbor encoder includes three parts: (1) neighbor sampling based on entity semantic matching: selecting a fixed number of neighbors from the entity neighbor set; (2) neighbor aggregator: aggregating the sampled neighbors in a certain way to obtain the neighbor representation e of entity e nei ; (3) Fusion: The entity e’s own structure is represented by e e With neighbors expressed nei Fusion, get the final entity representation e′ e .

[0016] Furthermore, the neighbor sampling based on entity semantic matching is implemented as follows:

[0017] First, preprocess the entity names to remove pause words and special symbols contained in the entity names;

[0018] Then, use the word2vec pre-trained word vector to encode the entity name and embed the entity name into name e Defined as:

[0019]

[0020] Among them word i represents the embedding of the i-th word in the entity name after word2vec pre-training, and K represents the number of words in the entity name;

[0021] Finally, the cosine similarity between the name embedding of entity e and the name embedding of its neighbor entities is calculated, and the top n neighbor entities with the highest similarity to entity e are selected to obtain the final entity neighbor set N e ={(r 0 ,e 0 ,τ 0 ),(r 2 ,e 2 ,τ 2 ), ...}, |N e |=n;

[0022] The neighbor aggregator is implemented as follows:

[0023] The neighbor aggregator uses a LSTM-based neighbor aggregator and an attention mechanism-based neighbor aggregator to aggregate the neighbor information of the entity;

[0024] In the LSTM-based neighbor aggregator, N e The entities in are input into LSTM in chronological order, and the neighbor representation e of entity e is obtained. nei :

[0025] e nei =LSTM(x), x = {e 2 , e 0 , ...}

[0026] where {e 2 ,e 0 ,……} is the neighbor entity embedding set sorted in chronological order, |x|=n;

[0027] In the attention-based neighbor aggregator, the importance of each neighbor to entity e is calculated:

[0028]

[0029]

[0030] Where N e is the set of neighbors of entity e, e e is the embedding representation of entity e, e e [k] means e e The kth element of Yes i The conjugate representation of , Re(·) represents the real part;

[0031] After obtaining the weight of each neighbor according to the above formula, the neighbors are aggregated according to the weight, and finally the entity neighbor representation e is obtained nei :

[0032]

[0033] The fusion is achieved as follows:

[0034] Get the neighbor representation of the entity e nei After that, it is represented by the entity itself e Fusion, including additive fusion, multiplicative fusion and gated fusion;

[0035] The additive fusion simply adds the entity's own structural representation and the neighbor representation, and the final entity representation is defined as:

[0036]

[0037] The multiplication fusion multiplies the entity's own structural representation with the neighbor representation, and the final entity representation is defined as:

[0038]

[0039] The gated fusion filters out important information for the entity, and the final representation of the entity is defined as:

[0040]

[0041] where γ∈[0,1] is the gating factor.

[0042] Furthermore, the neighbor encoder is used to obtain the enhanced representation e′ of the head and tail entities. s and e′ o After that, a scoring function is used to evaluate the probability that the quadruple (s, r, o, τ) is established; the relationship in the quadruple exhibits a unique time constraint, which includes (a) the repetitiveness of the relationship, (b) the order between the relationships, and (c) the time interval between the relationships; in order to capture the time constraint, a scoring function is used to score the quadruple, and the scoring function is defined as follows:

[0043] f stru (s,r,o,τ)=TIMEPLEX(s,r,o,τ)

[0044] =f TX (s,r,o,τ)+κf Pair (s,r,o,τ)+λf Rec (s,r,o,τ)

[0045]

[0046] Among them, f TX (s,r,o,τ) is the score of the four-tuple (s,r,o,τ), r sτ , is an embedding representation specific to relation r; κ, λ, a, b, c are all hyperparameters; f Rec (s,r,o,τ) is the score of the time constraint (a), f Pair (s, r, o, τ) are the scores of the temporal constraint (b) and the time constraint (c).

[0047] Furthermore, in the quaternary type module, the quaternary (s, r, o, τ) is embedded into the real number space to obtain the initial embedding c of the head entity s, tail entity o, relation r and time τ in the type module. s , c o , c r , The score of the quadruple about the entity type is defined as:

[0048]

[0049] Among them, η, μ are weight factors, c s [k] means c s The kth element of .

[0050] Furthermore, the score f of the four-tuple structure module is stru (s, r, o, τ) and the score f of the four-tuple type module type (s, r, o, τ) are added together according to the set weights as the final score of the four-tuple (s, r, o, τ), which is specifically defined as follows:

[0051] f final (s,r,o,τ)=af stru (s,r,o,τ)+βf type (s,r,o,τ)

[0052] Among them, α and β are weight factors.

[0053] Furthermore, the context-aware model is trained using a log-likelihood loss function, which is defined as follows:

[0054]

[0055] Among them, G is the set of correct quadruplets; Pr(o|s,r,τ) is the probability of entity o when predicting the tail entity, and the calculation formula is as follows:

[0056]

[0057] in, is a set of four-tuples after random replacement of tail entities and filtering operations;

[0058] Similarly, we obtain Pr(s|o,r,τ) and Pr(τ|s,r,o).

[0059] The present invention also provides a context-aware knowledge completion system based on unsupervised type constraints, including a memory, a processor, and computer program instructions stored in the memory and executable by the processor. When the processor executes the computer program instructions, the above method steps can be implemented.

[0060] Compared with the prior art, the present invention has the following beneficial effects:

[0061] 1. The existing knowledge representation learning method using time granularity only uses the year information in the timestamp, which easily leads to unreasonable timestamp merging, and discarding the month and day information easily causes the loss of important information. However, the present invention makes full use of the complete information in the timestamp, and the model can merge the timestamp in a more fine-grained manner, and the merging method is more reasonable; in addition, more accurate time information also enables the model to achieve more accurate predictions.

[0062] 2. Compared with the existing knowledge representation learning model that integrates neighbor information, the present invention takes into account the time information of entity neighbors. When aggregating neighbor information, it can filter out important information based on the temporal distance of neighbors, thereby capturing effective information for the model.

[0063] 3. Existing models that use entity type information are all built on static knowledge graphs, ignoring the dynamic nature of facts. However, the present invention can learn the embedding of entities under type constraints under specific relationships and time, and does not require explicit type input, which improves the universality of the model. In addition, the present invention simultaneously considers the neighbor information and type information of the entity, which complement each other and significantly improves the quality of entity embedding and the performance of the model in knowledge completion tasks. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 It is the data distribution diagram of YAGO11k and Wikidata12k in different time periods in the prior art.

[0065] Figure 2 It is an entity neighbor information graph in the prior art.

[0066] Figure 3 It is an overall framework diagram of a context-aware model based on unsupervised type constraints in an embodiment of the present invention.

[0067] Figure 4 2 is a schematic diagram of the structure of a neighbor encoder according to an embodiment of the present invention.

[0068] Figure 5 It is a schematic diagram of the fusion method in an embodiment of the present invention. DETAILED DESCRIPTION

[0069] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0070] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0071] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0072] The present embodiment provides a context-aware knowledge completion method based on unsupervised type constraints, and constructs a context-aware model for unsupervised type constraints (CAUT) based on unsupervised type constraints, which includes an adaptive time granularity aggregation module, a quadruple structure module, and a quadruple type module: first, by setting the time granularity, the data set is preprocessed to make the data as balanced as possible in time distribution, so as to solve the problem caused by the long-tail distribution of the data to the model; secondly, in the quadruple structure module, the entity context information is aggregated through the neighbor encoder to enhance the embedding representation of the entity; in the quadruple type module, the quadruple is embedded into the real number space, and the quadruple type module obtains the entity representation of the type constraint in an unsupervised environment, and further constrains the entity embedding representation through the entity potential type information, so as to improve the model completion capability; finally, the score of the quadruple structure module and the score of the quadruple type module are aggregated according to the set weight to obtain the final score of the quadruple.

[0073] The main ideas of the CAUT model are: First, to address the problem that some data sets have a long-tail distribution in time, which leads to poor model performance, we use time granularity to merge timestamps to make the model distribution as balanced as possible. Second, we use the entity's potential neighbor information and type information to enhance the entity's embedding representation, thereby improving the model's completion performance.

[0074] The overall structure of the CAUT model is as follows: Figure 3 First, in order to solve the problem of poor model performance caused by uneven temporal distribution of some data sets, the present invention introduces an adaptive temporal granularity aggregation module; secondly, in order to make full use of entity neighbor information, the neighbor encoder is used in the quadruple structure module to obtain the enhanced representation of the head and tail entity after aggregating neighbors. s , e o In addition, the present invention also introduces a quadruple type module to obtain the quadruple score f under type constraints without additional type input. type Finally, the score f of the quadruple structure module is stru The score f of the quadruple type module type According to a certain weight aggregation, the final score f of the quadruple is obtainedfinal .

[0075] The relevant definitions of the present invention are given below.

[0076] Definition 3-1 (Temporal Knowledge Graph, G) Where E is the set of entities, R is the set of relations, is a set of timestamps, and F is a set of knowledge.

[0077] Definition 3-2 (quaternion) A quaternion is (s, r, o, t) and (s, r, o, T = [t b ,t e ]) There are two forms. Among them, s,o∈E represents the head entity and the tail entity, r∈R represents the relationship, t is a timestamp such as "2021-11-12", T represents the time interval, and t b , t e are the start time and the end time respectively. A quadruple is also called a fact or a piece of knowledge.

[0078] Definition 3-3 (Entity Embedding Representation) s , represents the embedding of the head entity s and the tail entity o in the complex space, d c is the embedding dimension of the complex space. c s , represents the embedding of the head entity s and the tail entity o in the real space, and d is the embedding dimension of the real space.

[0079] Definition 3-4 (Relation Embedding Representation) The present invention relates to two embedding representations: the embedding representation of relation r in complex space Embedding representation of relation r in real space

[0080] Definition 3-5 (Time Embedding Representation) The time τ obtained after the timestamp t is processed by the time granularity aggregation module involves two embedding representations: represents the embedding representation of time τ in complex space, Represents the embedded representation of time τ in the real space.

[0081] 1. Adaptive time granularity aggregation module

[0082] In view of the uneven distribution of some data sets in time, the present invention sets a hyperparameter for the context-aware model: time granularity thre. Through time granularity, timestamps with low frequency of occurrence are merged into one timestamp, while timestamps with high frequency of occurrence form separate timestamps, so that the data is as evenly distributed as possible in time; the workflow of the adaptive time granularity aggregation module is as follows:

[0083] First, calculate the number of quadruplets num[t], t at each timestamp in the data set; second, sort the timestamps in chronological order; then, merge the timestamps according to the time granularity thre. When the following conditions are met, t 1 , t 2 ,……,t i are merged into the same time τ:

[0084]

[0085] After the above steps, the quaternion (s, r, o, t) becomes (s, r, o, τ), (s, r, o, T = [t b ,t e ]) becomes (s,r,o,T=[τ b ,τ e ]); for (s,r,o,T=[τ b ,τ e ]), and through enumeration operations, it is mapped into a four-tuple in the form of time points. For example, (s, r, o, T = [τ 0 ,τ 3 ]) is mapped to (s,r,o,τ 0 ), (s,r,o,τ 1 ), (s,r,o,τ 2 ) and (s,r,o,τ 3 ).

[0086] 2. Quadruple structure module

[0087] The four-tuple (s, r, o, τ) processed by the adaptive time granularity aggregation module is input into the four-tuple structure module. After the initialization embedding module, the initial vector representation e of the head entity s, the tail entity o, the relation r and the time τ in the complex space is obtained. s , e o , r, In order to capture the neighborhood information of the entity, e s , e o , r, τ are input into the neighbor encoder to obtain the enhanced representation e after the head and tail entity aggregated neighbors s , e o ; Then the four-tuple structure score f is obtained through the four-tuple scoring module stru .

[0088] 1) Neighborhood Encoder

[0089] like Figure 4As shown in the figure, the neighbor encoder can be divided into three parts: (1) Neighbor sampling based on entity semantic matching: selecting a fixed number of neighbors from the entity neighbor set; (2) Neighbor aggregator: aggregating the sampled neighbors in a certain way to obtain the neighbor representation e of entity e nei ; (3) Fusion: The entity e’s own structure is represented by e e With neighbors expressed nei Fusion, get the final entity representation e′ e .

[0090] (1) Neighbor Sampling Based on Entity Semantic Matching

[0091] The neighbor sampling based on entity semantic matching is implemented as follows:

[0092] First, the entity names are preprocessed to remove pause words and special symbols such as “(”, “!”, and “-” contained in the entity names.

[0093] Then, we use the word2vec pre-trained word vector to encode the entity name. Since the entity name may consist of multiple words, we embed the entity name into name. e Defined as:

[0094]

[0095] Among them word i represents the embedding of the i-th word in the entity name after word2vec pre-training, and K represents the number of words in the entity name.

[0096] Finally, the cosine similarity between the name embedding of entity e and the name embedding of its neighbor entities is calculated, and the top n neighbor entities with the highest similarity to entity e are selected to obtain the final entity neighbor set N e ={(r 0 ,e 0 ,τ 0 ),(r 2 ,e 2 ,τ 2 ), ...}, |N e |=n.

[0097] (2) Neighborhood Aggregator

[0098] The present invention adopts two time aggregators to aggregate the neighbor information of entities: a neighbor aggregator based on LSTM and a neighbor aggregator based on attention mechanism.

[0099] ① LSTM-based neighbor aggregator

[0100] In view of the good performance of LSTM in processing long sequence data, the present invention uses Ne The entities in are input into LSTM in chronological order, and the neighbor representation e of entity e is obtained. nei :

[0101] e nei =LSTM(x), x = {e 2 , e 0 , ...}

[0102] where {e 2 ,e 0 ,……} is the neighbor entity embedding set sorted in chronological order, |x|=n.

[0103] ②Neighborhood aggregator based on attention mechanism

[0104] Considering that each neighbor has different importance to entity e, if it is aggregated without distinction, it is easy to introduce useless information, resulting in a decrease in model performance. Therefore, the present invention introduces an attention mechanism to weighted aggregate neighbor information.

[0105] First, calculate the importance of each neighbor to entity e:

[0106]

[0107]

[0108] Where N e is the set of neighbors of entity e, e e is the embedding representation of entity e, e e [k] means e e The kth element of Yes i The conjugate representation of , Re(·) represents the real part.

[0109] After obtaining the weight of each neighbor according to the above formula, the neighbors are aggregated according to the weight, and finally the entity neighbor representation e is obtained nei :

[0110]

[0111] (3) Integration

[0112] Get the neighbor representation of the entity e nei After that, it is represented by the entity itself e Fusion includes three fusion methods: additive fusion, multiplicative fusion and gated fusion.

[0113] The additive fusion simply adds the entity's own structural representation and the neighbor representation, such as Figure 5 As shown in (a), the final entity representation is defined as:

[0114]

[0115] The multiplication fusion multiplies the entity's own structure representation with the neighbor representation, such as Figure 5 As shown in (b), the final entity representation is defined as:

[0116]

[0117] The above two methods treat entity structure information and neighbor information equally, which easily introduces invalid information in neighbor information and reduces model performance. Therefore, the present invention adopts a gating mechanism to filter out important information for entities, such as Figure 5 (c) The final entity representation is defined as:

[0118]

[0119] where γ∈[0,1] is the gating factor.

[0120] 2) Quadruple scoring

[0121] The enhanced representation e′ of the head and tail entities is obtained by using the neighbor encoder s and e′ o After that, a scoring function is used to evaluate the probability of the four-tuple (s, r, o, τ) being true; the relations in the four-tuple exhibit unique time constraints, such as (a) the repetitiveness of the relations, (b) the order between the relations, and (c) the time interval between the relations. In order to capture the time constraints, the present invention uses a scoring function to score the four-tuple, and the scoring function is defined as follows:

[0122] f stru (s,r,o,τ)=TIMEPLEX(s,r,o,τ)

[0123] =f TX (s,r,o,τ)+κf Pair (s,r,o,τ)+λf Rec (s,r,o,τ)

[0124]

[0125] Among them, f TX (s,r,o,τ) is the score of the four-tuple (s,r,o,τ), r sτ , is an embedding representation specific to relation r; κ, λ, a, b, c are all hyperparameters; f Rec (s,r,o,τ) is the score of the time constraint (a), f Pair(s, r, o, τ) are the scores of the temporal constraint (b) and the time constraint (c).

[0126] 3. Quadruple type module

[0127] For the quaternion (s, r, o, τ), the goal of this invention is to learn the embedding of entities under type constraints under specific relations and time. The quaternion (s, r, o, τ) is embedded into the real number space to obtain the initial embedding c of the head entity s, the tail entity o, the relation r and the time τ in the type module. s , c o , c r , Considering that some entity types are also affected by time, the present invention defines the score of the quadruple with respect to the entity type as:

[0128]

[0129] Among them, η, μ are weight factors, c s [k] means c s The kth element of .

[0130] 4. Final model scoring

[0131] The score f of the four-tuple structure module stru (s, r, o, τ) and the score f of the four-tuple type module type (s, r, o, τ) are added together according to the set weights as the final score of the four-tuple (s, r, o, τ), which is specifically defined as follows:

[0132] f final (s,r,o,τ)=αf stru (s,r,o,τ)+βf type (s,r,o,τ)

[0133] Among them, α and β are weight factors.

[0134] 5. Model Optimization

[0135] The present invention uses a log-likelihood loss function to train the context-aware model, and the loss function is defined as follows:

[0136]

[0137] Among them, G is the set of correct quadruplets; Pr(o|s,r,τ) is the probability of entity o when predicting the tail entity, and the calculation formula is as follows:

[0138]

[0139] in, is a set of four-tuples after random replacement of tail entities and filtering operations;

[0140] Similarly, we obtain Pr(s|o,r,τ) and Pr(τ|s,r,o).

[0141] The present invention also provides a context-aware knowledge completion system for implementing the above method, including a memory, a processor, and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the above method steps can be implemented.

[0142] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0143] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A context-aware knowledge completion method based on unsupervised type constraints, characterized in that: A context-aware model based on unsupervised type constraints is constructed, which includes an adaptive time granularity aggregation module, a quadruple structure module, and a quadruple type module. First, the dataset is preprocessed by setting the time granularity to balance the data distribution in time. Second, in the quadruple structure module, the entity context information is aggregated through the neighbor encoder to enhance the embedding representation of the entity. In the quadruple type module, the quadruple is embedded into the real number space. The quadruple type module obtains the entity representation with type constraints in an unsupervised environment, and further constrains the entity embedding representation through the entity potential type information to improve the model completion capability. Finally, the score of the quadruple structure module and the score of the quadruple type module are aggregated according to the set weight to obtain the final score of the quadruple. The context-aware model sets a hyperparameter: time granularity thre. Through the time granularity, timestamps with low frequency of occurrence are merged into one timestamp, while timestamps with high frequency of occurrence form separate timestamps, so that the data is as balanced as possible in time distribution; the workflow of the adaptive time granularity aggregation module is as follows: First, calculate the number of quadruplets num[t], t at each timestamp in the data set; second, sort the timestamps in chronological order; Then, timestamps are merged according to the time granularity thre, and t1, t2, ..., t i are merged into the same time τ: After the above steps, the quaternion (s, r, o, t) becomes (s, r, o, τ), (s, r, o, T = [t b ,t e ]) becomes (s,r,o,T=[τ b ,τ e ]); for (s,r,o,T=[τ b ,τ e ]), and map it into a four-tuple in the form of time points through enumeration operations; The four-tuple (s, r, o, τ) processed by the adaptive time granularity aggregation module is input into the four-tuple structure module. After the initialization embedding module, the initial vector representation of the head entity s, the tail entity o, the relation r and the time τ in the complex space is obtained. In order to capture the neighborhood information of the entity, e s , e o , r,τ is input into the neighbor encoder to obtain the enhanced representation e′ after the head and tail entity aggregated neighbors s , e′ o ; Then the four-tuple structure score f is obtained through the four-tuple scoring module stru ; The neighbor encoder consists of three parts: (1) Neighbor sampling based on entity semantic matching: selecting a fixed number of neighbors from the entity neighbor set; (2) Neighbor aggregator: aggregating the sampled neighbors in a certain way to obtain the neighbor representation e of entity e. nei ; (3) Fusion: The entity e’s own structure is represented by e e With neighbors expressed nei Fusion, get the final entity representation e′ e ; The neighbor sampling based on entity semantic matching is implemented as follows: First, preprocess the entity names to remove pause words and special symbols contained in the entity names; Then, use the word2vec pre-trained word vector to encode the entity name and embed the entity name into name e Defined as: Among them word i represents the embedding of the i-th word in the entity name after word2vec pre-training, and K represents the number of words in the entity name; Finally, the cosine similarity between the name embedding of entity e and the name embedding of its neighbor entities is calculated, and the top n neighbor entities with the highest similarity to entity e are selected to obtain the final entity neighbor set N e ={(r0,e0,τ0),(r2,e2,τ2),…},|N e |=n; The neighbor aggregator is implemented as follows: The neighbor aggregator uses a LSTM-based neighbor aggregator and an attention mechanism-based neighbor aggregator to aggregate the neighbor information of the entity; In the LSTM-based neighbor aggregator, N e The entities in are input into LSTM in chronological order, and the neighbor representation e of entity e is obtained. nei : e nei =LSTM(x),x={e2,e0,……} where {e2, e0, ...} is the neighbor entity embedding set sorted in chronological order, |x| = n; In the attention-based neighbor aggregator, the importance of each neighbor to entity e is calculated: Where N e is the set of neighbors of entity e, e e is the embedding representation of entity e, e e [k] means e e The kth element of Yes i The conjugate representation of , Re(·) represents the real part; After obtaining the weight of each neighbor according to the above formula, the neighbors are aggregated according to the weight, and finally the entity neighbor representation e is obtained nei : The fusion is achieved as follows: Get the neighbor representation of the entity e nei After that, it is represented by the entity's own structure e Fusion, including additive fusion, multiplicative fusion and gated fusion; The additive fusion simply adds the entity's own structural representation and the neighbor representation, and the final entity representation is defined as: The multiplication fusion multiplies the entity's own structural representation with the neighbor representation, and the final entity representation is defined as: The gated fusion filters out important information for the entity, and the final representation of the entity is defined as: Where γ∈[0,1] is the gating factor; The enhanced representation e′ of the head and tail entities is obtained by using the neighbor encoder s and e′ o After that, a scoring function is used to evaluate the probability that the quadruple (s, r, o, τ) is established; the relationship in the quadruple exhibits a unique time constraint, which includes (a) the repetitiveness of the relationship, (b) the order between the relationships, and (c) the time interval between the relationships; in order to capture the time constraint, a scoring function is used to score the quadruple, and the scoring function is defined as follows: f stru (s,r,o,τ)=TIMEPLEX(s,r,o,τ) =f TX (s,r,o,τ)+kf Pair (s,r,o,τ)+λf Rec (s,r,o,τ) Among them, f TX (s,r,o,τ) is the score of the quadruple (s,r,o,τ), is an embedding representation specific to relation r; κ, λ, a, b, c are all hyperparameters; f Rec (s,r,o,τ) is the score of the time constraint (a), f Pair (s, r, o, τ) are the scores of the time constraint (b) and the time constraint (c); In the quaternary type module, the quaternary (s, r, o, τ) is embedded into the real number space to obtain the initial embedding of the head entity s, tail entity o, relation r and time τ in the type module. The score of the quadruple about the entity type is defined as: Among them, η, μ are weight factors, c s [k] means c s The kth element of ; The score f of the four-tuple structure module stru (s, r, o, τ) and the score f of the four-tuple type module type (s, r, o, τ) are added together according to the set weights as the final score of the four-tuple (s, r, o, τ), which is specifically defined as follows: f final (s,r,o,τ)=αf stru (s,r,o,τ)+βf type (s,r,o,t) Among them, α and β are weight factors; The context-aware model is trained using a log-likelihood loss function, which is defined as follows: Among them, G is the set of correct quadruplets; Pr(o|s,r,τ) is the probability of entity o when predicting the tail entity, and the calculation formula is as follows: in, is a set of four-tuples after random replacement of tail entities and filtering operations; Similarly, we obtain Pr(s|o,r,τ) and Pr(τ|s,r,o).

2. A context-aware knowledge completion system based on unsupervised type constraints, characterized in that: The method comprises a memory, a processor and computer program instructions stored in the memory and capable of being executed by the processor. When the processor executes the computer program instructions, the method steps as claimed in claim 1 can be implemented.

Images