Dynamic Knowledge Graph Completion Method, Apparatus, and Electronic Device

Abstract

The present invention discloses a method, apparatus, and electronic device for dynamic knowledge graph completion. The method for dynamic knowledge graph completion includes: obtaining known entities, known relationships, and known time information in the quadruple relational expression of the dynamic knowledge graph to be completed; obtaining a final entity embedding corresponding to the known time information based on the known entities and the known relationships; predicting the probability of the final entity embedding as a missing entity; wherein, obtaining the final entity embedding corresponding to the known time information includes: using the known entities, the known relationships, and the first time step corresponding to the known time information as the input of a structure encoder to obtain a first structured entity embedding at the first time step; and using the first structured entity embedding as the final entity embedding. The present invention reflects the multi-hop structure information of the dynamic knowledge graph through structured entity representation, fully excavates neighborhood information, and improves the completion rate of the dynamic knowledge graph.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and particularly to a method and apparatus for dynamic knowledge graph completion and an electronic device. Background Art

[0002] In recent years, knowledge graphs, as a form of structured human knowledge, have received extensive attention in the academic community. According to existing research, knowledge graphs can be classified into static knowledge graphs and dynamic knowledge graphs. Dynamic knowledge graphs contain a large amount of dynamic factual knowledge. However, dynamic knowledge graphs still have the problem of being incomplete. Reasoning to complete the missing dynamic factual knowledge in dynamic knowledge graphs is a challenging task and is crucial for applications such as event prediction, social network analysis, and recommendation systems.

[0003] Static knowledge graphs represent facts as triples (subject, relation, object) (subject, predicate, object), such as (Zhang XX, participate in, diving competition), while dynamic knowledge graphs associate each triple with a timestamp, such as (Zhang XX, participate in, diving competition, 2000). Dynamic knowledge graphs are considered to be composed of discrete timestamps, which means they can be represented as a series of static knowledge graph snapshots. The task of reasoning to complete the missing facts in these knowledge graph snapshots is called dynamic knowledge graph completion (Temporal Knowledge Graph Completion, abbreviated as TKGC).

[0004] Knowledge Graph Embedding (abbreviated as KGE) is the prerequisite and support for Knowledge Graph Completion (abbreviated as KGC), aiming to map entities and relationships to a low-dimensional vector space to represent the semantic information of entities and relationships. Traditional knowledge graph representation methods ignore the known time information and are unable to handle knowledge reasoning tasks related to time dimension information. To solve this problem, researchers at home and abroad have begun to encode the known time information into the knowledge graph representation in recent years to improve the performance of knowledge graph reasoning and completion. This kind of knowledge graph representation containing known time information can be called Temporal Knowledge Graph Embedding (abbreviated as TKGE) and is used for the reasoning and completion of dynamic knowledge graphs. However, most existing dynamic knowledge graph representation methods simply embed the known time information into the knowledge representation. These methods are still relatively preliminary and only consider the known time information while ignoring the topological structure information of the graph itself. Therefore, there is still much room for improvement in the comprehensive modeling of time and structure information.

[0005] The current research on dynamic knowledge graph representation mainly focuses on how to embed known temporal information into knowledge representation. The earliest work proposed to first learn the temporal order among relations (such as wasBorIn → wonPrize → diedIn), and then incorporate these relation orders as constraints during the knowledge graph representation phase, without directly integrating the known temporal information into the learned representation. TransE proposed various methods for representing known temporal information. For example, representing the concatenation of known temporal information and relations, representing time, entities, and relations in the same vector space, having separate representations for time points, and using time points as coefficients affecting the representation of triple relations. HyTE divides the dynamic knowledge graph into multiple static subgraphs, each corresponding to a timestamp. Then, it projects the entities and their relations in each subgraph onto a hyperplane specific to the timestamp to learn the common representation of the hyperplane (normal vector) and the distribution of knowledge graph elements over time. However, it performs poorly when the number of timestamps is large and cannot be generalized to new timestamps. TA-TransE decomposes the given timestamp into a sequence composed of time tokens, then concatenates the relation token and time modifier token sequences (such as since or until) with the time token sequence and processes them as the input of an LSTM to obtain the predicate sequence representation after encoding. DE-SimplE believes that some features of entities are fixed while some change over time, so it proposes a diachronic embedding function to control the entity feature representation at different time points.

[0006] Existing work on dynamic knowledge graph completion mainly focuses on making some improvements in the representation of known temporal information, studying time-dependent scoring functions, and combining them with static knowledge graph representation methods to score the likelihood of missing facts, thereby completing the dynamic knowledge graph completion task. Although these methods can effectively complete the missing dynamic factual knowledge, they do not consider the multi-hop structure information in the dynamic knowledge graph and have poor performance in mining and completing neighborhood information.

[0007] Moreover, existing methods lack the ability to utilize temporal facts in nearby knowledge graph snapshots to answer queries. Facts such as (Zhang XX, won, Women's 3m Springboard Championship, 1996) or (Zhang XX, participated in, sports meeting, 2000) can help answer the query for the tail entity (Zhang XX, won,?, 2000). Summary of the Invention

[0008] In view of the above, the present invention aims to provide a method, device, and electronic device for dynamic knowledge graph completion, and correspondingly proposes a computer-readable storage medium. Through these aspects, the multi-hop structure information of the dynamic knowledge graph can be reflected through structured entity representation, neighborhood information can be fully mined, and the completion rate of the dynamic knowledge graph can be improved.

[0009] The technical solution adopted by the present invention is as follows:

[0010] In a first aspect, the present invention provides a method for dynamic knowledge graph completion, including:

[0011] Obtain the known entities, known relationships, and known time information in the quadruple relational formula of the dynamic knowledge graph to be completed, where the known entities include the head entity or the tail entity;

[0012] Obtain the final entity embedding corresponding to the known time information based on the known entities and the known relationships;

[0013] Predict the probability of the final entity embedding as the missing entity;

[0014] Among them, obtaining the final entity embedding corresponding to the known time information includes:

[0015] Take the known entities, the known relationships, and the first time step corresponding to the known time information as the input of the structure encoder to obtain the first structured entity embedding at the first time step;

[0016] Take the first structured entity embedding as the final entity embedding.

[0017] In one possible implementation, obtaining the final entity embedding corresponding to the known time information further includes:

[0018] Take the last time step when the known entity before the first time step is active as the second time step;

[0019] Take the first structured entity embedding and the first dynamic entity embedding at the second time step as the input of the time encoder to obtain the second dynamic entity embedding at the first time step as the final dynamic entity embedding at the first time step;

[0020] And,

[0021] Take the final dynamic entity embedding as the final entity embedding.

[0022] In one possible implementation, obtaining the final entity embedding corresponding to the known time information further includes:

[0023] Take the last time step when the known entity before the first time step is active as the second time step;

[0024] Take the known entities, the known relationships, and the second time step as the input of the structure encoder to obtain the second structured entity embedding at the second time step;

[0025] Using the attribution theory, the third structured entity embedding at the first time step is obtained by combining the first structured entity embedding and the second structured entity embedding;

[0026] Taking the third structured entity embedding and the first dynamic entity embedding at the second time step as the input of the time encoder, the third dynamic entity embedding at the first time step is obtained as the final dynamic entity embedding at the first time step;

[0027] And,

[0028] Taking the final dynamic entity embedding as the final entity embedding.

[0029] In one possible implementation, obtaining the final entity embedding corresponding to the known time information further includes:

[0030] Using a gating mechanism, a comprehensive entity embedding is obtained based on the first structured entity embedding and the final dynamic entity embedding as the final entity embedding.

[0031] In one possible implementation, the structure encoder includes a first training model based on a multi-relational message passing neural network.

[0032] In one possible implementation, the time encoder includes a second training model based on a recurrent neural network.

[0033] In one possible implementation, obtaining the second dynamic entity embedding at the first time step includes:

[0034] Calculating the first decay rate of the first dynamic entity embedding;

[0035] Calculating the fourth dynamic entity embedding at the second time step based on the first decay rate and the first dynamic entity embedding;

[0036] Taking the first structured entity embedding and the fourth dynamic entity embedding as the input of the time encoder, the second dynamic entity embedding at the first time step is obtained.

[0037] In one possible implementation, obtaining the third structured entity embedding includes:

[0038] Calculating the second decay rate of the known entity between the second time step and the first time step, and taking the second decay rate as the first weight of the second structured entity embedding for the third structured entity embedding;

[0039] Calculating the weighted sum using the first weight, the first structured entity embedding, and the second structured entity embedding as the third structured entity embedding.

[0040] In one possible implementation, obtaining the third structured entity embedding includes:

[0041] Calculating a second decay rate of the known entity between the second time step and the first time step, and using the second decay rate as a first weight of the second structured entity embedding for the third structured entity embedding;

[0042] Taking the nearest time step after the first time step when the known entity is in an active state as the third time step;

[0043] Using the known entity, the known relationship, and the third time step as inputs to a structure encoder to obtain a fourth structured entity embedding at the third time step;

[0044] Calculating a third decay rate of the known entity between the third time step and the first time step and a fourth decay rate of the known entity within the first time step, using the third decay rate as a second weight of the fourth structured entity embedding for the third structured entity embedding, and using the fourth decay rate as a third weight of the first structured entity embedding for the third structured entity embedding;

[0045] Calculating a weighted sum using the first structured entity embedding, the second structured entity embedding, the fourth structured entity embedding, and the corresponding decay rates as the third structured entity embedding.

[0046] In one possible implementation, obtaining the comprehensive entity embedding includes:

[0047] Respectively calculating the frequencies of the known entity, the known relationship, and the corresponding relationship between the known entity and the known relationship over a time window that includes the time point or time period corresponding to the known time information;

[0048] The frequencies of the known entity, the known relationship, and the corresponding relationship between the known entity and the known relationship form a frequency vector of the missing entity corresponding to the dynamic knowledge graph to be completed;

[0049] Obtaining a fourth weight of the first structured entity embedding for the comprehensive entity embedding according to the frequency vector;

[0050] Calculating a weighted sum according to the fourth weight, the first structured entity embedding, and the final dynamic entity embedding as the comprehensive entity embedding.

[0051] In a second aspect, the present invention also provides a dynamic knowledge graph completion device, including a task receiving module, a final entity embedding obtaining module, and a prediction module;

[0052] The task receiving module is used to obtain the known entities, known relationships, and known time information in the quadruple relationship of the dynamic knowledge graph to be completed, and the known entities include the head entity or the tail entity;

[0053] The final entity embedding obtaining module is used to obtain the final entity embedding corresponding to the known time information according to the known entity and the known relationship;

[0054] The prediction module is used to predict the probability of the final entity embedding as the missing entity;

[0055] Among them, the final entity embedding obtaining module includes a structure encoder, and the structure encoder is used to obtain the first structured entity embedding at the first time step by using the known entity and the first time step corresponding to the known time information, and use the first structured entity embedding as the final entity embedding.

[0056] In one possible implementation manner, the final entity embedding obtaining module further includes a time step determination sub-module and a time encoder;

[0057] The time step determination sub-module is used to use the last time step when the known entity before the first time step is in an active state as the second time step;

[0058] The time encoder is used to use the first structured entity embedding and the first dynamic entity embedding at the second time step as the input of the time encoder, obtain the second dynamic entity embedding at the first time step as the final dynamic entity embedding at the first time step, and use the final dynamic entity embedding as the final entity embedding.

[0059] In one possible implementation manner, the structure encoder is further used to use the attribution theory to combine the first structured entity embedding and the second structured entity embedding at the second time step to obtain the third structured entity embedding at the first time step;

[0060] The time encoder is further used to use the third structured entity embedding and the first dynamic entity embedding at the second time step to obtain the third dynamic entity embedding at the first time step as the final dynamic entity embedding at the first time step, and use the final dynamic entity embedding as the final entity embedding.

[0061] In one possible implementation manner, the final entity embedding obtaining module further includes a gating unit, and the gating unit is used to obtain a comprehensive entity embedding according to the first structured entity embedding and the final dynamic entity embedding as the final entity embedding.

[0062] In a third aspect, the present invention further provides an electronic device, including:

[0063] One or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the electronic device, cause the electronic device to execute the above-mentioned dynamic knowledge graph completion method.

[0064] In a fourth aspect, the present invention further provides a computer-readable storage medium, in which a computer program is stored, and when it runs on a computer, it causes the computer to execute the above-mentioned dynamic knowledge graph completion method.

[0065] In a fifth aspect, the present invention further provides a computer program product that, when executed by a computer, is used to execute the method in the first aspect or any possible implementation manner of the first aspect.

[0066] In a possible design of the fifth aspect, the relevant programs involved in the product can be stored in whole or in part in the memory packaged together with the processor, or can be stored in whole or in part in the storage medium not packaged together with the processor.

[0067] The concept of the present invention is that when completing a dynamic knowledge graph, a structure encoder is used to capture the topological structure information in the knowledge graph, screen the neighborhood information of entities, and complete the knowledge graph based on the screening result, improving the structuring and completion rate of the dynamic knowledge graph. The present invention also considers the temporal facts in the nearby knowledge graph snapshots, uses a temporal encoder to integrate the entity representation information across time steps, optimizes the topological structure information of entities, and provides a solid foundation for accurate completion. In addition, the present invention also eliminates the adverse effects brought by temporal variability and temporal sparsity to the completion of the dynamic knowledge graph. BRIEF DESCRIPTION OF THE DRAWINGS

[0068] To make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be further described below in conjunction with the drawings, where:

[0069] Figure 1 is a flowchart of the dynamic knowledge graph completion method provided by the present invention;

[0070] Figure 2 is a flowchart of a preferred embodiment for obtaining the final entity embedding provided by the present invention;

[0071] Figure 3 is a flowchart of obtaining the third structured entity embedding at the first time step provided by the present invention;

[0072] Figure 4 Structural diagram of the dynamic knowledge graph completion device provided by the present invention;

[0073] Figure 5 Structural diagram of the final entity embedding acquisition module provided by the present invention;

[0074] Figure 6 For Figure 2 Schematic structural diagram of the corresponding dynamic knowledge graph completion device;

[0075] Figure 7 Schematic structural diagram of an embodiment of the electronic device provided by the present invention. Detailed implementation manners

[0076] The embodiments of the present invention will be described in detail below. The examples of the embodiments are shown in the accompanying drawings, where the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions from beginning to end. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be construed as a limitation of the present invention.

[0077] First, the following basic description is made for dynamic knowledge graph completion:

[0078] Given a dynamic knowledge graph , where . is the knowledge graph snapshot at time step . And respectively represent the union of the entity sets of all time step knowledge graph snapshots and the union of the relationship sets of all time step knowledge graph snapshots, and are known. Represents the set of triples contained in the knowledge graph snapshot at time step , where , , . Let represent the set of correct triples at time step , that is .

[0079] The dynamic knowledge graph completion problem is to rank candidate head entities or tail entities in the case of a given head entity query or tail entity query , where , but . Among them, the head entity query and the tail entity query are the quadruple relational expressions of the dynamic knowledge graph to be completed.

[0080] For the foregoing core concept, the present invention provides at least one embodiment of a dynamic knowledge graph completion method, such as Figure 1 shown, which may include the following steps:

[0081] S110: Obtain the known entities, known relationships, and known time information in the quadruple relationship of the dynamic knowledge graph to be completed, where the known entities include the head entity or the tail entity.

[0082] For example, in the above head entity query among them, is the known entity, is the known relationship, is the known time information, is the missing entity. In the above tail entity query among them, is the known entity, is the known relationship, is the known time information, is the missing entity.

[0083] The following takes the tail entity query as an example to illustrate the present invention.

[0084] S120: Obtain the final entity embedding corresponding to the known time information based on the known entity and the known relationship.

[0085] Specifically, use at least one training model to obtain the final entity embedding. This part will be further described in the following content.

[0086] S130: Predict the probability of the final entity embedding as the missing entity.

[0087] Specifically, as an embodiment, a scoring function training model is used to predict the probability.

[0088] As an embodiment, in step S120, obtaining the final entity embedding corresponding to the known time information includes:

[0089] S1201: Use the known entity, the known relationship, and the first time step corresponding to the known time information as the input of the structure encoder to obtain the first structured entity embedding of the first time step. Use the first structured entity embedding as the final entity embedding for probability prediction.

[0090] The structure encoder is used to generate structured entity embeddings based on the dynamic knowledge graph snapshot at each time step.

[0091] Specifically, as an embodiment, the structure encoder includes at least a first training model based on a multi-relational message passing neural network. As an example, this first training model is a Relational graph convolutional network (RGCN) model. Please refer to Figure 6 .

[0092] Based on the model with L layers, the first time step corresponding to the known entity, the known relationship, and the known time information is used as the input of the structure encoder. The output of the model is the hidden layer embedding of the L-th layer within the first time step . This output is used as the first structured entity embedding . This hidden layer embedding summarizes the neighborhood information in the layer multi-hop structure within the dynamic knowledge graph snapshot . within .

[0093] Among them, the hidden layer embedding of the th layer

[0094] (1)

[0095] (2)

[0096] Among them, is the hidden layer embedding output by the 0th layer of the first time step , is the entity embedding matrix, is the one-hot embedding of the entity of the node , and are the relationship transformation matrix and the head entity transformation matrix of the th layer of the training model, is the set of neighborhood entities of the entity of the node connected by the relationship . Its size serves as a normalization constant for averaging neighborhood node information.

[0097] Thus, the first structured entity embedding of the first time step .

[0098] It can be understood that the present invention can use any other multi-relational graph encoder composed of a multi-relational message passing neural network as the first training model, such as CompGCN, EdgeGAT, etc.

[0099] In this embodiment, the structure attributes inherent in the multi-relational message passing neural network are used to capture the L-layer topological structure information in the knowledge graph, thereby screening the neighborhood information of entities based on known relationships, and complementing the knowledge graph based on the screening results to improve the structuring and completion rate of the dynamic knowledge graph.

[0100] Neither the existing methods nor the above embodiments utilize the temporal facts in the knowledge graph snapshots near the known temporal information in the query to answer the query. For example, facts such as (Zhang XX, won, women's 3m springboard championship, 1996) or (Zhang XX, participated in, sports meeting, 2000) are helpful for answering the query (Zhang XX, won,?, 2000) for the tail entity.

[0101] Based on the above considerations, in a preferred embodiment, obtaining the final entity embedding corresponding to the known temporal information further includes:

[0102] S1202: Use the last time step when the previous known entity was active as the second time step before the first time step .

[0103] S1203: Use the first structured entity embedding and the first dynamic entity embedding at the second time step as the input to the time encoder to obtain the second dynamic entity embedding at the first time step . The second dynamic entity embedding at the first time step .

[0104] In this preferred embodiment, the second dynamic entity embedding at the first time step is used as the final dynamic entity embedding at the first time step , and the final dynamic entity embedding is used as the final entity embedding, as shown. Figure 2 .

[0105] The time encoder is used to integrate the structured entity embeddings across time steps and utilize the temporal facts in the knowledge graph snapshots near the known temporal information in the query to answer the query. Specifically, the time encoder includes a second training model based on a recurrent neural network (such as Long short-term memory (LSTM), Gate Recurrent Unit (GRU), etc.).

[0106] The following uses GRU as an example to illustrate this preferred embodiment.

[0107] In a possible implementation, the second dynamic entity embedding at the first time step is obtained , including:

[0108] S12031: Calculate the first decay rate of the first dynamic entity embedding .

[0109] (3)

[0110] Wherein, and are learnable parameters, , , is the number of knowledge graph snapshots input to the model according to timestamps, please refer to Figure 6 .

[0111] S12032: Calculate the fourth dynamic entity embedding at the second time step according to the first decay rate and the first dynamic entity embedding (4)

[0112] When

[0113] is non-zero vector. . is a non-zero vector.

[0114] S12033: Take the first structured entity embedding and the fourth dynamic entity embedding as the input of the time encoder, and obtain the second dynamic entity embedding at the first time step (5)

[0115] In this preferred implementation, the second training model can access all time steps during training. Although there is missing data within each time step, all (incomplete) knowledge graph snapshot information

[0116] is available during the training process. is available.

[0117] In this preferred implementation, the information of specific time steps before the first time step is integrated, so that entity queries refer to the information of past time steps.

[0118] Although the above-mentioned embodiments well solve the problems of data structuring and information integration of nearby time steps, such dynamic knowledge graph data has the problem of temporal sparsity. The problem of temporal sparsity indicates that only a small portion of entities are active in the knowledge graph snapshot at each time step (an entity is active at the time step corresponding to the knowledge graph snapshot if the entity has at least one adjacent entity in the same knowledge graph snapshot). In existing knowledge graph completion methods, the same embedding is usually assigned to inactive entities at different time steps, and such a processing method cannot fully represent time-sensitive features.

[0119] Based on the above problems, the present application proposes a preferred embodiment. Based on the above step S1202, obtaining the final entity embedding corresponding to the known time information further includes:

[0120] S1204: Using the known entity, the known relationship, and the second time step as the input of the structure encoder to obtain the second structured entity embedding of the second time step .

[0121] S1205: Using the attribution theory, combining the first structured entity embedding and the second structured entity embedding to obtain the third structured entity embedding of the first time step

[0122] (6)

[0123] As a possible embodiment, as shown in Figure 3 , obtaining the third structured entity embedding includes:

[0124] S12051: Calculating the second decay rate of the known entity between the second time step and the first time step

[0125] (7)

[0126] Wherein, and are learnable parameters.

[0127] Using the second decay rate as the first weight of the second structured entity embedding for the third structured entity embedding .

[0128] ​​S12052: Use the first weight and the first structured entity embedding , the second structured entity embedding to calculate a weighted sum as the third structured entity embedding

[0129] (8)

[0130] This embodiment solves the temporal sparsity caused by inactive entities in the knowledge graph snapshot before the first time step . On this basis, in a preferred embodiment, obtaining the third structured entity embedding further includes:

[0131] After S12051, execute S12053: Take the nearest time step when the known entity is active after the first time step as the third time step .

[0132] S12054: Use the known entity, the known relationship, and the third time step as the input of the structure encoder to obtain the fourth structured entity embedding at the third time step .

[0133] S12055: Calculate the third decay rate of the known entity between the third time step and the first time step and the fourth decay rate of the known entity within the first time step

[0134] (9)

[0135] (10)

[0136] Use the third decay rate as the second weight for the third structured entity embedding , and use the fourth decay rate as the third weight for the third structured entity embedding of the first structured entity embedding .

[0137] S12056: Use the first structured entity embedding , the second structured entity embedding , and the fourth structured entity embedding and the corresponding decay rates to calculate a weighted sum as the third structured entity embedding

[0138] (11)

[0139] This preferred embodiment simultaneously eliminates the temporal sparsity brought about by inactive entities in the knowledge graph snapshots at past and future time steps.

[0140] S1206: Embed the third structured entity and the first dynamic entity embedding at the second time step as the input of the time encoder to obtain the third dynamic entity embedding at the first time step

[0141] (12)

[0142] In this preferred embodiment, the third dynamic entity embedding is used as the final dynamic entity embedding at the first time step, and this final dynamic entity embedding is used as the final entity embedding.

[0143] This preferred embodiment combines the stale representations of inactive entities with temporal representations using attribution theory, uses a GRU model to reflect the influence of past and future inactive entities on the structured entity embedding, and uses a Bi-GRU model to optimize the structured entity embedding.

[0144] Existing dynamic knowledge graph completion methods ignore the impact brought about by temporal variability. Temporal variability means that in a real-world dynamic knowledge graph, when answering different queries, the model can access different amounts of reference time information in nearby knowledge graph snapshots, and these reference time information have different weights due to the constraints of specific entities and relationships in the queries. For example, in a sports event dataset, there are more quadruple data containing the head entity-relationship pair (Zhang XX, obtained) than quadruple data containing (Li XX, obtained) during the period from 1996 to 2000.

[0145] Based on the above considerations, in a preferred embodiment, obtaining the final entity embedding corresponding to the known time information further includes:

[0146] S1207: Using a gating mechanism, obtain a comprehensive entity embedding based on the first structured entity embedding and the final dynamic entity embedding as the final entity embedding.

[0147] Figure 2 Illustrates the entire process of obtaining the final entity embedding corresponding to the known time information in this preferred embodiment.

[0148] Entity embedding also depends on the amount of dynamic knowledge it participates in during the most recent time step. Based on this, in this preferred embodiment, a frequency-based gating mechanism is used to fuse the structured entity embedding output by the structure encoder and the dynamic entity embedding output by the time encoder in a frequency-related manner. To enable an entity to locate its position in a quadruple, it is distinguished according to the query type (head entity or tail entity query) and the entity position (whether the known entity is the head entity or the tail entity in the fact being queried).

[0149] Define the term "pattern" to describe a non-empty subset of the quadruple (s, r, o, t). The number of facts having this pattern within the time window is defined as the temporal frequency of the pattern. For example, for the quadruple (Zhang XX, participate in, sports meeting, 1992), the temporal frequency of the pattern (Zhang XX, participate in) is the number of quadruples (Zhang XX, participate in, *, t0), where t0 is within the time window (e.g., from 2000 to 2014).

[0150] As a possible implementation, obtaining the comprehensive entity embedding includes:

[0151] S12071: Calculate the frequencies of the known entity, the known relationship, and the correspondence between the known entity and the known relationship over the time window, where the time window includes the time point or time period corresponding to the known time information.

[0152] Based on the above, the temporal pattern frequencies (TPFs) associated with the quadruple (s, r, o, t) include:

[0153] (1) Head entity frequency

[0154] (2) Tail entity frequency

[0155] (3) Relationship frequency

[0156] (4) Head entity - relationship frequency

[0157] (5) Relationship - tail entity frequency

[0158] S12072: The frequencies of the known entity, the known relationship, and the correspondence between the known entity and the known relationship form a frequency vector of the missing entity corresponding to the dynamic knowledge graph to be completed.

[0159] Without loss of generality, the tail entity query is taken as an example to illustrate this preferred embodiment. For the gating mechanism (s, r,?, t), the goal is to predict the missing tail entity in the quadruple.

[0160] When answering the tail entity query (s, r,?, t), the model can only access the frequency vector

[0161] S12073: Obtain the fourth weight of the first structured entity embedding with respect to the comprehensive entity embedding based on the frequency vector .

[0162] As a possible implementation Learned through a two-layer neural network, that is , .

[0163] S12074: Calculate the weighted sum based on the fourth weight , the first structured entity embedding , the final dynamic entity embedding as the comprehensive entity embedding.

[0164] Specifically, use the frequency vector to define a gate on the comprehensive entity embedding of the tail entity in the tail entity query :

[0165] (13)

[0166] It can be understood that replacing the second dynamic entity embedding of the above tail entity with the third dynamic entity embedding can better solve the problem of time variability in the tail entity query.

[0167] It can be understood that for the head entity query, the frequency vector , define a gate on the comprehensive entity embedding of the head entity :

[0168] (14)

[0169] where is the weight of the first structured entity embedding of the head entity with respect to the comprehensive entity embedding; is the first structured entity embedding of the head entity, and its acquisition method is the same as that of the first structured entity embedding of the tail entity ; is the second dynamic entity embedding of the head entity, and its acquisition method is the same as that of the second dynamic entity embedding of the tail entity .

[0170] It can be understood that the second dynamic entity of the above-mentioned head entity is embedded in Replaced with third dynamic entity embedding of head entity , which can better solve the problem of time variability in head entity queries.

[0171] For step S130, let Denote the probability (i.e., score) of the final entity embedding of the above query as the four-tuple consisting of the tail entity or the head entity, and let DEC represent any suitable decoding function of the static knowledge graph, such as TransE, HyTE, TA-TransE, DE-SimplE, etc. The score of the four-tuple is defined as follows:

[0172] (15)

[0173] and The final entity embeddings representing the head entity and the tail entity are obtained through step S120. Learnable embeddings representing relations, obtained through existing word embedding methods.

[0174] In order to train the model using this scoring function, a gradient-based mini-batch optimization learning model is used.

[0175] Parameters. For each triple , sample a set of negative entities .

[0176] Without loss of generality, the cross entropy loss function for negative examples in tail entity queries is as follows:

[0177] (16)

[0178] Therefore, the training loss function of the scoring function training model is the sum of the two query losses: .

[0179] Understandably, the loss function in head entity query and the loss function of negative sampling are similar.

[0180] Corresponding to the above embodiments and preferred solutions, the present invention also provides an embodiment of a dynamic knowledge graph completion device, such as Figure 4 As shown, it may specifically include a task receiving module 410, a final entity embedding obtaining module 420 and a prediction module 430;

[0181] The task receiving module 410 is used to obtain known entities, known relations and known time information in the four-tuple relational expression of the dynamic knowledge graph to be completed, and the known entities include head entities or tail entities.

[0182] The final entity embedding obtaining module 420 is configured to obtain a final entity embedding corresponding to the known time information based on the known entity and the known relationship.

[0183] The prediction module 430 is configured to predict the probability of the final entity embedding as a missing entity.

[0184] In one possible implementation, as Figure 5 shown, the final entity embedding obtaining module 420 includes a structure encoder 4201, and the structure encoder 4201 is configured to obtain a first structured entity embedding at the first time step by using the known entity and the first time step corresponding to the known time information. In one possible implementation, the first structured entity embedding is used as the final entity embedding.

[0185] In one possible implementation, the structure encoder 4201 is further configured to obtain a second structured entity embedding at the second time step by using the known entity and the second time step.

[0186] In one possible implementation, the structure encoder 4201 is further configured to obtain a fourth structured entity embedding at the third time step by using the known entity and the third time step.

[0187] In one possible implementation, as Figure 5 shown, the final entity embedding obtaining module 420 further includes a time step determination sub-module 4202 and a time encoder 4203.

[0188] The time step determination sub-module 4202 is configured to use the last time step when the known entity before the first time step is active as the second time step.

[0189] The time encoder 4203 is configured to use the first structured entity embedding and the first dynamic entity embedding at the second time step as inputs of the time encoder to obtain a second dynamic entity embedding at the first time step.

[0190] In one possible implementation, the second dynamic entity embedding is used as the final dynamic entity embedding at the first time step, and the final dynamic entity embedding is used as the final entity embedding.

[0191] In one possible implementation, the structure encoder 4201 is further configured to use attribution theory to obtain the third structured entity embedding at the first time step by combining the first structured entity embedding and the second structured entity embedding at the second time step; the time encoder 4203 is further configured to obtain the third dynamic entity embedding at the first time step by using the third structured entity embedding and the first dynamic entity embedding at the second time step.

[0192] In one possible implementation, the third dynamic entity embedding is used as the final dynamic entity embedding at the first time step, and the final dynamic entity embedding is used as the final entity embedding.

[0193] In one possible implementation, as Figure 5 shown, the final entity embedding obtaining module 420 further includes a gating unit 4204, and the gating unit 4204 is configured to obtain a comprehensive entity embedding based on the first structured entity embedding and the final dynamic entity embedding as the final entity embedding.

[0194] It should be understood that the division of each component of the dynamic knowledge graph completion device shown above Figure 4 is only a division of logical functions. In actual implementation, they can be fully or partially integrated into a physical entity, or physically separated. And these components can all be implemented in the form of software called by a processing element; they can also all be implemented in the form of hardware; or some components can be implemented in the form of software called by a processing element, and some components can be implemented in the form of hardware. For example, a certain module above can be a separately established processing element, or can be implemented by integrating it into a certain chip of an electronic device. The implementation of other components is similar. In addition, these components can be fully or partially integrated together, or can be independently implemented. In the implementation process, each step of the above method or each of the above components can be completed by the integrated logic circuit in the processor element or the instruction in the form of software.

[0195] For example, the above components can be one or more integrated circuits configured to implement the above method, such as: one or more application specific integrated circuits (ASICs), or, one or more digital signal processors (DSPs), or, one or more field programmable gate arrays (FPGAs), etc. Again, these components can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0196] Based on the above embodiments and their preferred solutions, those skilled in the art can understand that in actual operation, the present invention is applicable to multiple implementation manners. The present invention uses the following carriers as illustrative explanations:

[0197] (1) An electronic device, which may include:

[0198] One or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory, and the one or more computer programs include instructions that, when executed by the device, cause the device to perform the steps / functions of the foregoing embodiments or equivalent implementation manners.

[0199] Figure 7 FIG. is a schematic structural diagram of an embodiment of the electronic device of the present invention. Among them, the device may be an electronic device or a circuit device built into the above electronic device. The above electronic device may be a PC, a server, a smart terminal (such as a mobile phone, a tablet, a watch, glasses, etc.), a smart TV, a teller machine, a robot, a drone, an ICV, a smart (motor) vehicle, and vehicle-mounted equipment, etc. The specific form of the XXX device in this embodiment is not limited.

[0200] Specifically, as Figure 7 shown, the electronic device 900 includes a processor 910 and a memory 930. Among them, the processor 910 and the memory 930 can communicate with each other through an internal connection path to transmit control and / or data signals. The memory 930 is used to store computer programs, and the processor 910 is used to call and run the computer programs from the memory 930. The above processor 910 and the memory 930 may be integrated into a processing device, and more commonly, they are independent components. The processor 910 is used to execute the program code stored in the memory 930 to implement the above functions. Specifically, when implemented, the memory 930 may also be integrated in the processor 910, or independent of the processor 910.

[0201] In addition, in order to make the functions of the electronic device 900 more complete, the device 900 may further include one or more of an input unit 960, a display unit 970, an audio circuit 980, a camera 990, and a sensor 901, etc. The audio circuit may further include a speaker 982, a microphone 984, etc. Among them, the display unit 970 may include a display screen.

[0202] Furthermore, the above electronic device 900 may further include a power supply 950 for supplying electrical energy to various components or circuits in the device 900.

[0203] It should be understood that Figure 4The electronic device 900 shown can implement each process of the method provided in the foregoing embodiments. The operations and / or functions of the various components in the device 900 can respectively implement the corresponding processes in the foregoing method embodiments. For details, reference can be made to the descriptions of the method, apparatus, and other embodiments in the foregoing text. To avoid repetition, the detailed descriptions are appropriately omitted here.

[0204] It should be understood that Figure 4 The processor 910 in the electronic device 900 shown may be a system-on-chip (SOC). The processor 910 may include a central processing unit (hereinafter referred to as CPU), and may further include other types of processors, which will be introduced in detail later.

[0205] In summary, the various processors or processing units inside the processor 910 can cooperate together to implement the previous method process, and the corresponding software programs of the various processors or processing units can be stored in the memory 930.

[0206] (2) A readable storage medium stores a computer program or the foregoing device. When the computer program or the foregoing device is executed, the computer is caused to execute the steps / functions of the foregoing embodiments or equivalent embodiments.

[0207] In several embodiments provided by the present invention, if any function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on such an understanding, some technical solutions of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of the following software product.

[0208] (3) A computer program product (the product may include the foregoing device). When the computer program product runs on a terminal device, the terminal device is caused to execute the dynamic knowledge graph completion method of the foregoing embodiments or equivalent embodiments.

[0209] As can be seen from the description of the above embodiments, those skilled in the art can clearly understand that all or part of the steps in the above implementation methods can be implemented by means of software plus a necessary general hardware platform. Based on such an understanding, the above computer program product may include, but is not limited to, an APP; continuing from the previous text, the above device / terminal may be a computer device (such as a mobile phone, a PC terminal, a cloud platform, a server, a server cluster, or a network communication device such as a media gateway, etc.). Moreover, the hardware structure of this computer device may specifically include: at least one processor, at least one communication interface, at least one memory, and at least one communication bus; the processor, the communication interface, and the memory can all complete communication with each other through the communication bus. Among them, the processor may be a central processing unit CPU, a DSP, a microcontroller, or a digital signal processor, and may also include a GPU, an embedded neural network processor (Neural-network Process Units; hereinafter referred to as: NPU), and an image signal processor (Image Signal Processing; hereinafter referred to as: ISP). The processor may also include a specific integrated circuit ASIC, or one or more integrated circuits configured to implement the embodiments of the present invention. In addition, the processor may have the function of operating one or more software programs, and the software programs may be stored in storage media such as the memory; and the aforementioned memory / storage medium may include: non-volatile memory, such as a non-removable disk, a USB flash drive, a mobile hard disk, an optical disc, etc., as well as a read-only memory (hereinafter referred to as: ROM), a random access memory (hereinafter referred to as: RAM), etc.

[0210] In the embodiments of the present invention, "at least one" means one or more, and "a plurality" means two or more. "And / or" describes the association relationship of associated objects, indicating that three relationships may exist. For example, A and / or B may represent the cases of A existing alone, A and B existing simultaneously, and B existing alone. Where A and B may be singular or plural. The character " / " generally indicates that the associated objects before and after are an "or" relationship. "At least one of the following" and its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, and c may represent: a, b, c, a and b, a and c, b and c, or a and b and c, where a, b, and c may be single or multiple.

[0211] Those skilled in the art can realize that the various modules, units, and method steps described in the embodiments disclosed in this specification can be implemented by a combination of electronic hardware, computer software, and electronic hardware. Whether these functions are executed in a hardware or software manner depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present invention.

[0212] In addition, each embodiment in this specification is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other. In particular, for embodiments such as devices and equipment, since they are basically similar to the method embodiments, the relevant parts can be referred to the partial description of the method embodiments. The embodiments of the devices and equipment described above are only illustrative. The modules and units described as separate components may or may not be physically separated, that is, they can be located in one place or distributed to multiple places, such as the nodes of a system network. Specifically, some or all of the modules and units can be selected according to actual needs to achieve the purpose of the above embodiment solutions. Those skilled in the art can understand and implement without creative efforts.

[0213] The structure, features, and effects of the present invention have been described in detail based on the embodiments shown in the drawings above. However, the above are only the preferred embodiments of the present invention. It should be noted that for the technical features involved in the above embodiments and their preferred modes, those skilled in the art can reasonably combine and match them into various equivalent solutions without departing from and without changing the design concept and technical effects of the present invention. Therefore, the present invention is not limited by the scope shown in the drawings. Any changes made according to the concept of the present invention or modified into equivalent embodiments with equivalent changes still fall within the spirit covered by the specification and the drawings, and should be within the protection scope of the present invention.

Claims

1. A method for dynamic knowledge graph completion, characterized in that, it is applied to answering queries for sports events, and the method for dynamic knowledge graph completion includes: Obtain the known entities, known relationships, and known time information in the quadruple relationship formula of the dynamic knowledge graph to be completed, where the known entities include the head entity or the tail entity; Obtain the final entity embedding corresponding to the known time information based on the known entity and the known relationship; Predict the probability of the final entity embedding as the missing entity; Among them, obtaining the final entity embedding corresponding to the known time information includes: Taking the known entity, the known relationship, and the first time step corresponding to the known time information as the input of the structure encoder, and obtaining the first structured entity embedding of the first time step; Taking the first structured entity embedding as the final entity embedding; Among them, obtaining the final entity embedding corresponding to the known time information further includes: Taking the last time step when the known entity before the first time step is active as the second time step; Taking the first structured entity embedding and the first dynamic entity embedding of the second time step as the input of the time encoder, and obtaining the second dynamic entity embedding of the first time step as the final dynamic entity embedding of the first time step; And, Taking the final dynamic entity embedding as the final entity embedding; Among them, the time encoder is used to integrate structured entity embeddings across time steps and answer queries using time facts in the knowledge graph snapshots near the known time information in the query; Among them, obtaining the final entity embedding corresponding to the known time information further includes: Taking the last time step when the known entity before the first time step is active as the second time step; Taking the known entity, the known relationship, and the second time step as the input of the structure encoder, and obtaining the second structured entity embedding of the second time step; Using attribution theory, combining the first structured entity embedding and the second structured entity embedding to obtain the third structured entity embedding of the first time step; Taking the third structured entity embedding and the first dynamic entity embedding of the second time step as the input of the time encoder, and obtaining the third dynamic entity embedding of the first time step as the final dynamic entity embedding of the first time step; And, Taking the final dynamic entity embedding as the final entity embedding.

2. The method for dynamic knowledge graph completion according to claim 1, characterized in that, obtaining the final entity embedding corresponding to the known time information further includes: Using a gating mechanism, obtaining a comprehensive entity embedding based on the first structured entity embedding and the final dynamic entity embedding as the final entity embedding.

3. The method for dynamic knowledge graph completion according to claim 1, characterized in that, the structure encoder includes a first training model based on a multi-relational message passing neural network.

4. The method for dynamic knowledge graph completion according to claim 1, characterized in that, the time encoder includes a second training model based on a recurrent neural network.

5. The dynamic knowledge graph completion method according to claim 4, wherein, obtaining the second dynamic entity embedding at the first time step includes: calculating a first decay rate of the first dynamic entity embedding; calculating a fourth dynamic entity embedding at the second time step based on the first decay rate and the first dynamic entity embedding; using the first structured entity embedding and the fourth dynamic entity embedding as inputs to the time encoder to obtain the second dynamic entity embedding at the first time step.

6. The dynamic knowledge graph completion method according to claim 1, wherein, obtaining the third structured entity embedding includes: calculating a second decay rate of the known entity between the second time step and the first time step, and using the second decay rate as a first weight of the second structured entity embedding for the third structured entity embedding; calculating a weighted sum using the first weight, the first structured entity embedding, and the second structured entity embedding as the third structured entity embedding.

7. The dynamic knowledge graph completion method according to claim 1, wherein, obtaining the third structured entity embedding includes: calculating a second decay rate of the known entity between the second time step and the first time step, and using the second decay rate as a first weight of the second structured entity embedding for the third structured entity embedding; taking the nearest time step when the known entity is active after the first time step as the third time step; using the known entity, the known relationship, and the third time step as inputs to the structure encoder to obtain a fourth structured entity embedding at the third time step; calculating a third decay rate of the known entity between the third time step and the first time step and a fourth decay rate of the known entity within the first time step, using the third decay rate as a second weight of the fourth structured entity embedding for the third structured entity embedding, and using the fourth decay rate as a third weight of the first structured entity embedding for the third structured entity embedding; calculating a weighted sum using the first structured entity embedding, the second structured entity embedding, and the fourth structured entity embedding and their corresponding decay rates as the third structured entity embedding.

8. The dynamic knowledge graph completion method according to claim 2, wherein, obtaining the comprehensive entity embedding includes: respectively calculating the frequencies of the known entity, the known relationship, and the corresponding relationship between the known entity and the known relationship on a time window, where the time window includes the time point or time period corresponding to the known time information; the frequencies of the known entity, the known relationship, and the corresponding relationship between the known entity and the known relationship form a frequency vector of the missing entity corresponding to the dynamic knowledge graph to be completed; obtaining a fourth weight of the first structured entity embedding for the comprehensive entity embedding based on the frequency vector; Calculate the weighted sum based on the fourth weight, the first structured entity embedding, and the final dynamic entity embedding as the comprehensive entity embedding.

9. A dynamic knowledge graph completion device, characterized in that it is applied to answer queries for sports events. The dynamic knowledge graph completion device includes a task receiving module, a final entity embedding obtaining module, and a prediction module; The task receiving module is used to obtain the known entity, known relationship, and known time information in the quadruple relationship of the dynamic knowledge graph to be completed, and the known entity includes the head entity or the tail entity; The final entity embedding obtaining module is used to obtain the final entity embedding corresponding to the known time information based on the known entity and the known relationship; The prediction module is used to predict the probability of the final entity embedding as the missing entity; wherein, the final entity embedding obtaining module includes a structure encoder, and the structure encoder is used to obtain the first structured entity embedding at the first time step by using the known entity and the first time step corresponding to the known time information, and use the first structured entity embedding as the final entity embedding; wherein, the final entity embedding obtaining module further includes a time step determination sub-module and a time encoder; The time step determination sub-module is used to use the last time step when the known entity before the first time step is active as the second time step; The time encoder is used to use the first structured entity embedding and the first dynamic entity embedding at the second time step as the input of the time encoder, obtain the second dynamic entity embedding at the first time step as the final dynamic entity embedding at the first time step, and use the final dynamic entity embedding as the final entity embedding; wherein, the time encoder is used to integrate the structured entity embeddings across time steps and use the time facts in the knowledge graph snapshot near the known time information in the query to answer the query; wherein, the structure encoder is further used to obtain the second structured entity embedding at the second time step by using the known entity, the known relationship, and the second time step; and use the attribution theory to combine the first structured entity embedding and the second structured entity embedding to obtain the third structured entity embedding at the first time step; The time encoder is further used to obtain the third dynamic entity embedding at the first time step by using the third structured entity embedding and the first dynamic entity embedding at the second time step as the final dynamic entity embedding at the first time step, and use the final dynamic entity embedding as the final entity embedding.

10. The dynamic knowledge graph completion device according to claim 9, characterized in that the final entity embedding obtaining module further includes a gating unit, and the gating unit is used to obtain a comprehensive entity embedding based on the first structured entity embedding and the final dynamic entity embedding as the final entity embedding.

11. An electronic device, characterized in that it includes: One or more processors, a memory, and one or more computer programs, wherein the one or more computer programs are stored in the memory and the one or more computer programs include instructions that, when executed by the electronic device, cause the electronic device to perform the dynamic knowledge graph completion method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that, the computer-readable storage medium stores a computer program which, when running on a computer, causes the computer to perform the dynamic knowledge graph completion method according to any one of claims 1 to 8.

Images