Time-aware multi-graph structure tense knowledge graph completion model construction method

Abstract

The invention relates to a method for constructing a time-aware multi-graph structure tense knowledge graph completion model, which comprises a multi-type structure information encoder, according to the method, facts related to an entity to be queried on a tense knowledge graph are divided into three parts, namely facts occurring before query time, facts occurring after the query time and facts concurrent with the query time, and three different attention networks are adopted to process neighbor structure information under the three types. The information of the three different neighbor structures passes through a linear layer and then is aggregated to entity embedding; according to the multi-period time information decoder, a plurality of periodic time inserts with the period lengths being prime numbers are constructed, an attention mechanism is used for judging which period inserts the queried head entities and relations are more sensitive to so as to combine a new period insert, and finally the period insert is used as a filter to be applied to the Conv-TransE-based decoder. The multi-graph structure tense knowledge graph completion model constructed by the method has better expression ability.

Description

Technical Field

[0001] The present invention relates to the technical field of knowledge graph completion, and specifically to a method for constructing a time-aware multi-graph structure temporal knowledge graph completion model. Background Art

[0002] Static knowledge graph completion has been widely explored. TransE[1], TransR[2] and other translation-based models project entities and relations into a low-dimensional vector space and regard relations as translation vectors between entities. That is, the embedding representation of entities and relations should satisfy the formula e s +r r ≈e o (e s ,e o are the head and tail entity embedding representations, respectively, r =Representation of relation embedding). DistMult[3], CompleEx[4] and other models based on tensor decomposition decompose entities and relations into the form of low-dimensional tensor products. In addition, some studies use GCN[5] to obtain the structural information of knowledge graphs. R-GCN[6] extends GCN by combining relation-specific filters. CompGCN[7] proposes a heterogeneous graph representation algorithm for multi-relation graphs that can simultaneously learn entity and relation representations. JGAN[8] extends GCN by simultaneously extracting local graph structure features and multi-relation features. DisenKGAT[9] further improves the ability of graph neural networks to process complex data by introducing a disentangled attention mechanism to handle multiple potential relationships between entities. It distinguishes and learns the independent effects of different types of relations in the knowledge graph on the entity embedding representation after disentanglement, thereby achieving a more fine-grained modeling of complex interactions between entities. However, all of the above models are built on static knowledge graphs and cannot be used for the task of temporal knowledge graph completion (TKGC).

[0003] The TKGC interpolation task aims to infer missing facts in the past. Early methods attempted to use temporal information to extend the static knowledge graph completion (KGC) model to the temporal knowledge graph completion (TKGC) task, mainly by introducing timestamps or time intervals to capture the changes of entities and relations over time. TTransE

[10] introduced temporal embedding to extend TransE to TKG. HyTE

[11] projects entities and relations onto time-specific hyperplanes. TA-DistMult

[12] processes temporal information into time series and then learns the features between relations and time series through a long short-term memory neural network (LSTM). Graph neural networks (GNNs) have been widely used for representation learning of static knowledge graphs. In temporal knowledge graphs, this type of model extends GNNs by considering the evolution of entities and relations over time. The TeMP

[13] model uses a graph convolutional network to generate representations of entities at each timestamp and uses frequency-based gating techniques to capture the temporal characteristics of entity representations. The T-GAP

[14] model uses a two-stage graph neural network encoding for a specific subgraph of the queried TKG, focusing on the time difference between the query time and each event, and performs reasoning through a graph attention mechanism to exploit both temporal information and graph structure information.

[0004] The introspection task of temporal knowledge graphs has been widely used in many fields. For example, in the medical field, it can be used to analyze the condition and assist in medical decision-making. In personalized recommendation systems, the introspection task can recommend products, content or services that users may be interested in based on their historical behaviors and preferences. In addition, it can also be applied to expert systems, question-answering systems and other fields. The current knowledge representation learning method has achieved remarkable results in the task of static knowledge completion, but the research on temporal knowledge graph completion is far from perfect.

[0005] Early TKGC models attempted to introduce time constraints or time embedding to transition from static KGC models to TKGC models. However, such methods can only mine shallow time information and cannot be extended to "invisible" (i.e., outside the dataset) timestamps, limiting the model's expressive power. Graph neural networks have advantages in capturing the complex semantics of entities and relationships, but existing KGC models based on graph neural networks are mostly built on static KGC, ignoring time information during the convolution process, and therefore perform poorly on TKGC tasks. Summary of the invention

[0006] The purpose of the present invention is to provide a method for constructing a time-aware multi-graph structure temporal knowledge graph completion model, and the multi-graph structure temporal knowledge graph completion model constructed by this method has better expression ability.

[0007] In order to achieve the above-mentioned purpose, the technical solution adopted by the present invention is: a method for constructing a time-aware multi-graph structure temporal knowledge graph completion model, comprising:

[0008] A multi-type structural information encoder that divides facts related to the query entity on the temporal knowledge graph into three parts: facts that occurred before the query time, facts that occurred after the query time, and facts that occurred concurrently with the query time. Three different attention networks are used to process the neighbor structure information of these three types, and the three different neighbor structure information are aggregated into the entity embedding after passing through a linear layer; and

[0009] A multi-cycle temporal information decoder constructs multiple periodic temporal embeddings with prime period lengths, uses an attention mechanism to determine which periodic embeddings the queried head entities and relations are more sensitive to, and thus combines a new periodic embedding, which is finally used as a filter in a Conv-TransE-based decoder.

[0010] Furthermore, in order to learn facts that are far away from the timestamp to be predicted under limited resources, three types of neighbors are sampled, and only n facts of each type are selected for learning neighbor structure information; the sampling strategy is:

[0011] First, for the facts before the time to be predicted, after the time to be predicted, and concurrent with the time to be predicted, directly select the first n / 2 facts respectively;

[0012] Then, sort the facts in the order of their occurrence time, randomly sort the facts that are concurrent with the time to be predicted, and then generate a probability density function p according to the sequence length of the sequence using formula (1) i , using the probability density function p i Extract n / 2 entities from different types of facts;

[0013]

[0014] Among them, type represents the neighbor type, which includes three types: before the prediction time, after the prediction time, and now concurrent with the prediction time; represents all facts with o as the head entity or tail entity in the type class neighborhood, express The number of facts contained in

[0015] Finally, for these three types, the n / 2 facts obtained from the first two parts are mixed so that n facts are obtained for each type.

[0016] Furthermore, the multi-type structural information encoder adopts a multi-head scaled dot product attention mechanism to calculate the attention weight of an entity to its neighbors; for each entity, some facts with it as the tail entity are found according to the corresponding sampling method, and the attention weight of the mixed embedding representation obtained by adding the head entity embedding and the relationship embedding in these facts and the embedding representation of the query entity o is calculated, that is, e is calculated. s +r r With e o The attention weights are then used to weight the embedding representations of the query entity’s neighbor entities according to the obtained attention weights to obtain the final embedding representation.

[0017] The query, key, and value of the multi-head scaled dot-product attention mechanism are as follows:

[0018]

[0019] Among them, e o is the embedding of the entity to be queried, e s is the embedding of the head entity s connected to o, r r is the embedding of the relation r connected to o; i is the attention head; d is the dimension of the embedding representation, k =d / k, k is the number of attention heads;

[0020] Multi-head scaling dot product attention weight α i The calculation formula is as follows:

[0021]

[0022] Wherein, T represents the transpose operation;

[0023] Finally, the final embedding representation is obtained based on the calculated multi-head scaled dot product attention weights

[0024]

[0025] Among them, Π represents the splicing operation;

[0026] In order to enable the model to comprehensively and fully consider different types of structural information, three different linear layers are used to transform the structural information aggregated by the attention network. The formula is as follows:

[0027]

[0028] Among them, W before ,W after ,W now ,W e_self ∈R d×d They are linear transformation matrices respectively; the last item eo ×W self It is used to maintain the information of entity o itself, making the embedding representation of entity o more stable;

[0029] At the same time, in order to better model relationship embedding, the structural information of the relationship is aggregated. By aggregating the entities connected to the relationship at the timestamp to be predicted, the model can better understand the preference of the relationship at this timestamp, thereby achieving more accurate prediction; the formula for relationship aggregation is as follows:

[0030]

[0031] in, Right now is the set of head and tail entities of all facts involving relation r occurring at time t, is the number of entities in the set; W r_neigh ,W r_self ∈R d×d are linear transformation matrices respectively;

[0032] Through the above structural information encoding operations, the model can obtain entity and relationship embeddings with richer information, thereby improving the performance of the knowledge graph completion task.

[0033] Furthermore, the multi-cycle time information decoder uses a prime number as the cycle length, defines a set of different cycle lengths as M, and the time cycle embedding matrix used by the decoder is defined as C l , C l ∈R d×l represents a matrix with a period length of l and a dimension of d×l, l∈M; Indicates taking out the matrix C l The i-th column of , i = t%l, t is the time to be predicted, % represents the modulo operation; the period embedding vector obtained by the attention layer is defined as c;

[0034] The time period embedding is regarded as a time filter for extracting information about entities and relations at different timestamps. Its usage is shown in Equation (7). First, the query head entity is embedded into e s and relation embedding r Through a one-dimensional convolution kernel, it is used to capture the complex semantic information between entities and relations:

[0035] e s,r =δ(e s ,r r ) (7)

[0036] Among them, δ(·) represents a one-dimensional convolution operation;

[0037] The mixed embedding e obtained by the above operations,r That is, it contains both the head entity and relationship information of the query; the attention network is used to judge the embedding e obtained after one-dimensional convolution of the head entity and relationship s,r Pay more attention to which period embeddings are used to capture complex period information. The formula is as follows:

[0038]

[0039] Among them, Π represents the splicing operation, W q ,W k ,W v ∈R d×d are three different linear transformation matrices; M is the set of period lengths, It means taking the t%lth column from the embedding matrix with period length l as the period embedding vector, % is the modulo operation;

[0040] Then, the attention of the entity to the embedding at different timestamps is calculated using the following formula:

[0041]

[0042] According to the attention paid to different period embeddings, these period embeddings are aggregated to obtain the final complex period embedding:

[0043] c=α c V (10)

[0044] Through the above operations, the aggregation of complex period embedding for different queries is completed;

[0045] Then, the complex period embedding is regarded as a temporal filter, and the embedding obtained by the convolution operation is s,r Through this time filter, we filter out the information reflected by the embedding at a specific time, and multiply the filtered embedding with all entity embeddings to get the probability of different queries and all entities occurring at a specific time:

[0046] f(o|s,r,t)=soft max(e s,r ·c×E) (11)

[0047] Among them, E is the embedding matrix composed of the embedding representations of all entities.

[0048] Furthermore, the entity link prediction task query = (s, r, o, t) is regarded as a multi-classification problem, where each category corresponds to each candidate entity and the number of categories is the size of the entity set; the model is trained with the goal of minimizing the following loss function:

[0049]

[0050] Compared with the prior art, the present invention has the following beneficial effects: the existing temporal knowledge graph completion model does not comprehensively consider the impact of facts occurring in the past, present and future on entity embedding during modeling, resulting in the inaccuracy of the aggregated entity. The present invention achieves comprehensive consideration of the historical evolution and development trend of the entity by sampling facts occurring in the past, present and future respectively, so that the entity embedding can better reflect its actual significance at different times. In addition, the existing temporal knowledge graph completion model often lacks modeling for periodic patterns, especially the complex periodic patterns mentioned in the present invention. Most models can only model simple periodic information such as year, month and day, and cannot well reflect the significance of embedding representation at different times. The present invention uses an attention mechanism to realize the combination of multiple prime number periods, so that the model can adaptively combine multiple different periodic embeddings, significantly improving the expressive power of the model. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] Figure 1 It is an architecture diagram of a multi-type structural information encoder of a multi-graph structure temporal knowledge graph completion model in the implementation of the present invention;

[0052] Figure 2 It is an architectural diagram of a multi-cycle time information decoder of a multi-graph structure temporal knowledge graph completion model in an implementation of the present invention;

[0053] Figure 3 The embedding with a period of 2 and the embedding with a period of 3 in the embodiment of the present invention are combined to obtain the embedding with a period of 6;

[0054] Figure 4 : It is a (query-cycle) attention heat map in an embodiment of the present invention. DETAILED DESCRIPTION

[0055] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0056] It should be noted that the following detailed descriptions are exemplary and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0057] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0058] 1. General idea and pre-definition

[0059] This embodiment provides a method for constructing a time-aware multi-graph structure temporal knowledge graph completion model. The constructed time-aware multi-graph structure knowledge graph completion model (TAMG) mainly consists of two parts:

[0060] (1) An encoder composed of a graph attention network that can aggregate neighbor entities in three different situations - a multi-type structural information encoder: It divides the facts related to the query entity on the temporal knowledge graph into three parts: facts that occurred before the query time, facts that occurred after the query time, and facts that occurred concurrently with the query time, and uses three different graph attention networks to aggregate the neighbor structure information under these three types. The three different structural information are aggregated into entity embedding after passing through a linear layer. The architecture of the multi-type structural information encoder is shown in Figure 1. Figure 1 shown.

[0061] (2) A decoder that can decode complex periodic information - a multi-periodic time information decoder: It constructs multiple periodic time embeddings with prime period lengths, uses an attention mechanism to determine which periodic embeddings the queried head entities and relations are more sensitive to, and then combines a new periodic embedding. Finally, the periodic embedding is used as a filter in the Conv-TransE-based decoder. The architecture of the multi-periodic time information decoder is shown in Figure 2. Figure 2 shown.

[0062] The relevant definitions of the present invention are given below.

[0063] Definition 1 (Temporal Knowledge Graph, G) G = {G1, G2, …, G t}, where G t Represents a subgraph consisting of all facts that occurred at timestamp t.

[0064] Definition 2: (Entity, s, o) s, o are used to refer to the head entity and tail entity in each fact, respectively.

[0065] Definition 3 (relation, r) r is used to refer to the relation in each fact.

[0066] Definition 4: (time, t) t is used to refer to a specific timestamp.

[0067] Definition 5 (fact, Quaternion; fact set, F) Quaternion = (s, r, o, t) represents a quadruple, and all facts can constitute the fact set F.

[0068] Definition 6 (Entity Embedding, e; Entity Matrix, E) e represents entity embedding, e s ,e o Represent the embedding of entity s and o respectively; E represents the entity matrix composed of all entity embeddings.

[0069] Definition 7 (Relation Embedding, r; Relation Matrix, R) r represents the relation embedding, r r represents the embedding of relation r; R represents the relation matrix composed of all relation embeddings.

[0070] Definition 8 (Introduction) The temporal knowledge graph completion task under the introduction task is defined as: based on timestamp t, and the facts before and after timestamp t {G1,…,G t-1 ,G t ,G t+1 ,…}, predict the answer to the object query (s,r,?,t) or the subject query (?,r,o,t).

[0071] 2TAMG Model Design

[0072] 2.1 Multi-type structural information encoder

[0073] In order to obtain a more accurate embedding representation of the head entity, it is obviously necessary to learn the neighbor information of the head entity. For entities in the temporal knowledge graph, the impact of facts at different times on their structural information is obviously different, so the present invention uses a method to distinguish the impact between entities by time. The neighbors of the entity are divided into three types: before the time to be predicted, after the time to be predicted, and concurrent with the time to be predicted, because facts at different time types may have different impacts on the structural information of the entity.

[0074] In this method, different importance is given to neighbor entities in each time type. Facts before the query time may reveal the historical evolution and previous associations of the head entity; facts concurrent with the query time may imply the current state of the head entity or events that occurred at the same time; facts after the query time may indicate the future development trend or potential impact of the head entity. By using three different attention networks to process the neighbor information in these three time types, the model can effectively learn and utilize this information. Each attention network can focus on feature extraction and relationship modeling in different time types, thereby comprehensively capturing the complex relationship between the head entity and its neighbors. The advantage of this method is that it can make full use of temporal information and provide more accurate and comprehensive analysis of the head entity structure information.

[0075] In order to learn facts that are far away from the timestamp to be predicted with limited resources, the present invention samples three types of neighbors, and only selects n facts of each type for learning neighbor structure information.

[0076] 2.1.1 Sampling Strategy

[0077] The sampling strategy can be divided into the following three parts: (1) Directly select the first n / 2 facts with the smallest time difference to the time to be predicted. (2) Select n / 2 facts based on the time difference to the time to be predicted, the smaller the time difference, the greater the probability of extraction. (3) The sampling results obtained in the first two parts are mixed together.

[0078] First, for the facts before the time to be predicted, after the time to be predicted, and concurrent with the time to be predicted, directly select the first n / 2 facts respectively.

[0079] Then, sort the facts in the order of their occurrence time (randomly sort the facts that are concurrent with the time to be predicted), and then generate a probability density function p according to the sequence length of the sequence using formula (1) i , use this probability density function to extract n / 2 entities from different types of facts.

[0080]

[0081] Among them, type indicates which type of neighbor is used for aggregation. Type includes three types: before the prediction time, after the prediction time, and now concurrent with the prediction time. represents all facts with o as the head entity or tail entity in the type class neighborhood, express The number of facts contained in .

[0082] Finally, for these three types, the n / 2 facts obtained from the first two parts are mixed so that n facts are obtained for each type.

[0083] 2.1.2 Structural Information Encoding

[0084] This encoder uses the following multi-head scaled dot product (ScaledDot-Product Attention) attention mechanism to calculate the attention weight of the entity to its neighbors. For each entity, according to the corresponding sampling method, find some facts with it as the tail entity, calculate the head entity embedding and relationship embedding in these facts, and add the mixed embedding representation to obtain the attention weight of the embedding representation of the query entity o, that is, calculate e s +r r With e oThen, the embedding representations of the query entity’s neighbor entities are weighted averaged according to the weight to obtain the final embedding representation.

[0085] The query, key, and value of the multi-head scaled dot-product attention mechanism are as follows:

[0086]

[0087] Among them, e o is the embedding of the entity to be queried, e s is the embedding of the head entity s connected to o, r r is the embedding of the relation r connected to o; i is the attention head; d is the dimension of the embedding representation, k =d / k, where k is the number of attention heads.

[0088] The query, key, and value of the multi-head scaled dot-product attention mechanism are as follows:

[0089]

[0090] Among them, e o is the embedding of the entity to be queried, e s is the embedding of the head entity s connected to o, r r is the embedding of the relation r connected to o; i is the attention head; d is the dimension of the embedding representation, k =d / k, where k is the number of attention heads.

[0091] Multi-head scaling dot product attention weight α i The calculation formula is as follows:

[0092]

[0093] Wherein, T represents the transpose operation.

[0094] Finally, the final embedding representation is obtained based on the calculated multi-head scaled dot product attention weights

[0095]

[0096] Among them, Π represents the splicing operation.

[0097] In order to enable the model to comprehensively and fully consider different types of structural information, three different linear layers are used to transform the structural information aggregated by the attention network. The formula is as follows:

[0098]

[0099] Among them, W before,W after ,W now ,W e_self ∈R d×d They are linear transformation matrices respectively; the last item e o ×W self It is used to maintain the information of entity o itself and make the embedding representation of entity o more stable.

[0100] At the same time, in order to better model relationship embedding, the present invention aggregates the structural information of the relationship. By aggregating the entities connected to the relationship at the timestamp to be predicted, the model can better understand the preference of the relationship at this timestamp, thereby achieving more accurate prediction. The formula for relationship aggregation is as follows:

[0101]

[0102] in, Right now is the set of head and tail entities of all facts involving relation r occurring at time t, is the number of entities in the set; W r_neigh ,W r_self ∈R d×d are linear transformation matrices respectively.

[0103] Through the above structural information encoding operations, the model can obtain entity and relationship embeddings with richer information, thereby improving the performance of the knowledge graph completion task.

[0104] 2.2 Multi-cycle time information decoder

[0105] The periods of actual occurrence are often different, and how to fit these different periods is the research focus of this invention. Since prime numbers cannot be divided by 1 and are not divisible by themselves, using prime numbers as the period length can maximize the combination of multiple different periods. Figure 3 As shown, the embedding with a prime number of 2 as the period length and the embedding with a prime number of 3 as the period can be combined to form an embedding with a period of 6. Therefore, the present invention proposes a period information extraction decoder with a prime number as the period length, which regards the time period embedding as a filter for extracting information about entities and relationships at different timestamps. The set of different period lengths is defined as M, and the time period embedding matrix used by the decoder is defined as C in uppercase bold letters. l , C l ∈R d×l represents a matrix with a period length of l(l∈M) and a dimension of d×l; Indicates taking out the matrix C l The i-th column of , i = t%l, t is the time to be predicted, % represents the modulo operation; the cycle embedding vector finally obtained by the attention layer is defined as c with lowercase bold letters.

[0106] Each natural number can be uniquely expressed as the product of prime numbers. In order to enable multiple cycle embeddings to be combined into more cycle embeddings with different cycle lengths, the present invention provides the following cardinality cycle embeddings, and they are all composed of prime numbers. In the data sets of YAGO and WIKI, which are in units of years, the cycle lengths of the initial cycle embeddings used are 1, 2, 3, 5, 7 and 11, i.e., M = {1, 2, 3, 5, 7, 11}, respectively, which can form a variety of different cycles, such as financial crises that may occur every few years or decades. Combining these cardinality cycles can obtain a variety of different cycles to better fit the occurrence law of such events. In the data set of ICEWS, which is in units of days, the cycles of the initial cycle embeddings used are 1, 2, 3, 5, 7, 11 and 31, i.e., M = {1, 2, 3, 5, 7, 11, 31}, respectively. If you want to achieve a monthly cycle, then use 2 × 3 × 5 to get it. At the same time, because 31 is a prime number that cannot be obtained by multiplying other numbers, an additional cycle embedding with 31 as a cycle is added.

[0107] The present invention regards period embedding as a time filter, and its usage is shown in formula (7). First, the query head entity is embedded into e s and relation embedding r Through a one-dimensional convolution kernel, it is used to capture the complex semantic information between entities and relations:

[0108] e s,r =δ(e s ,r r ) (7)

[0109] Here, δ(·) represents a one-dimensional convolution operation.

[0110] The mixed embedding e obtained by the above operation s,r That is, it contains the query head entity and relationship information at the same time. The sensitivity of head entities and relationships to different types of periodic embeddings may be different. For example, some events may occur once every two weeks, so it will pay more attention to embeddings with period lengths of 2 and 7. Therefore, the present invention uses an attention network to determine the embedding e obtained after the head entity and relationship are subjected to one-dimensional convolution s,r Pay more attention to which period embeddings are used to capture complex period information. The formula is as follows:

[0111]

[0112]

[0113] Among them, Π represents the splicing operation, W q ,W k ,Wv ∈R d×d are three different linear transformation matrices. M is the set of period lengths, It means that the t%lth column is taken out from the embedding matrix with a period length of l as the period embedding vector, and % is the modulo operation;

[0114]

[0115] Where Π represents the splicing operation, W q ,W k ,W v ∈R d×d are three different linear transformation matrices. M is the set of period lengths, It means that from the embedding matrix with period length l, the t%lth column is taken as the period embedding vector, and % is the modulo operation. 2 and C 3 For example, through the remainder operation, when t = 1, the embedded and When t = 2, remove the embedded and Similarly, when t=7, the embedded and When t=8, the embedded and It can be observed that the same embedding is obtained at t=1 and t=7, and the same embedding is obtained at t=2 and t=8, which means that an embedding with a period of 6 is successfully achieved by combining two embeddings with periods of 2 and 3.

[0116] Next, use “Scaled Dot-Product Attention” to calculate the entity’s attention to the embedding at different timestamps. The calculation formula is as follows:

[0117]

[0118] According to the attention paid to different period embeddings, these period embeddings are aggregated to obtain the final complex period embedding:

[0119] c=α c V (10)

[0120] Through the above operations, the aggregation of complex period embeddings for different queries is completed; for example, some queries often repeat with a period of 15, so it may be necessary to embed C with a period of 3. 3 and embedded C with a period of 5 5 is more interested; some queries often repeat with a period of 14, so it is more interested in embedding C with a period of 2 2 and embedded C with a period of 77 More interested.

[0121] Then, the complex period embedding is regarded as a temporal filter, and the embedding e obtained by the previous convolution operation is s,r Through this time filter, we can filter out the information reflected by the embedding at a specific time. By multiplying the filtered embedding with all entity embeddings, we can get the probability of different queries and all entities occurring at a specific time:

[0122] f(o|s,r,t)=softmax(e s,r ·c×E) (11)

[0123] Among them, E is the embedding matrix composed of the embedding representations of all entities.

[0124] 2.3 Loss Function

[0125] The entity link prediction task query = (s, r,?, t) can be regarded as a multi-classification problem, where each category corresponds to each candidate entity, and the number of categories is the size of the entity set. The goal of model training is to minimize the following loss function:

[0126]

[0127] 3 Experiments

[0128] 3.1 Experimental Setup

[0129] 3.1.1 Dataset

[0130] In order to comprehensively and accurately evaluate the performance of the model, the following different datasets are used to evaluate the TAMG model, namely YAGO11k, Wikidata12k, and ICEWS14. The sampling frequencies of them when constructing the knowledge graph are one year, one year, and one day, respectively. Among them, YAGO11k is a dataset based on YAGO3

[15] , which has rich entity and relationship information; Wikidata12k is a dataset based on Wikidata

[16] . Wikidata is a large, open knowledge base maintained by volunteers from all over the world. It contains a large number of entities, attributes, and relationships, and is constantly updated; the ICEWS

[17] dataset is a large dataset focusing on global crisis warning. It is mainly used to capture and process millions of data from digital news media, social media, and other sources to predict, track, and respond to events around the world. ICEWS14 contains some facts that occurred in ICEWS in 2014. The details of the dataset are shown in Table 1:

[0131] Table 1 Dataset details

[0132]

[0133]

[0134] 3.1.2 Evaluation Metrics

[0135] In the knowledge graph entity prediction task, mean reciprocal rank (MRR) and the hit rate of the top N (Hit@N, N=1, 3, 10) are commonly used evaluation indicators to measure the performance of model link (entity) prediction.

[0136] For each test sample, the model predicts a set of candidate entities (answers), and MRR considers the ranking position of the correct entity in the candidate entity. The MRR value is obtained by summing the reciprocal ranking of the correct entities in all test samples and dividing it by the total number of test samples. The larger the MRR value, the better the model effect.

[0137] Hit@N is an indicator to measure whether there is a correct entity among the first N candidate entities predicted by the model. The calculation method is: for each test sample, if the first N candidate entities predicted by the model contain the correct entity, the Hit@N score of the test sample is 1, otherwise it is 0. The final evaluation indicator is the average of the Hit@N scores of all test samples. The larger the Hit@N value, the better the model effect.

[0138] In order to more accurately evaluate the performance of the model, it is necessary to filter the candidate entities. This patent follows previous work, and all experiments filter the candidate entities under time-aware filtering conditions, that is, filtering out all correct entities except the query object from the candidate answers at each timestamp to more fairly evaluate the performance of the model.

[0139] 3.1.3 Baseline

[0140] The baseline of this embodiment includes some classic static KGC and TKGC models, as well as some latest TKGC models. The static KGC includes TransE[1], DistMult[3] and ComplEx[4]; the classic TKGC models include TTransE

[10] , HyTE

[11] , TA-DistMult

[12] and DE-SimplE

[18] , as well as some latest works such as TeRo

[19] , TeLM

[20] , TILP

[21] , SANe

[22] , BoxTE

[23] and Joint-MT

[24] .

[0141] 3.1.4 Experimental details

[0142] This model is implemented using Pytorch, and all experiments are run on RTX2080. The initial parameters of the model are initialized using random numbers that conform to the normal distribution, and the optimizer used by the model is Adam. In order to prevent overfitting, the model adds a Dropout layer at a specific position of the decoder. The parameter settings of the model on each data set are shown in Table 2:

[0143] Table 2 Experimental settings

[0144]

[0145]

[0146] 3.2 Experimental Results

[0147] 3.2.1 Link Prediction

[0148] Link prediction experiment is an important task in the field of knowledge graph completion, which aims to predict missing entity relationships or links in the knowledge graph. The model proposed in this paper achieves the best performance compared with other baselines, verifying the effectiveness of this method. The specific experimental results are shown in Tables 3 and 4. Bold indicates the best result, and underlined indicates the second best result. The performance data of other models in the experiment are all from published papers.

[0149] Table 3. Experimental results of YAGO and Wikidata link prediction

[0150]

[0151]

[0152] Table 4 ICEWS14 link prediction experimental results

[0153]

[0154] Compared with the traditional static knowledge graph model, the present invention successfully optimizes the performance of link prediction by modeling time information, achieving significant performance improvement. Compared with the model that directly integrates time coding into the traditional knowledge graph, this model uses periodic embedding to encode time, and comprehensively considers the impact of facts at different times on entities, making the structural information of entities more comprehensive and the performance improvement is also very significant. Compared with the latest temporal knowledge graph model, the multi-type structural information encoder used in this embodiment can comprehensively and accurately model the impact of facts at different times on entities, while also reflecting the preference of relationships at specific times, achieving better results. In addition, more time periods are combined by periodic embedding with prime numbers as periods, thereby enhancing the performance of the decoder. Finally, this model achieved the best performance on each data set and achieved significant performance improvement.

[0155] 3.2.2 Time Scalability

[0156] An obvious benefit of using period embedding to simulate embedded period information is that it can model unknown time very well. For example, although the fact that occurred on January 6 may be missing, it can be composed of 6 with embeddings of period 2 and period 3, thereby achieving modeling of time that has never appeared. In order to verify this feature, this embodiment conducted the experiment shown in Table 5 to prove that this model can be extended to unknown time and has good time scalability.

[0157] Table 5 ICEWS14 invisible timestamp link prediction results

[0158]

[0159] Following the method of Geol et al.

[18] , the facts that appear on the 5th, 15th, and 25th of each month in the ICEWS4 dataset are excluded from the training set, and these excluded facts are randomly divided into a validation set and a test set. Finally, the dataset is named ICEWS4-unseen. This example compares the model with several recent baselines that have used this dataset. The results show that the model can make good use of existing time information to infer unknown time information, and the performance achieved exceeds these baselines in all indicators.

[0160] GAP

[26] uses discrete time displacement embedding and trains different embeddings for different time differences. If some time differences appear less frequently in the training set, the corresponding time displacement embedding cannot be well trained, which will greatly affect the performance of the model. For models such as TComplEx and TNTComplex proposed by Lacroix et al.

[25] that require time embedding for each timestamp, their performance is more significantly reduced compared to the original ICEWS14 dataset. The periodic embedding with a prime period adopted by this model can ensure that the timestamp is split into multiple periodic embeddings of different period lengths, thereby increasing the probability of each periodic embedding being trained, thereby reducing the performance degradation caused by lack of information.

[0161] 3.2.3 Irregular time

[0162] This example samples KG snapshots from the original ICEWS14 dataset. The time interval between two adjacent snapshots can be randomly assigned to 1, 2, 3, or 4. In this way, a dataset simulating the observation and collection of TKG data at irregular time stamps is created, named ICEWS14-irregular, to verify that the model can be well generalized to irregular time stamps.

[0163] Table 6 ICEWS14 irregular interval timestamp link prediction results

[0164]

[0165] From Table 6, we can observe that the lower performance of TeMP compared with this model is because it uses the graph structure information of a fixed short time span to learn the embedding, and cannot capture a lot of graph structure information outside this time span. T-GAP uses time displacement embedding to encode different time differences. However, the experimental results show that this model is better than T-GAP in distinguishing irregular time intervals because the period information extraction decoder of this model has multiple periods to simulate the representation of multiple time differences. SANe divides the time embedding into three types of embeddings: year, month, and day according to the timestamp, but compared with this model, the time information it can capture is more limited. This model can capture multiple period information, and can also comprehensively consider the impact of multiple types of neighbor information on entity embedding.

[0166] 3.3 Ablation Experiment

[0167] In order to verify the effectiveness of each module, this embodiment conducted an ablation experiment, and the results are shown in Table 7, where w / ocycle means that the cycle information extraction module in the decoder is removed; w / o fact_before and w / o fact_after respectively mean that the modeling of facts occurring before the prediction time and facts occurring after the prediction time in the encoder is removed; w / o rel_agg means that the relational structure information encoding in the encoder is removed; w / o cycle_att means that the attention mechanism in the cycle information extraction module is removed, and the attention mechanism is directly replaced by weighted average embedding of each cycle; w / o prime means that ordinary natural number cycle embedding is used to replace prime number cycle embedding. The experimental results are shown in the following table:

[0168] Table 7 Ablation experiment

[0169]

[0170]

[0171] After removing the modeling of facts that occurred before the prediction time and after the prediction time in the encoder, the performance of the model has declined significantly. Through this experiment, we can draw the following conclusions: Facts before the query time can reveal the historical evolution and previous associations of the head entity, thereby helping the model to obtain a more accurate embedding representation; facts after the query time indicate the future development trend or potential impact of the head entity. By comprehensively considering this trend and impact, the accuracy of the embedding representation can also be effectively improved. Considering these two embedding representations at the same time can make the final embedding representation used for prediction more accurate and comprehensive, thereby improving the performance of the link prediction task.

[0172] After removing the relational structure encoding part in the encoder, the performance of the model has declined to a certain extent. This is because aggregating entities that have interacted in the current time subgraph for relations can help understand the preference on the current time subgraph, thereby improving the performance of the link prediction task.

[0173] In the period information extraction module, the weighted average of each period embedding is used to replace the attention mechanism, and the performance of the model also shows a particularly significant decline. This shows that different entities and relations do have different sensitivities to different period embeddings. The use of the attention mechanism can better determine which period embedding length the query head entity and relationship is more sensitive to, so as to use these period embeddings to extract more accurate embeddings at the timestamp to be predicted, thereby improving the performance of the link prediction task.

[0174] When the number of periods embedded in the cycle is replaced from prime number periods 1, 3, 5, 7, 11 to 1, 2, 3, 4, 5, it is found that the performance of the model has declined to a certain extent. This is because when the period length is a prime number, it can maximize the combination of multiple periods, making the period information extraction decoder representable richer periods, thereby improving the performance of the model.

[0175] After removing the period information extraction module in the decoder, the model's performance on each data set dropped significantly, which illustrates the effectiveness of the period information extraction module proposed in the TAMG model. This method can indeed extract the information displayed by entity and relationship embeddings at different timestamps, thereby effectively improving the accuracy of link prediction tasks.

[0176] 3.4 Case Study

[0177] We randomly select 6 queries in the ICEWS14 dataset and plot their attention heat maps for different period embeddings, as shown in Figure 4 As shown; the ID-name comparison table of the queried entity-relationship pairs is shown in Table 8, where the “-” sign in front of the relationship indicates an inverse relationship.

[0178] Table 8 ID-name comparison table

[0179]

[0180] It can be observed that the model is more sensitive to the periodic embedding with a period length of 1. This is because the period of the embedding is 1, which means that the model will be trained on this embedding regardless of the timestamp. At the same time, since many events will repeat on the second day, the attention value of this periodic embedding will be significantly higher than other values.

[0181] It can be clearly observed that the attention values ​​of period embeddings with a period length other than 1 are clearly distinguishable. Most queries pay more attention to period embeddings with a period length of 7, which is consistent with the reality because many events occur repeatedly in weeks.

[0182] In addition, the queries with entity-relationship numbers 4962-344, 1973-286, 5901-56, and 1054-114 have relatively high attention in the periodic embeddings of various different period lengths. Obviously, the relations Make an appeal or request and Make statement appearing in these four queries occur frequently in real life, so the period is also relatively complex. This also verifies the original intention of the invention to design the periodic attention, that is, to combine more periodic embeddings of different lengths through periodic embeddings of different period lengths to improve the expressiveness of the model.

[0183] The method for constructing a time-aware multi-graph structured temporal knowledge graph completion model provided by the present invention adopts a novel structural information encoder in the field of temporal knowledge graph completion, which models the neighbor structure information of the entity from three different time dimensions to make the structural information of the entity more comprehensive and accurate. At the same time, the encoder can also model the structural information of the relationship at the time to be predicted, so as to identify the preference of the relationship at the time to be predicted. The periodic information extraction decoder proposed in the present invention regards periodic embedding as a filter and uses prime numbers as the period length to combine more types of periodic embeddings. In this way, the model can effectively use periodic embeddings to filter out the representation of the query at the corresponding query time.

[0184] The references cited in this article are as follows:

[0185] [1]Bordes A,Usunier N,Garcia-Duran A,et al.Translating embeddings formodeling multi-relational data[J].Advances in neural information processingsystems,2013,26.

[0186] [2]Lin Y,Liu Z,Sun M,et al.Learning entity and relation embeddingsfor knowledge graph completion[C] / / Proceedings ofthe AAAI conference onartificial intelligence.2015,29(1).

[0187] [3]Yang B,Yih W,He X,et al.Embedding entities and relations forlearning and inference in knowledge bases[J].arXiv preprint arXiv:1412.6575,2014.

[0188] [4]Trouillon T,Welbl J,Riedel S,et al.Complex embeddings for simplelink prediction[C] / / International conference on machine learning.PMLR,2016:2071-2080.

[0189] [5]Duvenaud D K,Maclaurin D,Iparraguirre J,et al.Convolutionalnetworks on graphs for learning molecular fingerprints[J].Advances in neuralinformation processing systems,2015,28.

[0190] [6]Schlichtkrull M,Kipf T N,Bloem P,et al.Modeling relational datawith graph convolutional networks[C] / / The Semantic Web:15th InternationalConference,ESWC 2018,Heraklion,Crete,Greece,June 3–7,2018,Proceedings15.Springer International Publishing,2018:593-607.

[0191] [7]Vashishth S,Sanyal S,Nitin V,et al.Composition-based multi-relational graph convolutional networks[J].arXiv preprint arXiv:1911.03082,2019.

[0192] [8]Huang J,Lu T,Zhu J,et al.Multi-relational knowledge graphcompletion method with local information fusion[J].Applied Intelligence,2022,52(7):7985-7994.

[0193] [9]Wu J,Shi W,Cao X,et al.DisenKGAT:Knowledge graph embedding withdisentangled graph attention network[C] / / Proceedings ofthe 30th ACMinternational conference on information&knowledge management.2021:2140-2149.

[0194]

[10] Leblay J,Chekol M W.Deriving validity time in knowledge graph[C] / / Companion proceedings ofthe the web conference 2018.2018:1771-1776.

[0195]

[11] Dasgupta S S,Ray S N,Talukdar P P.HyTE:Hyperplane-basedTemporally aware Knowledge Graph Embedding[C] / / EMNLP.2018:2001-2011.

[0196]

[12] García-Durán A, S,Niepert M.Learning sequence encoders fortemporal knowledge graph completion[J].arXiv preprint arXiv:1809.03202,2018.

[0197]

[13] Wu J,Cao M,Cheung J C K,et al.Temp:Temporal message passing fortemporal knowledge graph completion[J].arXiv preprint arXiv:2010.03526,2020.

[0198]

[14] Jung J,Jung J,Kang U.Learning to walk across time forinterpretable temporal knowledge graph completion[C] / / Proceedings of the 27thACM SIGKDD Conference on Knowledge Discovery&Data Mining.2021:786-795.

[0199]

[15] Mahdisoltani F,Biega J,Suchanek F.Yago3:A knowledge base frommultilingual wikipedias[C] / / 7th biennial conference on innovative datasystems research.CIDR Conference,2014.

[0200]

[16] Erxleben F,Günther M, M,et al.Introducing Wikidata to thelinked data web[C] / / International semantic web conference.Berlin,Germany:Springer,2014:50-65.

[0201]

[17] Boschee E,Lautenschlager J,O’Brien S,et al.ICEWS Coded Event Data[M / OL].[2021-01-22].https: / / doi.org / 10.7910 / DVN / 28075.

[0202]

[18] Goel R,Kazemi S M,Brubaker M,et al.Diachronic embedding fortemporal knowledge graph completion[C] / / Proceedings oftheAAAI conference onartificial intelligence.2020,34(04):3988-3995.

[0203]

[19] Xu C,Nayyeri M,Alkhoury F,et al.TeRo:A time-aware knowledge graphembedding via temporal rotation[J].arXiv preprint arXiv:2010.01029,2020.

[0204]

[20] Xu C,Chen Y Y,Nayyeri M,et al.Temporal knowledge graph completionusing a linear temporal regularizer and multivector embeddings[C] / / Proceedings of the 2021Conference of the North American Chapter of theAssociation for Computational Linguistics:Human Language Technologies.2021:2569-2578.

[0205]

[21] Xiong S,Yang Y,Fekri F,et al.Tilp:Differentiable learningoftemporal logical rules on knowledge graphs[J].arXivpreprint arXiv:2402.12309,2024.

[0206]

[22] LiY,Zhang X,Zhang B,et al.Each snapshotto each space:Spaceadaptation for temporal knowledge graph completion[C] / / International SemanticWeb Conference.Cham:Springer International Publishing,2022:248-266.

[0207]

[23] Messner J,Abboud R,Ceylan I I.Temporal knowledge graph completionusing box embeddings[C] / / Proceedings oftheAAAI Conference onArtificialIntelligence.2022,36(7):7779-7787.

[0208]

[24] Zhang, Fu, et al. "Joint framework for tensor decomposition-basedtemporal knowledge graph completion." Information Sciences 654(2024):119853.

[0209]

[25] Lacroix T,Obozinski G,Usunier N.Tensor decompositions fortemporal knowledge base completion[J].arXiv preprint arXiv:2004.04926,2020.

[0210] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0211] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0212] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0213] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0214] The above is only a preferred embodiment of the present invention, and does not limit the present invention in other forms. Any technician familiar with the profession may use the above disclosed technical content to change or modify it into an equivalent embodiment with equivalent changes. However, any simple modification, equivalent change and modification made to the above embodiment according to the technical essence of the present invention without departing from the technical solution of the present invention still belongs to the protection scope of the technical solution of the present invention.

Claims

1. A method for constructing a time-aware multi-graph structure temporal knowledge graph completion model, characterized in that: include: A multi-type structural information encoder that divides facts related to the query entity on the temporal knowledge graph into three parts: facts that occurred before the query time, facts that occurred after the query time, and facts that occurred concurrently with the query time. Three different attention networks are used to process the neighbor structure information of these three types, and the three different neighbor structure information are aggregated into the entity embedding after passing through a linear layer; and A multi-cycle temporal information decoder constructs multiple periodic temporal embeddings with prime period lengths, uses an attention mechanism to determine which periodic embeddings the queried head entities and relations are more sensitive to, and thus combines a new periodic embedding, which is finally used as a filter in a Conv-TransE-based decoder.

2. The method for constructing a time-aware multi-graph structure temporal knowledge graph completion model according to claim 1 is characterized in that: In order to learn facts that are far away from the timestamp to be predicted with limited resources, three types of neighbors are sampled, and only n facts of each type are selected for learning neighbor structure information; the sampling strategy is: First, for the facts before the time to be predicted, after the time to be predicted, and concurrent with the time to be predicted, directly select the first n / 2 facts respectively; Then, sort the facts in the order of their occurrence time, randomly sort the facts that are concurrent with the time to be predicted, and then generate a probability density function p according to the sequence length of the sequence using formula (1) i , using the probability density function p i Extract n / 2 entities from different types of facts; Among them, type represents the neighbor type, which includes three types: before the prediction time, after the prediction time, and now concurrent with the prediction time; represents all the facts with o as the head entity or tail entity in the type class neighborhood, express The number of facts contained in Finally, for these three types, the n / 2 facts obtained from the first two parts are mixed so that n facts are obtained for each type.

3. The method for constructing a time-aware multi-graph structure temporal knowledge graph completion model according to claim 1 is characterized in that: The multi-type structural information encoder adopts a multi-head scaled dot product attention mechanism to calculate the attention weight of an entity to its neighbors; for each entity, some facts with it as the tail entity are found according to the corresponding sampling method, and the attention weight of the hybrid embedding representation and the embedding representation of the query entity o is obtained by adding the head entity embedding and the relationship embedding in these facts, that is, calculating e s +r r With e o The attention weight of Then, the neighbor entity embedding representations of the query entity are weighted averaged according to the obtained attention weights to obtain the final embedding representation; The query, key, and value of the multi-head scaled dot-product attention mechanism are as follows: Among them, e o is the embedding of the entity to be queried, e s is the embedding of the head entity s connected to o, r r is the embedding of the relation r connected to o; i is the attention head; d is the dimension of the embedded representation, dk = d / k, k is the number of attention heads; Multi-head scaling dot product attention weight α i The calculation formula is as follows: Wherein, T represents the transpose operation; Finally, the final embedding representation is obtained based on the calculated multi-head scaled dot product attention weights Among them, Π represents the splicing operation; In order to enable the model to comprehensively and fully consider different types of structural information, three different linear layers are used to transform the structural information aggregated by the attention network. The formula is as follows: Among them, W before , W after , W now , W e_self ∈R d×d are linear transformation matrices respectively; the last item e o ×W self It is used to maintain the information of entity o itself, making the embedding representation of entity o more stable; At the same time, in order to better model relationship embedding, the structural information of the relationship is aggregated. By aggregating the entities connected to the relationship at the timestamp to be predicted, the model can better understand the preference of the relationship at this timestamp, thereby achieving more accurate prediction; the formula for relationship aggregation is as follows: in, Right now is the set of head and tail entities of all facts involving relation r occurring at time t, is the number of entities in the set; W r_neigh , W r_self ∈R d×d are linear transformation matrices respectively; Through the above structural information encoding operations, the model can obtain entity and relationship embeddings with richer information, thereby improving the performance of the knowledge graph completion task.

4. The method for constructing a time-aware multi-graph structure temporal knowledge graph completion model according to claim 1, characterized in that: The multi-cycle time information decoder uses a prime number as the cycle length, defines a set of different cycle lengths as M, and the time cycle embedding matrix used by the decoder is defined as C l , C l ∈R d×l represents a matrix with a period length of l and a dimension of d×l, l∈M; Indicates taking out the matrix C l The i-th column of , i = t%l, t is the time to be predicted, % represents the modulo operation; the period embedding vector obtained by the attention layer is defined as c; The time period embedding is regarded as a time filter for extracting information about entities and relations at different timestamps. Its usage is shown in Equation (7). First, the query head entity is embedded into e s and relation embedding r Through a one-dimensional convolution kernel, it is used to capture the complex semantic information between entities and relations: And s，r =δ(e s ,r r ) (7) Among them, δ(·) represents a one-dimensional convolution operation; The mixed embedding e obtained by the above operation s，r That is, it contains both the head entity and relationship information of the query; the attention network is used to judge the embedding e obtained after one-dimensional convolution of the head entity and relationship s，r Pay more attention to which period embeddings are used to capture complex period information. The formula is as follows: Among them, Π represents the splicing operation, W q , W k , W v ∈R d×d are three different linear transformation matrices; M is the set of period lengths, It means taking the t%lth column from the embedding matrix with period length l as the period embedding vector, % is the modulo operation; Then, the attention of the entity to the embedding at different timestamps is calculated using the following formula: According to the attention paid to different period embeddings, these period embeddings are aggregated to obtain the final complex period embedding: c=a c V (10) Through the above operations, the aggregation of complex period embedding for different queries is completed; Then, the complex period embedding is regarded as a temporal filter, and the embedding obtained by the convolution operation is s,r Through this time filter, we filter out the information reflected by the embedding at a specific time, and multiply the filtered embedding with all entity embeddings to get the probability of different queries and all entities occurring at a specific time: f(o|s,r,t)=soft max(e s，r ·c×E) (11) Among them, E is the embedding matrix composed of the embedding representations of all entities.

5. The method for constructing a time-aware multi-graph structure temporal knowledge graph completion model according to claim 1, characterized in that: The entity link prediction task query = (s, r, o, t) is regarded as a multi-classification problem, where each category corresponds to each candidate entity, and the number of categories is the size of the entity set; The model is trained with the goal of minimizing the following loss function:

Images