A block decomposition-based temporal knowledge graph completion method

Abstract

The application discloses a kind of based on block decomposition's time sequence knowledge graph completion method, make it effectively improve the accuracy of missing fact prediction in large-scale complex scene.This application is first divided into several knowledge sets according to the timestamp mark in time sequence knowledge graph, and constructs several knowledge subgraphs by the knowledge in each knowledge set, to provide support for finding the dependent information between different knowledge subgraphs to complete missing knowledge;Second, the knowledge subgraph is input into the block decomposition model, and a dual factor matrix and a position coding core tensor are constructed to obtain the embedding vector representation of the knowledge subgraph to construct the potential relationship between different knowledge;Finally, the confidence score of missing knowledge is calculated according to the embedding vector representation of the knowledge subgraph, and the knowledge with the largest confidence score is added to the time sequence knowledge graph to improve the accuracy of predicting missing facts.The application can make more accurate completion to the incomplete time sequence knowledge graph.

Description

TECHNICAL FIELD

[0001] The present application relates to a block decomposition-based time sequence knowledge graph completion method, which can effectively improve the accuracy of predicting missing facts in large-scale complex scenarios. BACKGROUND

[0002] The time sequence knowledge graph is a graph-structured representation of time-sensitive facts, and each fact can be represented as a quadruple "(subject entity, relation, object entity, timestamp)". The time sequence knowledge graph completion aims to answer queries about missing facts, mainly in the form of "(subject entity, relation,?, timestamp)".

[0003] In order to complete the missing facts, one challenge is to comprehensively explore the relevant existing facts, which carry relevant information about the missing facts to some extent. These facts are usually associated with timestamps to indicate their valid time. Since additional learning of timestamps is required, effective use of these timestamp characteristics is a key problem in the process of time sequence knowledge graph completion, which is referred to as expression characteristics.

[0004] There are two types of models that can model expression characteristics, namely time operation guided models and time vector guided models. The first type of model (e.g. HyTE, TeLM, TeRo) uses some specific operations to combine time information in entity and relation embeddings, such as encoding time into specific spaces to extract time information (i.e. hyperplane space or complex space), and finally mining expression characteristics from time embedding information to complete missing facts. This model essentially does not directly use time information, and inevitably ignores some useful expression characteristics, such as the position change of timestamps between different facts. Therefore, the time vector guided model (e.g. TComplEX, ChronoR, TuckERT) directly generates time embeddings for time information and integrates them as additional information with original fact embeddings, which is more advantageous and flexible than the time operation guided model.

[0005] However, existing methods all ignore the representation type diversity and position-dependent variability of the existence of timestamp expression characteristics. Representation type diversity refers to the diversity of timestamp types in the dataset. For example, there are 4 types of time representation in Wikipedia, of which 3 are in the form of time period, including complete time annotation, only start time and only end time; the other is in the form of time point. Different timestamp types of known facts provide important information for the inference of unknown facts.

[0006] In addition, there is also static knowledge that has always been true, which can provide certain background knowledge for entities in the time sequence knowledge graph, and ignoring them may cause the completion result to deviate. However, previous work can only encode single-granularity time information, and cannot handle different timestamp types of facts at the same time, thereby affecting the comprehensive evaluation of unknown facts. Position-dependent variability refers to the position variability and mutual dependence of expression characteristics. We observe that previous work only cares about whether a fact is true or false, and ignores the time position information of different facts. We believe that the order and distance of the occurrence time of different facts have certain dependence, and are particularly important for the completion of unknown facts. SUMMARY

[0007] In order to solve the above-mentioned deficiencies in the prior art, unlike the existing method of modeling from the fact characteristics, the present application addresses the above two challenges from the perspective of time characteristics.

[0008] The main idea of the present application is as follows: first, the present application is divided into several knowledge sets according to the timestamp annotations in the time sequence knowledge graph, and several knowledge sub-graphs are constructed through the knowledge in each knowledge set, to provide support for finding the dependency information between different knowledge sub-graphs to complete the missing knowledge; second, the knowledge sub-graph is input into the block decomposition model, and a dualization factor matrix and a position encoding core tensor are constructed to obtain the embedding vector representation of the knowledge sub-graph to construct the potential relationship between different knowledge; finally, the confidence score of the missing knowledge is calculated according to the embedding vector representation of the knowledge sub-graph, and the knowledge with the largest confidence score is added to the time sequence knowledge graph to improve the accuracy of predicting missing facts.

[0009] The present application has the beneficial effects that: the present application transfers the time sequence knowledge graph completion problem from modeling for entities to modeling for timestamps, and through the introduction of the block decomposition model and the intuitive modeling of different types and different positions of timestamps on the factor matrix and the core tensor, the time sequence knowledge graph completion task is completed under the condition of ensuring the accuracy. The present application uses dual vectors and position encoding technology to process the factor matrix and the core tensor of the block decomposition model respectively, comprehensively evaluates different time types and different position facts, and further improves the sensitivity to timestamps, so as to make more accurate completion to the incomplete time sequence knowledge graph. BRIEF DESCRIPTION OF DRAWINGS

[0010] Figure 1 An example graph of the dual factor matrix method in one embodiment;

[0011] Figure 2 An example graph of the position encoding method in one embodiment. DETAILED DESCRIPTION

[0012] The present application will be further described below in conjunction with the drawings and embodiments.

[0013] The application comprises the following steps:

[0014] Step one: divide into several knowledge sets according to the timestamp label in the time sequence knowledge graph, and construct a knowledge subgraph through the knowledge in each knowledge set, to provide support for finding the dependency information between different knowledge subgraphs to complete the missing knowledge.

[0015] In combination with the data set characteristics of the time sequence knowledge graph, the time sequence knowledge graph is divided into knowledge subgraphs G={G1, G2,... G t}, t represents the number of subgraphs, and is mainly composed of four parts, namely G t ={n, r, v, τ}. Wherein the head entity n∈N, the relationship r∈R, the tail entity v∈V, and the timestamp τ∈T. N is the set of the head entity n, R is the set of the relationship r, V is the set of the tail entity v, and T is the set of the timestamp τ.

[0016] Step two: input the knowledge subgraph into the block decomposition model, and construct the dual factor matrix and the position coding core tensor, to obtain the embedding vector representation of the knowledge subgraph to construct the potential relationship between different knowledge.

[0017] Firstly, design a block decomposition model X, and input the knowledge subgraph into the block decomposition model, and the output is several initial member tensors {X1, X2,... X n} decomposed, wherein each member tensor is composed of a core tensor and three factor matrices, that is, X≈Z×1N×2R×3V, wherein is the core tensor, and are the factor matrices of the head entity, the relationship and the tail entity, and × n represents n-modulo product.

[0018] Then, construct the dual relationship vector representation factor matrix, wherein r a ,r b ∈R and τ a ,τ b ∈T represent the relationship r a at the start time τ a and the relationship r b at the end time τ b .

[0019] Finally, construct the position coding embedding representation core tensor. The application sorts the timestamps in T in ascending order, and obtains T=t1, t2,... t nThe time-ordered probability is formed after position encoding processing of the sorted time stamps. Based on the time-ordered probability, cross-entropy is introduced as a time-ordered constraint on the time stamp embedding, further aggregating and distinguishing the dependency relationship between different time stamps to fully utilize the time stamp information to complete the unknown facts.

[0020] Step three: calculate the confidence score of the missing knowledge according to the embedding vector representation of the knowledge subgraph, and add the knowledge with the largest confidence score to the time sequence knowledge graph. First, sort the possible entities in the missing fact in descending order of probability. Then, the confidence score is used to evaluate the possible entities. Finally, after several training convergences, the entity with the highest probability is taken as the predicted missing entity, and it is completed to the time sequence knowledge graph, and the obtained is the completed time sequence knowledge graph.

[0021] Embodiment:

[0022] Step one: divide into several knowledge sets according to the time stamp annotation in the time sequence knowledge graph, and construct a knowledge subgraph through the knowledge in each knowledge set, which provides support for finding the dependency information between different knowledge subgraphs to complete the missing knowledge.

[0023] In some embodiments, the knowledge subgraph can mainly consist of four parts, defined as:

[0024] G={G1,G2,…G t} (1)

[0025] G t ={n,r,v,τ} (2)

[0026] Where t represents the number of subgraphs, the head entity n∈N, the relationship r∈R, the tail entity v∈V, and the timestamp τ∈T. N is the set of head entities n, R is the set of relationships r, V is the set of tail entities v, and T is the set of timestamps τ.

[0027] Step two: input the knowledge subgraph into the block decomposition model, and construct the dual factor matrix and the position encoded core tensor to obtain the embedding vector representation of the knowledge subgraph to construct the potential connection between different knowledge.

[0028] In a preferred embodiment

[0029] First, the constructed block decomposition model is a tensor decomposition model, mainly consisting of a core tensor and three factor matrices, defined as:

[0030] X≈Z×1N×2R×3V (3)

[0031] Where is the core tensor, and is the factor matrix for head entity, relation and tail entity, and n denotes n-mode product.

[0032] Then, the dual relation vector representation factor matrix is constructed. As shown in Figure 1 This embodiment takes the time point as a specific form of time period, and uniformly uses [τ a ,τ b ]. It is defined as:

[0033] φ(n,r,v,[τ a ,τ b ])=Mean(φ(n,r a ,v,τ a )+φ(n,r b ,v,τ b )) (4)

[0034] where r a ,r b ∈R and τ a ,τ b ∈T represent the relation r a at the start time τ a and the relation r b at the end time τ b , and Mean(·) represents the mean value operation.

[0035] Finally, as shown in Figure 2 , the position encoding embedding representation core tensor is constructed to obtain the embedding vector representation of the knowledge subgraph to construct the potential connection between different knowledge. This embodiment sorts the timestamps in T in ascending order to obtain T=t1,t2,…t n . It is defined as:

[0036]

[0037]

[0038] where 0≤i≤d-1.

[0039] After position encoding processing on the sorted timestamps, the time order probability of the ith pair of timestamps (t m ,t n ) is defined as:

[0040] P i =softmax((T mn ) T ζ i ) (7)

[0041] where denotes a parameter vector.

[0042]

[0043] wherein, is the time difference between any fact m and fact n, is the order symbol between any facts, specifically:

[0044]

[0045] Finally, based on the time sequence probability, cross-entropy is introduced as the time sequence constraint, defined as:

[0046]

[0047] where I represents the total number of timestamps, C represents the number of categories, and ξ(i,c)∈{0,1}.

[0048] Step three: calculate the confidence score of the missing knowledge based on the embedding vector representation of the knowledge subgraph, add the knowledge with the highest confidence score to the time sequence knowledge graph to improve the accuracy of predicting missing facts.

[0049] The confidence score in the embodiment is defined as:

[0050]

[0051] wherein is the training batch, is the negative sample obtained by replacing the positive sample (n, r, v, τ).

[0052] Based on the application, the accuracy of time sequence knowledge graph completion can be effectively improved, and the effectiveness of downstream applications can be improved, and the long-standing problem of incomplete time sequence knowledge graph in the field of natural language processing can be effectively solved.

Claims

1.A method for block decomposition based temporal knowledge graph completion, characterized in that, The method comprises the following steps: (1) dividing into several knowledge sets according to the timestamp annotation in the time sequence knowledge graph, and constructing a knowledge subgraph through the knowledge in each knowledge set; The knowledge set refers to constructing a knowledge set G according to a time interval of knowledge in a time sequence knowledge graph; the knowledge subgraph refers to obtaining G={G1, G2, …G t} by knowledge in the knowledge set G; t represents the number of knowledge subgraphs. (2) inputting the knowledge subgraph into a block decomposition model, and constructing a dualized factor matrix and a position coded core tensor to obtain an embedding vector representation of the knowledge subgraph to construct the potential connection between different knowledge; The block decomposition model refers to a tensor decomposition model comprising a core tensor and three factor matrices; the dualized factor matrix refers to constructing a dual relationship vector representation factor matrix; the position coded core tensor refers to constructing a position coding embedding representation core tensor; (3) calculating the confidence score of the missing knowledge according to the embedding vector representation of the knowledge subgraph, and adding the knowledge with the largest confidence score to the time sequence knowledge graph; The confidence score refers to ranking the entities that may appear in the missing fact in descending order of probability, and then taking the entity with the maximum probability as the predicted missing entity; Wherein the position coding in step (2) is established by: A, sort the timestamps in the timestamp set T in ascending order, and include all the times appearing in a single data set in the sorting range; B, position coding processing is performed on the sorted timestamps to form a time sequence probability; C, based on the time sequence probability, cross entropy is introduced as a time sequence constraint on the timestamp embedding; the dependency relationship between different timestamps is aggregated and distinguished to utilize the timestamp information to complete the unknown fact. 2.The block decomposition based temporal knowledge graph completion method of claim 1, wherein, The knowledge sub-graph G in step (1) t consists of four parts, namely: G t = {n, r, v, τ}; Wherein the head entity n∈N, the relationship r∈R, the tail entity v∈V, and the timestamp τ∈T; N is the set of the head entity n, R is the set of the relationship r, and V is the set of the tail entity v. 3.The block decomposition based temporal knowledge graph completion method of claim 1, wherein, The block decomposition model X in step (2) is composed of four parts, namely: ； where Z is the core tensor, N, R and V are the factor matrices of head entity, relation and tail entity, respectively, and × n denotes the n-mode product. 4.The block decomposition based temporal knowledge graph completion method of claim 3, wherein, The block decomposition model inputs a knowledge subgraph and outputs several initial member tensors Each member tensor is composed of a core tensor and three factor matrices. 5.The block decomposition based temporal knowledge graph completion method of claim 3, wherein, The dualized factor matrix in step (2) is denoted by and respectively, the relationship r a at the start time τ a and the relationship r b at the end time τ b are then taken, and an averaging operation is applied thereto.

Images