Time sequence knowledge graph reasoning model construction method and reasoning method

Abstract

The invention provides a time sequence knowledge graph inference model construction method. The method comprises the steps that S1, a historical time sequence knowledge graph is acquired, and a plurality of queries are constructed based on the historical time sequence knowledge graph; wherein the historical time sequence knowledge graph comprises a plurality of fact tetrads, and each fact tetrad comprises a head entity, a relation, a tail entity and a timestamp; each query is a missing item fact tetrad lacking a tail entity or a head entity, and each query is provided with an entity label corresponding to the head entity or the tail entity lacked by the query; s2, constructing an initial model, wherein the initial model comprises a global historical evolution module, a self-adaptive entity attention module, a historical information module and a scoring module; and S3, taking the historical time sequence knowledge graph and the plurality of queries as input, taking the target score of each query as prediction output, and performing multi-round iterative training on the initial model according to a preset target function until the model converges to obtain a time sequence knowledge graph inference model.

Description

Technical Field

[0001] The present invention relates to the field of knowledge reasoning, and more specifically, to a temporal knowledge graph reasoning technology in the field of knowledge reasoning, and more specifically, to a temporal knowledge graph reasoning model construction method and a reasoning method. Background Art

[0002] Temporal knowledge graph reasoning technology is an important means of completing the temporal knowledge graph. It involves completing incomplete temporal knowledge graphs and using known knowledge to infer unknown future knowledge. Temporal knowledge graph reasoning can be used in various fields, from completing incomplete temporal knowledge graphs to downstream applications such as knowledge base question answering, recommendation systems in e-commerce software, and medical decision-making assistance. On the one hand, temporal knowledge graph reasoning studies the complex dynamic laws of event occurrence and the mutual influence between events, such as the temporal laws in which certain events repeat, and the laws in which a group of events often co-occur. It can help various fields to more comprehensively understand the interactions and dependencies between events, so as to make more accurate predictions of future events. On the other hand, temporal knowledge graph reasoning models the timeliness of events and explains the historical status, development laws, and future trends of a single important event, which is extremely important for enterprises and researchers to make decisions in strategic planning and risk management in various fields.

[0003] Although existing temporal knowledge graph reasoning techniques can effectively model co-occurring events within temporal subgraphs and the evolution of facts across temporal subgraphs, they usually do not fully utilize historical information and pattern information brought by important entities. Specifically, existing methods use graph neural networks to model temporal subgraphs, and use temporal point processes, autoregressive architectures, etc. to model the evolution of facts across temporal subgraphs. However, in this process, the repetitiveness and periodicity of some facts in the evolution process are not fully modeled, and such patterns are the key to accurate reasoning. Modeling common patterns in historical information helps improve the performance of future fact prediction. In addition, when learning temporal subgraphs, existing methods use graph neural networks to pass messages between nodes with topological edges to learn contextual connections between entities. This modeling method fully models explicit edge relationships, but lacks the capture of potential pattern information and ignores the discovery and utilization of prior or implicit evolution laws.

[0004] In summary, the existing technology still has two defects when performing temporal knowledge graph reasoning. The first is the neglect of historical evolution information: the historical evolution pattern of facts changing over time in the temporal knowledge graph is not fully considered, which leads to the inability to accurately capture the temporal dependencies between facts, affecting the accuracy of prediction; the second is the lack of association information between entities: in the reasoning process, the potential association information between entities is often not fully utilized, resulting in the inability to fully capture the structural characteristics of the temporal knowledge graph, which in turn affects the performance of fact prediction.

[0005] It should be noted that this background technology is only used to introduce the relevant information of the present invention to help understand the technical solution of the present invention, but it does not mean that the relevant information is necessarily the prior art. In the absence of evidence that the relevant information has been disclosed before the application date of the present invention, the relevant information shall not be regarded as the prior art. Summary of the invention

[0006] Therefore, the purpose of the present invention is to overcome the defects of the above-mentioned prior art and provide a method for constructing a temporal knowledge graph reasoning model and a temporal knowledge graph reasoning method.

[0007] The objectives of the present invention are achieved through the following technical solutions.

[0008] According to a first aspect of the present invention, a method for constructing a temporal knowledge graph reasoning model is provided, the method comprising: step S1, obtaining a historical temporal knowledge graph and constructing multiple queries based thereon; wherein the historical temporal knowledge graph comprises multiple fact quadruplets, each fact quadruplet comprises a head entity, a relationship, a tail entity and a timestamp; each query is a missing fact quadruplet with a missing tail entity or a head entity, and each query is provided with an entity label corresponding to its missing tail entity or head entity; step S2, constructing an initial model, which comprises a global historical evolution module, an adaptive entity attention module, a historical information module and a scoring module, wherein: the global historical evolution module is used to extract semantic information of the historical temporal knowledge graph to obtain an initial entity representation and a relationship representation containing historical evolution information corresponding to the historical temporal knowledge graph; the adaptive entity attention module is used to extract the potential dependency relationship between each entity in the historical temporal knowledge graph in a manner of adaptive graph and diffusion convolution based on the initial entity representation to obtain a target entity representation corresponding to the historical temporal knowledge graph; the historical information module is used to evaluate the target entity representation, the relationship representation and the historical information of the query according to a preset evaluation method. The historical auxiliary score of the query is calculated in this way; wherein the historical information of the query is the entity information of the existing fact quadruple formed with the query at different timestamps; the historical auxiliary score of the query is the predicted probability that each entity in the historical time series knowledge graph is a candidate entity and forms a complete fact quadruple with the query, calculated based on the target entity representation, the relationship representation and the historical information of the query; the scoring module is used to calculate the score of each entity in the historical time series knowledge graph as a candidate entity and forms a complete fact quadruple with the query based on the target entity representation and the relationship representation according to a preset scoring method to obtain a global score of the query; and the global score of the query and the historical auxiliary score of the query are calculated according to a preset calculation method to obtain a target score of the query; wherein the target score of the query is the predicted probability that each entity in the historical time series knowledge graph is a candidate entity and forms a complete fact quadruple with the query, calculated based on the global score and the historical auxiliary score; step S3, taking the historical time series knowledge graph and multiple queries as input, the target score of each query is the predicted output, and the initial model is iteratively trained for multiple rounds according to the preset objective function until the model converges to obtain a time series knowledge graph inference model.

[0009] In some embodiments of the present invention, the fact quadruples at the same timestamp in the historical temporal knowledge graph constitute a time subgraph, and the global historical evolution module includes a composite graph convolutional network and a temporal evolution submodule, wherein: the composite graph convolutional network is used to extract the structural features of each time subgraph to obtain the entity representation of each time subgraph; the temporal evolution submodule is used to extract the relationship representation of each time subgraph according to a preset extraction method based on the entity representation of each time subgraph and the static relationship representation of its own configuration; it is also used to fuse the entity representation of the time subgraph at the previous moment timestamp with the entity representation of the time subgraph at the current moment timestamp to obtain the initial entity representation of the historical temporal knowledge graph; and to fuse the relationship representation of the time subgraph at the previous moment timestamp with the relationship representation of the time subgraph at the current moment timestamp to obtain the relationship representation of the historical temporal knowledge graph.

[0010] In some embodiments of the present invention, the preset extraction method is:

[0011]

[0012] in, Indicates the timestamp is The relational representation of the temporal subgraph, represents the vector concatenation operation, represents the average pooling function; Indicates the timestamp after fusion processing is Entity representation of the temporal subgraph; Indicates the timestamp is The entity set consisting of all entities that form fact quads with each relationship in the temporal subgraph of ; Represents a static relationship representation.

[0013] In some embodiments of the present invention, the preset evaluation method is:

[0014]

[0015] in,

[0016]

[0017]

[0018] in,

[0019]

[0020] in, Represents the historical auxiliary score, represents the candidate entity set consisting of all candidate entities, Represents an entity, Indicates relationship, Represents the entity information of the existing fact quadruple formed with the query at different timestamps, represents the normalization function, History indicator vector, represents the guidance vector, represents the activation function, represents the weight matrix, Represents the entity in the target entity representation The representation of Relationship Representation The representation of Indicates timestamp To timestamp All time between other entities in the subgraph and the entity in the query and relationship The cumulative frequency of forming fact quadruplets, Indicates timestamp Other entities in the temporal subgraph are related to the entities in the query and relationship The frequency of forming fact quadruplets, Indicates timestamp Other entities in the temporal subgraph are related to the entities in the query and relationship The frequency of forming fact quadruplets, Indicates timestamp Other entities in the temporal subgraph are related to the entities in the query and relationship The frequency of forming fact quadruplets; represents a constant, represents the product symbol, Indicates other entities not in the query and relationship Form the indicator vector corresponding to the fact quadruple, Represents other entities related to the query and relationship Form the indicator vector corresponding to the fact quadruple.

[0021] In some embodiments of the present invention, the preset scoring method is:

[0022]

[0023] in, represents the global score, represents the target entity representation, Represents a relationship representation, represents the scoring function.

[0024] In some embodiments of the present invention, the preset calculation method is:

[0025]

[0026] in, represents the target score, represents a hyperparameter.

[0027] In some embodiments of the present invention, the preset objective function is:

[0028]

[0029] in,

[0030]

[0031]

[0032] in, represents the objective function; Represents the entity prediction loss; represents the historical scoring loss; represents a hyperparameter; The number of time subgraphs representing the historical temporal knowledge graph; Represents a fact quad; Represents the head entity; Represents the tail entity; Indicates a timestamp; The number of entities representing the historical temporal knowledge graph; Represents the entity among all candidate entities Whether the value is consistent with the query entity tag. If it is consistent with the entity tag , otherwise ; represents the logarithmic function; Indicates that the target score is based on entity With the entities in the query and relationship The predicted probability of forming a complete fact quadruple, Indicates the entity in the historical auxiliary scoring With the entities in the query and relationship The predicted probability of forming a complete fact quad.

[0033] According to the second aspect of the present invention, a temporal knowledge graph reasoning method is provided, the method comprising: step T1, obtaining a historical temporal knowledge graph to be processed and constructing one or more queries based thereon; step T2, using the temporal knowledge graph reasoning model constructed by the method described in the first aspect of the present invention to perform knowledge reasoning on all queries based on the historical temporal knowledge graph to be processed, so as to obtain a target score for each query; step T3, selecting from the target score of each query a candidate entity corresponding to a predicted probability that meets preset requirements and the query to form a complete fact quadruple as a knowledge reasoning result.

[0034] Compared with the prior art, the advantages of the present invention are: (1) a global historical evolution module is set up to extract the historical evolution pattern of facts changing over time in the temporal knowledge graph, thereby accurately capturing the temporal dependencies between facts, which helps to improve prediction accuracy; (2) an adaptive entity attention module is set up to mine the potential correlation information between entities in the temporal knowledge graph, thereby comprehensively capturing the structural features of the temporal knowledge graph, which helps to improve prediction accuracy; (3) a historical information module is set up to reduce the candidate space of prediction candidate entities, which helps to improve prediction accuracy. BRIEF DESCRIPTION OF THE DRAWINGS

[0035] The embodiments of the present invention are further described below with reference to the accompanying drawings, in which:

[0036] Figure 1 A schematic diagram of the process of constructing a temporal knowledge graph reasoning model according to an embodiment of the present invention;

[0037] Figure 2 A schematic diagram of the composition of a temporal knowledge graph reasoning model according to an embodiment of the present invention;

[0038] Figure 3 It is an internal structure entity diagram of the temporal knowledge graph reasoning model according to an embodiment of the present invention. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical solutions and advantages of the present invention more clearly understood, the present invention is further described in detail below through specific embodiments in conjunction with the accompanying drawings. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0040] As mentioned in the background technology section, the existing technology still has two defects when performing temporal knowledge graph reasoning. The first is the neglect of historical evolution information: the historical evolution pattern of facts in the temporal knowledge graph over time is not fully considered, which leads to the inability to accurately capture the temporal dependencies between facts, affecting the accuracy of prediction; the second is the lack of association information between entities: in the reasoning process, the potential association information between entities is often not fully utilized, resulting in the inability to fully capture the structural characteristics of the temporal knowledge graph, which in turn affects the performance of fact prediction.

[0041] In order to solve the above problems, the inventors proposed a reasoning model that can fully model historical evolution information and inter-entity association information. In this model, a global historical evolution module, an adaptive entity attention module and a historical information module are set; wherein, the global historical evolution module uses a composite graph convolutional network to perform full-graph message transmission on each time subgraph of the temporal knowledge graph to update the entity representation, and uses a gating unit between adjacent time subgraphs to capture the evolution pattern of the entity, and also uses entities associated with the relationship to realize the representation evolution of the relationship, thereby fully capturing the semantic information of the temporal knowledge graph; the adaptive entity attention module mines the potential evolutionary associations between entities in the temporal subgraph that evolves over time through an adaptive graph, and uses diffusion convolution to apply the potential evolutionary associations between the mined entities to the high-order message transmission of the convolutional layer to realize the joint learning of structure and temporal evolution, thereby fully capturing the structural characteristics of the temporal knowledge graph; the historical information module assists knowledge reasoning by modeling the correlation between queries and historical entities, which can effectively improve the prediction performance of the model.

[0042] In summary, if Figure 1 As shown, a method for constructing a temporal knowledge graph reasoning model is provided, the method comprising: step S1, obtaining a historical temporal knowledge graph and constructing multiple queries based thereon; wherein the historical temporal knowledge graph comprises multiple fact quadruplets, each fact quadruplet comprises a head entity, a relationship, a tail entity and a timestamp; each query is a missing fact quadruplet that lacks a tail entity or a head entity, and each query is provided with an entity label corresponding to its own missing tail entity or head entity; step S2, constructing an initial model, which comprises a global historical evolution module, an adaptive entity attention module, a historical information module and a scoring module; step S3, taking the historical temporal knowledge graph and multiple queries as input, the target score of each query is the predicted output, and performing multiple rounds of iterative training on the initial model according to a preset objective function until the model converges to obtain a temporal knowledge graph reasoning model.

[0043] In order to better understand the present invention, each step is described in detail below in conjunction with specific embodiments.

[0044] 1. Step S1

[0045] In step S1, a historical time series knowledge graph is obtained and multiple queries are constructed based on it.

[0046] A historical time series knowledge graph is a directed multi-relational graph with timestamp information, which is composed of fact quadruplets. A fact quadruplet consists of a head entity, a tail entity, the relationship between the head and tail entities, and a timestamp. The timestamp indicates the time when the relationship between the head and tail entities occurs. The timestamp can be a time node (such as 2010) or a time interval (such as 2010-2013). Specifically, a historical time series knowledge graph can be represented as ,in, Represents an entity set, represents a set of relations, represents a timestamp set, Represents a set of facts. A fact quad can be represented as ,in, Represents the head entity, Indicates relationship, Represents the tail entity, Represents a timestamp. If the fact quadruple in the historical time series knowledge graph is sorted in ascending order according to the timestamp, and the fact quadruple with the same timestamp is integrated into a time subgraph, the complete historical time series knowledge graph can be represented as a sequence consisting of multiple time subgraphs. , Indicates timestamp The time subgraph below, Indicates timestamp The time subgraph below, Indicates timestamp The time subgraph below.

[0047] The query is a missing fact quad that is missing the tail entity or the head entity, which can be expressed as or or ,in, Represents an entity in the query (head entity or tail entity), " " indicates an entity (head or tail) that is missing from the query.

[0048] Step S2

[0049] In step S2, an initial model is constructed. Figure 2 As shown in Figure 2, the initial model includes a global historical evolution module, an adaptive entity attention module, a historical information module, and a scoring module. Figure 3 The internal structure of the initial model is shown to introduce each component of the initial model one by one.

[0050] 2.1 Global Historical Evolution Module

[0051] The global historical evolution module is used to extract the semantic information of the historical time series knowledge graph to obtain the initial entity representation and relationship representation containing the historical evolution information corresponding to the historical time series knowledge graph. The reason for setting up the global historical evolution module is that if you want to predict the query Entities missing from , entities need to be considered Global information on the knowledge graph, where global information includes All adjacent entity information (structural dependency information) that appears in the same time subgraph, as well as entity information in cross-time subgraphs The evolutionary pattern of the facts (temporal evolutionary pattern).

[0052] In order to enable the global historical evolution module to extract structural dependency information and temporal evolution patterns, the inventors proposed that a structural dependency submodule and a temporal evolution submodule can be set in the global historical evolution module; the structural dependency submodule is used to explore the structural features within the temporal subgraph; the temporal evolution submodule is used to model the temporal evolution features of entities in other temporal subgraphs. Based on this, the global historical evolution module can learn the neighborhood information of the temporal subgraphs contained in the historical temporal knowledge graph through message passing and the evolution information of the cross-temporal subgraphs.

[0053] According to one embodiment of the present invention, the global historical evolution module includes a composite graph convolutional network and a temporal evolution sub-module, wherein: the composite graph convolutional network is used to extract the structural features of each time sub-graph to obtain the entity representation of each time sub-graph; the temporal evolution sub-module is used to extract the relationship representation of each time sub-graph according to a preset extraction method based on the entity representation of each time sub-graph and the static relationship representation of its own configuration; it is also used to fuse the entity representation of the time sub-graph at the previous moment timestamp with the entity representation of the time sub-graph at the current moment timestamp to obtain the initial entity representation of the historical temporal knowledge graph; and to fuse the relationship representation of the time sub-graph at the previous moment timestamp with the relationship representation of the time sub-graph at the current moment timestamp to obtain the relationship representation of the historical temporal knowledge graph.

[0054] The composite graph convolutional network is used as a structural dependency submodule to explore the internal structural features of each time subgraph. Specifically, the composite graph convolutional network extracts the entity representation of each time subgraph according to the following methods based on the multi-layer network layers set by itself: initial embedding: the first network layer randomly initializes the initial embedding representation of each entity and each relationship in the time subgraph; message construction: in the second to the last network layer, each network layer constructs the embedding representation of each pair of connected head entities and tail entities according to the embedding representation of each entity and the embedding representation of each relationship transmitted by the previous network layer; message aggregation: in the second to the last network layer, each network layer aggregates all messages pointing to the same tail entity in an additive aggregation manner according to the embedding representation of all messages constructed, and obtains the embedding representation of multiple comprehensive messages; embedding update: in the second to the last network layer, each network layer updates the embedding representation of each entity in the time subgraph based on the embedding representation of multiple comprehensive messages obtained by aggregation processing and passes it to the next network layer. The entity representation of each time subgraph is the output of the last network layer of the composite graph convolutional network.

[0055] It should be noted that there may be entities without edge relationships in the temporal subgraph. For such entities without edge relationships, the composite graph convolutional network will give a self-loop edge when processing them to enable the entity to self-evolve and update. The process can be formalized as follows:

[0056]

[0057] in, Indicates Timestamp in the network layer The tail entity of the next time subgraph Embedded representation of ; ReLU activation function. is the regularization constant, which is numerically equal to the tail entity The in-degree of Indicates timestamp The fact set of the lower time subgraph; Indicates The first weight matrix of the network layer; Indicates Timestamp in the network layer Head entity of the next time subgraph Embedded representation of ; Indicates timestamp Head entity of the next time subgraph With tail entity Embedding representation of the relationship between them; Indicates The second weight matrix of the layer network layer, Indicates Timestamp in the network layer The tail entity of the next time subgraph The embedded representation of .

[0058] The temporal evolution submodule is used to learn the temporal evolution pattern across timestamps based on the entity representation of each temporal subgraph obtained by the composite graph convolutional network. In order to fully learn the information reflecting the evolution trend of each entity, such as Figure 3 As shown, the temporal evolution submodule adopts an RNN-based structure to learn the temporal evolution pattern, wherein the temporal evolution submodule includes multiple gating units, and only the update gate is retained in each gating unit; the temporal evolution submodule extracts the relational representation of each temporal subgraph according to a preset extraction method based on the entity representation of each temporal subgraph and the static relational representation of its own configuration; and retains the entity representation of the time subgraphs of adjacent timestamps to varying degrees through the update gate to obtain the initial entity representation of the historical temporal knowledge graph, and retains the relational representation of the time subgraphs of adjacent timestamps to varying degrees through the update gate to obtain the relational representation of the historical temporal knowledge graph.

[0059] According to one embodiment of the present invention, the preset extraction method is:

[0060]

[0061] in, Indicates the timestamp is The relational representation of the temporal subgraph, represents the vector concatenation operation, represents the average pooling function; Indicates the timestamp after fusion processing is Entity representation of the temporal subgraph; Indicates the timestamp is The entity set consisting of all entities that form fact quads with each relationship in the temporal subgraph of ; It should be noted that if there is no entity in the time subgraph that forms a fact quadruple with the relationship, the relationship can be initialized to a zero vector.

[0062] In order to better understand the working principle of the time series evolution submodule, Figure 3 This chapter explains the contents displayed in the global historical evolution module.

[0063] like Figure 3 As shown in Figure 2, the temporal evolution submodule models the evolution of entities in the temporal subgraph by retaining the entity representations of the temporal subgraphs of adjacent timestamps to varying degrees through the update gate (gating unit). Figure 3The entity evolution modeling process is briefly described based on the content shown in Figure 3 middle, Indicates the timestamp after update gate fusion processing The entity representation of the temporal subgraph under Indicates the timestamp after update gate fusion processing The entity representation of the temporal subgraph under Indicates the timestamp after update gate fusion processing The entity representation of the temporal subgraph under . Among them, , and The evolution modeling process can be expressed as:

[0064]

[0065]

[0066] in, Indicates timestamp The update gate parameters under represents the dot product operation, Indicates timestamp The entity representation of the temporal subgraph under (processed by the composite graph convolutional network), Indicates the timestamp after update gate fusion processing The entity representation of the temporal subgraph under represents the construction of the updated parameter matrix, represents the bias matrix, Represents the sigmoid function.

[0067]

[0068]

[0069] in, Indicates timestamp The update gate parameters under Indicates timestamp Entity representation of the temporal subgraph under (processed by a composite graph convolutional network).

[0070]

[0071]

[0072] in, Indicates timestamp The update gate parameters under Indicates timestamp Entity representation of the temporal subgraph under (processed by a composite graph convolutional network).

[0073] It should be noted that by retaining the entity representations of time subgraphs of adjacent timestamps to varying degrees through the update gate, we can eventually obtain the initial entity representation containing historical evolution information corresponding to the historical time series knowledge graph, and the initial entity representation of the historical time series knowledge graph is the entity representation of the time subgraph corresponding to the last timestamp after fusion processing by the update gate.

[0074] like Figure 3 As shown in the figure, the time series evolution submodule also extracts the relationship representation of each time subgraph according to the preset extraction method based on the entity representation of each time subgraph and the static relationship representation of its own configuration, and retains the relationship representation of the time subgraphs of adjacent timestamps to different degrees through the update gate to perform evolution modeling on the relationship in the time subgraph. Figure 3 The process of relationship evolution modeling is briefly described based on the content shown in Figure 3 middle, Indicates the timestamp after update gate fusion processing The relational representation of the temporal subgraph under Indicates the timestamp after update gate fusion processing The relational representation of the temporal subgraph under Indicates the timestamp after update gate fusion processing The relationship representation of the time subgraph under , and The evolution modeling process can be expressed as:

[0075]

[0076]

[0077]

[0078] in, Indicates that the update gate is used for information fusion processing. Indicates the timestamp after update gate fusion processing The relational representation of the temporal subgraph under Indicates timestamp The relational representation of the temporal subgraph under Indicates timestamp The entity set consists of all entities that form fact quads with each relationship in the temporal subgraph below, Indicates timestamp The set of facts for the next time subgraph.

[0079]

[0080]

[0081]

[0082] in, Indicates timestamp The relational representation of the temporal subgraph under Indicates timestamp The entity set consists of all entities that form fact quads with each relationship in the temporal subgraph below, Indicates timestamp The set of facts for the next time subgraph.

[0083]

[0084]

[0085]

[0086] in, Indicates timestamp The relational representation of the temporal subgraph under Indicates timestamp The entity set consists of all entities that form fact quads with each relationship in the temporal subgraph below, Indicates timestamp The set of facts for the next time subgraph.

[0087] It should be noted that by retaining the relationship representations of time subgraphs of adjacent timestamps to varying degrees through the update gate, we can eventually obtain the relationship representation containing historical evolution information corresponding to the historical time series knowledge graph, and the relationship representation of the historical time series knowledge graph is the relationship representation of the time subgraph corresponding to the last timestamp after fusion processing by the update gate. It should also be noted that there are some relationships with weak time-variability in the historical time series knowledge graph, such as "(A, CapitalOf, B)", which will not change for a long time in the future. For this type of relationship, a static representation can be set. Based on this, in the above relationship evolution modeling process, static relationship representations are set for all relationships. And use static relationship representation to perform relationship evolution learning, ensuring that dynamically updated relationship information can be continuously learned during the learning process, while keeping those relationships that rarely disappear or reappear remains unchanged.

[0088] 2.2 Adaptive Entity Attention Module

[0089] After the historical time series knowledge graph is processed by the global historical evolution module, the global structural information (explicit associations between entities) and evolution information of the entities in the historical time series knowledge graph are obtained. However, there may be fewer edge relationships between some entities with similar attributes in the historical time series knowledge graph. For example, a country entity A has fewer edge relationships with other entities. In order to predict the future behavior of country entity A, the behavior of other country entities with rich information can be referred to. Therefore, it is not enough to only learn the explicit associations between entities. It is also necessary to learn the implicit (potential) associations between entities in the historical time series knowledge graph. Based on this, the present invention sets an adaptive entity attention module to learn the potential associations between entities.

[0090] The adaptive entity attention module is used to extract the potential dependencies between entities in the historical time series knowledge graph based on the initial entity representation in an adaptive graph and diffusion convolution manner to obtain the target entity representation corresponding to the historical time series knowledge graph.

[0091] The adaptive graph calculates the similarity between entities in the historical time series knowledge graph through dot product similarity to measure the correlation between entities, and uses the similarity between entities as the weight of the edge to construct a new edge relationship, as shown in the following figure. Figure 3 An example of a new historical time series knowledge graph shown by the adaptive entity attention module in the figure (the dotted part in the example is the newly constructed edge relationship). Diffused convolution uses the adjacency relationship of each entity on the new historical time series knowledge graph to propagate the feature information of the entity to other neighboring entities, thereby obtaining a diffuse entity representation, so as to further capture the potential associations between entities on the historical time series knowledge graph. It should be noted that since adaptive graph and diffuse convolution are two mature technologies, only their basic principles are briefly outlined in this invention without further elaboration.

[0092] 2.3 Historical Information Module

[0093] The relationship representation and target entity representation of the historical time series knowledge graph obtained after processing by the global historical evolution module and the adaptive entity attention module can be used to predict the prediction probability of each entity in the historical time series knowledge graph as a candidate entity and the query to form a complete fact quadruple. However, the most important prior information for the prediction process is actually the repeated facts in the past. For example, if you want to predict the champion of basketball game A at time t (basketball game, champion,?,t), the team that has won the championship at past times such as t-1, t-2 is often more likely to win the championship. Based on this, the present invention introduces a historical information module, which uses the important prior information of repetition frequency to give important known entities a higher degree of attention to assist the final prediction.

[0094] The historical information module is used to calculate the historical auxiliary score of the query according to a preset evaluation method based on the target entity representation, relationship representation and historical information of the query; wherein the historical information of the query is the entity information of the existing fact quadruple formed with the query at different timestamps; the historical auxiliary score of the query is the predicted probability of each entity in the historical time series knowledge graph as a candidate entity and forming a complete fact quadruple with the query calculated based on the target entity representation, relationship representation and historical information of the query.

[0095] According to one embodiment of the present invention, the preset evaluation method is:

[0096]

[0097] in,

[0098]

[0099]

[0100] in,

[0101]

[0102] in, Represents the historical auxiliary score, represents the candidate entity set consisting of all candidate entities, Represents an entity, Indicates relationship, Represents the entity information of the existing fact quadruple formed with the query at different timestamps, represents the normalization function, History indicator vector, represents the guidance vector, represents the activation function, represents the weight matrix, Represents the entity in the target entity representation The representation of Relationship Representation The representation of Indicates timestamp To timestamp All time between other entities in the subgraph and the entity in the query and relationship The cumulative frequency of forming fact quadruplets, Indicates timestamp Other entities in the temporal subgraph are related to the entities in the query and relationship The frequency of forming fact quadruplets, Indicates timestamp Other entities in the temporal subgraph are related to the entities in the query and relationship The frequency of forming fact quadruplets, Indicates timestamp Other entities in the temporal subgraph are related to the entities in the query and relationship The frequency of forming fact quadruplets; represents a constant, represents the product symbol, Indicates other entities not in the query and relationship Form the indicator vector corresponding to the fact quadruple, Represents other entities related to the query and relationship The indicator vector corresponding to the fact quadruple is formed. Indicates that in a given entity ,relation and entity information Under the condition that Each entity in is a candidate entity and entity and relationship The predicted (estimated) probability of forming a fact. For example, the candidate entity set include , , , , , , , , and , entity information include , , and ,but Indicates that in a given entity ,relation and entity information , , and Under the conditions, , , , , , , , , and As candidate entities and entities respectively and relationship The predicted (estimated) probability of a fact. The corresponding indicator vector and The corresponding indicator vectors contain the same number of elements. If there are five entities , , , and ,and and Not associated with entity and relationship To form a fact quadruple, The corresponding indicator vector can be expressed as [1,1,0,0,0], The corresponding indicator vector can be expressed as [0,0,1,1,1]. In other cases The corresponding indicator vector and The corresponding indicator vector is determined with reference to the above content, and the present invention will not elaborate on it in detail.

[0103] It should be noted that the principle of the preset evaluation method can be understood as: raising the estimated probability of entities that form known facts with the query and reducing the estimated probability of entities that do not form known facts with the query. , whose candidate entities include , , , , , , , , and ,in, , , and for For entities that have formed known facts with the query before the moment, when calculating the historical auxiliary score, , , and The estimated probability of a candidate entity forming a complete fact quad with the query is higher than that of other candidate entities. Entity before time The frequency of known facts related to the query is higher than , and The frequency of known facts formed with the query, then the entity The estimated probability of a candidate entity forming a complete fact quadruple with the query is higher than , and As the estimated probability that the candidate entity and the query form a complete fact quadruple, the higher the estimated probability that the candidate entity and the query form a complete fact quadruple, the higher the probability that the candidate entity is predicted to be the correct entity.

[0104] It should also be noted that the historical information module uses prior information to assign higher estimated probabilities to candidate entities that have formed facts with the query, which can greatly reduce the candidate space for predicting candidate entities and achieve more accurate entity prediction.

[0105] 2.4 Scoring Module

[0106] The scoring module is used to calculate the score of each entity in the historical time series knowledge graph as a candidate entity and the query to form a complete fact quadruple based on the target entity representation and the relationship representation according to a preset scoring method to obtain a global score of the query; and calculate the global score of the query and the historical auxiliary score of the query according to a preset calculation method to obtain a target score of the query; wherein the target score of the query is the predicted probability of each entity in the historical time series knowledge graph as a candidate entity and the query to form a complete fact quadruple based on the global score and the historical auxiliary score.

[0107] According to an embodiment of the present invention, the preset scoring method is:

[0108]

[0109] in, represents the global score, represents the target entity representation, Represents a relationship representation, represents the scoring function. It can also be replaced by other scoring functions, such as Euclidean distance, Manhattan distance, cosine similarity, etc., and the present invention does not make any special limitations.

[0110] According to an embodiment of the present invention, the preset calculation method is:

[0111]

[0112] in, represents the target score, It should be noted that the hyperparameter The value of is adjusted according to the actual situation and is not particularly limited in the present invention.

[0113] It should be noted that the scoring module combines the historical auxiliary scores output by the historical information module to predict the prediction probability of each candidate entity and the query forming a complete fact quadruple, which can achieve more accurate entity prediction.

[0114] Step S3

[0115] In step S3, the historical time series knowledge graph and multiple queries are used as input, the target score of each query is used as the predicted output, and the initial model is iteratively trained for multiple rounds according to the preset objective function until the model converges to obtain the time series knowledge graph inference model.

[0116] According to one embodiment of the present invention, the preset objective function is:

[0117]

[0118] in,

[0119]

[0120]

[0121] in, represents the objective function; Represents the entity prediction loss; represents the historical scoring loss; represents a hyperparameter; The number of time subgraphs representing the historical temporal knowledge graph; Represents a fact quad; Represents the head entity; Represents the tail entity; Indicates a timestamp; The number of entities representing the historical temporal knowledge graph; Represents the entity among all candidate entities Whether the value is consistent with the query entity tag. If it is consistent with the entity tag , otherwise ; represents the logarithmic function; Indicates that the target score is based on entity With the entities in the query and relationship The predicted probability of forming a complete fact quadruple, Indicates the entity in the historical auxiliary scoring With the entities in the query and relationship The predicted probability of the complete fact quadruple. Among them, the hyperparameter It can be set to 1, or adjusted to other values ​​according to actual conditions, and the present invention does not impose any special limitations.

[0122] The temporal knowledge graph reasoning model constructed in the aforementioned embodiment can realize query prediction. Based on this working characteristic, the present invention also provides a temporal knowledge graph reasoning method, which includes: step T1, obtaining the historical temporal knowledge graph to be processed and constructing one or more queries based thereon; step T2, using the temporal knowledge graph reasoning model constructed in the aforementioned embodiment to perform knowledge reasoning on all queries based on the historical temporal knowledge graph to be processed to obtain the target score of each query; step T3, selecting the candidate entity corresponding to the predicted probability that meets the preset requirements from the target score of each query and the query to form a complete fact quadruple as the knowledge reasoning result.

[0123] The beneficial effects of the present invention are: (1) a global historical evolution module is set up to extract the historical evolution pattern of facts changing over time in the temporal knowledge graph, thereby accurately capturing the temporal dependencies between facts, which helps to improve prediction accuracy; (2) an adaptive entity attention module is set up to mine the potential correlation information between entities in the temporal knowledge graph, thereby comprehensively capturing the structural features of the temporal knowledge graph, which helps to improve prediction accuracy; (3) a historical information module is set up to reduce the candidate space of prediction candidate entities, which helps to improve prediction accuracy.

[0124] It should be noted that although the above describes the various steps in a specific order, it does not mean that the various steps must be executed in the above specific order. In fact, some of these steps can be executed concurrently or even in a different order as long as the required functions can be achieved.

[0125] The present invention may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present invention.

[0126] A computer-readable storage medium may be a tangible device that holds and stores instructions used by an instruction execution device. Computer-readable storage media may include, for example, but are not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, such as a punch card or a protruding structure in a groove on which instructions are stored, and any suitable combination thereof.

[0127] The embodiments of the present invention have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or technical improvements in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A method for constructing a temporal knowledge graph reasoning model, characterized in that: The method comprises: Step S1, obtaining a historical time series knowledge graph and constructing multiple queries based thereon; wherein the historical time series knowledge graph includes multiple fact quadruplets, each fact quadruplet includes a head entity, a relationship, a tail entity and a timestamp; each query is a missing fact quadruplet with a missing tail entity or a head entity, and each query is provided with an entity label corresponding to the missing tail entity or head entity; Step S2: construct an initial model, which includes a global historical evolution module, an adaptive entity attention module, a historical information module and a scoring module, wherein: The global historical evolution module is used to extract semantic information of the historical time series knowledge graph to obtain initial entity representation and relationship representation containing historical evolution information corresponding to the historical time series knowledge graph; The adaptive entity attention module is used to extract the potential dependency relationship between entities in the historical time series knowledge graph in a way of adaptive graph and diffusion convolution based on the initial entity representation, so as to obtain the target entity representation corresponding to the historical time series knowledge graph; The historical information module is used to calculate the historical auxiliary score of the query according to a preset evaluation method based on the target entity representation, the relationship representation and the historical information of the query; wherein the historical information of the query is the entity information of the existing fact quadruple formed with the query at different timestamps; the historical auxiliary score of the query is the predicted probability of each entity in the historical time series knowledge graph as a candidate entity and forming a complete fact quadruple with the query calculated based on the target entity representation, the relationship representation and the historical information of the query; The scoring module is used to calculate the score of each entity in the historical time series knowledge graph as a candidate entity and the query to form a complete fact quadruple based on the target entity representation and the relationship representation according to a preset scoring method to obtain a global score of the query; and calculate the global score of the query and the historical auxiliary score of the query according to a preset calculation method to obtain a target score of the query; wherein the target score of the query is the predicted probability of each entity in the historical time series knowledge graph as a candidate entity and the query to form a complete fact quadruple based on the global score and the historical auxiliary score; Step S3: Take the historical time series knowledge graph and multiple queries as input, and the target score of each query as the predicted output. Perform multiple rounds of iterative training on the initial model according to the preset objective function until the model converges to obtain the time series knowledge graph inference model.

2. The method according to claim 1, characterized in that The fact quadruples at the same timestamp in the historical time series knowledge graph constitute a time subgraph, and the global history evolution module includes a composite graph convolutional network and a time series evolution submodule, wherein: The composite graph convolutional network is used to extract the structural features of each time subgraph to obtain the entity representation of each time subgraph; The temporal evolution submodule is used to extract the relational representation of each time subgraph according to a preset extraction method based on the entity representation of each time subgraph and the static relational representation of its own configuration; it is also used to fuse the entity representation of the time subgraph at the previous moment timestamp with the entity representation of the time subgraph at the current moment timestamp to obtain the initial entity representation of the historical temporal knowledge graph; and to fuse the relational representation of the time subgraph at the previous moment timestamp with the relational representation of the time subgraph at the current moment timestamp to obtain the relational representation of the historical temporal knowledge graph.

3. The method according to claim 2, characterized in that The preset extraction method is: in, Indicates the timestamp is The relational representation of the temporal subgraph, represents the vector concatenation operation, represents the average pooling function; Indicates the timestamp after fusion processing is Entity representation of the temporal subgraph; Indicates the timestamp is The entity set consisting of all entities that form fact quads with each relationship in the temporal subgraph of ; Represents a static relationship representation.

4. The method according to claim 3, characterized in that The preset evaluation method is: in, in, in, Represents the historical auxiliary score, represents the candidate entity set consisting of all candidate entities, Represents an entity, Indicates relationship, Represents the entity information of the existing fact quadruple formed with the query at different timestamps, represents the normalization function, History indicator vector, represents the guidance vector, represents the activation function, represents the weight matrix, Represents the entity in the target entity representation The representation of Representing Relationships The representation of Indicates timestamp To timestamp All time between other entities in the subgraph and the entity in the query and relationship The cumulative frequency of forming fact quadruplets, Indicates timestamp Other entities in the temporal subgraph are related to the entities in the query and relationship The frequency of forming fact quadruplets, Indicates timestamp Other entities in the temporal subgraph are related to the entities in the query and relationship The frequency of forming fact quadruplets, Indicates timestamp Other entities in the temporal subgraph are related to the entities in the query and relationship The frequency of forming fact quadruplets; represents a constant, represents the product symbol, Indicates other entities not in the query and relationship Form the indicator vector corresponding to the fact quadruple, Represents other entities related to the query and relationship Form the indicator vector corresponding to the fact quadruple.

5. The method according to claim 4, characterized in that The preset scoring method is: in, represents the global score, represents the target entity representation, Represents a relationship representation, represents the scoring function.

6. The method according to claim 5, characterized in that The preset calculation method is: in, represents the target score, Represents a hyperparameter.

7. The method according to claim 6, characterized in that: The preset objective function is: in, in, represents the objective function; Represents the entity prediction loss; represents the historical scoring loss; represents a hyperparameter; The number of time subgraphs representing the historical temporal knowledge graph; Represents a fact quad; Represents the head entity; Represents the tail entity; Indicates a timestamp; The number of entities representing the historical temporal knowledge graph; Represents the entity among all candidate entities Whether the value is consistent with the query entity tag. If it is consistent with the entity tag , otherwise ; represents the logarithmic function; Indicates that the target score is based on entity With the entities in the query and relationship The predicted probability of forming a complete fact quadruple, Indicates the entity in the historical auxiliary scoring With the entities in the query and relationship The predicted probability of forming a complete fact quad.

8. A temporal knowledge graph reasoning method, characterized in that: The method comprises: Step T1: Obtain the historical time series knowledge graph to be processed and construct one or more queries based on it; Step T2: Using the time series knowledge graph reasoning model constructed by the method described in any one of claims 1 to 7 to perform knowledge reasoning on all queries based on the historical time series knowledge graph to be processed to obtain a target score for each query; Step T3: Select the candidate entity corresponding to the predicted probability that meets the preset requirements from the target score of each query and the query to form a complete fact quadruple as the knowledge reasoning result.

9. A computer-readable storage medium, characterized in that: A computer program is stored thereon, and the computer program can be executed by a processor to implement the steps of any method described in claims 1-8.

10. An electronic device, characterized in that: include: one or more processors, and A memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of any one of the methods of claims 1-8 by executing the executable instructions.

Images