A time sequence knowledge graph reasoning model construction method and a reasoning method

Abstract

The application provides a time sequence knowledge graph reasoning model construction method, the method comprises the following steps: S1, obtaining a historical time sequence knowledge graph and constructing a plurality of queries based thereon; wherein the historical time sequence knowledge graph comprises a plurality of fact quadruples, each fact quadruple comprises a head entity, a relation, a tail entity and a timestamp; each query is a missing tail entity or a head entity missing fact quadruple, and each query is provided with an entity label corresponding to the missing head entity or tail entity thereof; S2, constructing an initial model, which comprises a global historical evolution module, an adaptive entity attention module, a historical information module and a scoring module; S3, taking the historical time sequence knowledge graph and the plurality of queries as inputs, the target score of each query being a predicted output, and performing multiple rounds of iterative training on the initial model according to a preset target function until the model converges to obtain a time sequence knowledge graph reasoning model.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of knowledge reasoning, in particular to a time-series knowledge graph reasoning technology in the field of knowledge reasoning, and more particularly to a time-series knowledge graph reasoning model construction method and reasoning method. BACKGROUND

[0002] The time-series knowledge graph reasoning technology is an important means for time-series knowledge graph completion, which involves completing incomplete time-series knowledge graphs and inferring unknown future knowledge from known knowledge. Time-series knowledge graph reasoning can be used in various fields, from completing incomplete time-series knowledge graphs to downstream applications such as knowledge base question answering, recommendation systems in e-commerce software, and medical auxiliary decision-making. On the one hand, time-series knowledge graph reasoning studies the complex dynamic rules of event occurrence and the mutual influence between events, such as the time-series rules of the recurrence of certain events and the co-occurrence of a group of events, which can help various fields to more comprehensively understand the interaction and dependency between events, thereby making more accurate future event predictions. On the other hand, time-series knowledge graph reasoning models the time effectiveness of events, explaining the historical state, development rules, and future trends of a single important event, which is extremely important for decision-making in strategy planning and risk management in various fields by enterprises and researchers.

[0003] Although existing time-series knowledge graph reasoning technologies can effectively model co-occurring events within a time subgraph and the evolution pattern of facts across time subgraphs, they usually do not fully utilize historical information and the pattern information brought by important entities. Specifically, existing methods use graph neural networks to model time subgraphs and use time-series point processes, autoregressive architectures, etc. to model the evolution pattern of facts across time subgraphs. However, in this process, the repeatability and periodicity of some facts in the evolution process are not fully modeled, and such patterns are the key to accurate reasoning. Modeling common patterns in historical information helps to improve the performance of future fact prediction. In addition, existing methods learn time subgraphs by using graph neural networks to perform message passing between nodes with topological edges to learn the context relationship between entities. This modeling method fully models explicit edge relationships, but lacks the capture of potential pattern information and ignores the exploration and utilization of prior or implicit evolution rules.

[0004] In summary, the prior art has two defects in the temporal knowledge graph reasoning, one is the neglect of historical evolution information: the historical evolution pattern of the facts in the temporal knowledge graph with time is not fully considered, which leads to the inability to accurately capture the temporal dependency between the facts, affecting the accuracy of the prediction; the other is the lack of correlation information between entities: the potential correlation information between entities is often not fully utilized in the reasoning process, which leads to the inability to fully capture the structural characteristics of the temporal knowledge graph, and further affects the performance of the fact prediction.

[0005] It should be noted that: the background technology is only used to introduce the related information of the present application, so as to help understand the technical scheme of the present application, but it does not mean that the related information must be the prior art. In the absence of evidence that the related information has been disclosed before the filing date of the present application, the related information should not be regarded as prior art SUMMARY

[0006] Therefore, the purpose of the present application is to overcome the defects of the prior art, and to provide a temporal knowledge graph reasoning model construction method and a temporal knowledge graph reasoning method.

[0007] The purpose of the present application is achieved by the following technical scheme.

[0008] According to a first aspect of the present application, a method for constructing a time sequence knowledge graph reasoning model is provided, the method comprising: step S1, acquiring a historical time sequence knowledge graph and constructing a plurality of queries based thereon; wherein the historical time sequence knowledge graph comprises a plurality of fact quadruples, each fact quadruple comprising a head entity, a relation, a tail entity and a timestamp; each query is a missing tail entity or head entity missing fact quadruple, and each query is provided with an entity label corresponding to the missing tail entity or head entity thereof; step S2, constructing an initial model, which comprises a global historical evolution module, an adaptive entity attention module, a historical information module and a scoring module, wherein: the global historical evolution module is used to extract semantic information of the historical time sequence knowledge graph to obtain initial entity representations and relation representations of the historical time sequence knowledge graph containing historical evolution information; the adaptive entity attention module is used to extract potential dependency relationships between entities in the historical time sequence knowledge graph in an adaptive graph and diffusion convolution manner based on the initial entity representations to obtain target entity representations of the historical time sequence knowledge graph; the historical information module is used to calculate a historical auxiliary score of the query based on the target entity representations, the relation representations and the historical information of the query in a preset evaluation manner; wherein the historical information of the query is entity information of existing fact quadruples constituted by the query at different timestamps; the historical auxiliary score of the query is a prediction probability of each entity in the historical time sequence knowledge graph as a candidate entity and the query constituting a complete fact quadruple, which is calculated based on the target entity representations, the relation representations and the historical information of the query; the scoring module is used to calculate a score of each entity in the historical time sequence knowledge graph as a candidate entity and the query constituting a complete fact quadruple based on the target entity representations and the relation representations in a preset scoring manner to obtain a global score of the query; and the global score of the query and the historical auxiliary score of the query are calculated in a preset calculation manner to obtain a target score of the query; wherein the target score of the query is a prediction probability of each entity in the historical time sequence knowledge graph as a candidate entity and the query constituting a complete fact quadruple, which is calculated based on the global score and the historical auxiliary score; step S3, taking the historical time sequence knowledge graph and the plurality of queries as inputs, the target score of each query as a prediction output, and performing multi-round iterative training on the initial model according to a preset target function until the model converges to obtain a time sequence knowledge graph reasoning model.

[0009] In some embodiments of the present application, the fact quadruples at the same timestamp in the historical time sequence knowledge graph constitute a time subgraph, and the global historical evolution module comprises a composite graph convolution network and a time sequence evolution submodule, wherein: the composite graph convolution network is configured to extract the structural features of each time subgraph to obtain entity representations of each time subgraph; the time sequence evolution submodule is configured to extract relationship representations of each time subgraph based on the entity representations of each time subgraph and the self-configured static relationship representations according to a preset extraction manner; and further configured to fuse the entity representations of the time subgraph at the previous time stamp with the entity representations of the time subgraph at the current time stamp to obtain initial entity representations of the historical time sequence knowledge graph; and fuse the relationship representations of the time subgraph at the previous time stamp with the relationship representations of the time subgraph at the current time stamp to obtain relationship representations of the historical time sequence knowledge graph.

[0010] In some embodiments of the present application, the preset extraction manner is:

[0011]

[0012] wherein, represents the relationship representations of the time subgraph at the time stamp . represents a vector concatenation operation, represents an average pooling function. represents the entity representations of the time subgraph at the time stamp after fusion processing. represents a set of entities formed by all entities forming fact quadruples with each relationship in the time subgraph at the time stamp . represents a static relationship representation.

[0013] In some embodiments of the present application, the preset evaluation manner is:

[0014]

[0015] wherein,

[0016]

[0017]

[0018] wherein,

[0019]

[0020] wherein, represents a historical auxiliary score, represents a candidate entity set formed by all candidate entities, represents an entity, denotes a relation, denotes the entity information of the existing fact quadruple constituted by the query, denotes a normalization function, denotes a history indicator vector, denotes a guide vector, denotes an activation function, denotes a weight matrix, denotes the representation of an entity in the target entity representation, denotes the representation of a relation in the relation representation, denotes all time subgraphs between time stamp and time stamp in which other entities and entities and relations in the query form fact quadruples, denotes the frequency of other entities and entities and relations in the query form fact quadruples in the time subgraph of time stamp , denotes the frequency of other entities and entities and relations in the query form fact quadruples in the time subgraph of time stamp , denotes the frequency of other entities and entities and relations in the query form fact quadruples in the time subgraph of time stamp ; denotes a constant, denotes a multiplication symbol, denotes an indicator vector corresponding to other entities and relations that do not form fact quadruples with entities and relations in the query, denotes an indicator vector corresponding to other entities and relations

[0021] that form fact quadruples with entities and relations

[0022] in the query.

[0023] wherein, denotes a global score, denotes a target entity representation, denotes a relation representation, denotes a scoring function.

[0024] In some embodiments of the application, the preset calculation manner is:

[0025]

[0026] wherein, denotes the target score, denotes the hyperparameter.

[0027] In some embodiments of the application, the preset target function is:

[0028]

[0029] wherein,

[0030]

[0031]

[0032] wherein, denotes the target function; denotes the entity prediction loss; denotes the historical score loss; denotes the hyperparameter; denotes the number of time subgraphs of the historical time-sequential knowledge graph; denotes a fact quadruple; denotes the head entity; denotes the tail entity; denotes the timestamp; denotes the number of entities of the historical time-sequential knowledge graph; denotes the value whether the entity is consistent with the entity label of the query, if the entity is consistent with the entity label, then , otherwise ; denotes the logarithmic function; denotes the prediction probability that the entity and the entity and the relation in the query constitute a complete fact quadruple in the target score, denotes the prediction probability that the entity and the entity and the relation in the query constitute a complete fact quadruple in the historical auxiliary score.

[0033] According to a second aspect of the present application, a time sequence knowledge graph reasoning method is provided, which comprises the following steps: S1, obtaining a historical time sequence knowledge graph to be processed and constructing one or more queries based on the historical time sequence knowledge graph; S2, performing knowledge reasoning on all queries based on the historical time sequence knowledge graph to be processed by using a time sequence knowledge graph reasoning model constructed by the method of the first aspect of the present application, so as to obtain a target score of each query; and S3, selecting a candidate entity corresponding to a prediction probability meeting a preset requirement from the target score of each query, and constructing a complete fact quadruple with the query as a knowledge reasoning result.

[0034] Compared with the prior art, the present application has the following advantages: (1) a global historical evolution module is provided to extract the historical evolution mode of facts in the time sequence knowledge graph with time, so as to accurately capture the time sequence dependency between facts and help improve the prediction accuracy; (2) an adaptive entity attention module is provided to mine the potential association information between entities in the time sequence knowledge graph, so as to comprehensively capture the structural features of the time sequence knowledge graph and help improve the prediction accuracy; and (3) a historical information module is provided to reduce the candidate space of the prediction candidate entity, which helps improve the prediction accuracy. BRIEF DESCRIPTION OF DRAWINGS

[0035] The embodiments of the present application will be further described below with reference to the accompanying drawings, in which:

[0036] Figure 1 FIG. 1 is a flowchart of a time sequence knowledge graph reasoning model construction method according to an embodiment of the present application;

[0037] Figure 2 FIG. 2 is a schematic diagram of the composition of a time sequence knowledge graph reasoning model according to an embodiment of the present application;

[0038] Figure 3 FIG. 3 is an entity diagram of the internal structure of a time sequence knowledge graph reasoning model according to an embodiment of the present application. DETAILED DESCRIPTION

[0039] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and do not limit the present application.

[0040] As mentioned in the background section, the prior art also has two defects in temporal knowledge graph reasoning, one is the neglect of historical evolution information: without fully considering the historical evolution pattern of the facts in the temporal knowledge graph with time, it is difficult to accurately capture the temporal dependency between facts, affecting the accuracy of prediction; the other is the lack of correlation information between entities: in the reasoning process, the potential correlation information between entities is often not fully utilized, which leads to the inability to fully capture the structural characteristics of the temporal knowledge graph, and further affects the performance of fact prediction.

[0041] In order to solve the above problems, the inventors propose a reasoning model that can fully model historical evolution information and correlation information between entities. In this model, a global historical evolution module, an adaptive entity attention module and a historical information module are set up; wherein the global historical evolution module uses a composite graph convolution network to perform message passing on each time subgraph of the temporal knowledge graph to update the entity representation, and uses a gating unit between adjacent time subgraphs to capture the evolution pattern of the entity, and also uses the entity associated with the relationship to realize the representation evolution of the relationship, thereby fully capturing the semantic information of the temporal knowledge graph; the adaptive entity attention module uses an adaptive graph to mine the potential evolution correlation between entities in the time subgraph evolving with time, and uses a diffusion convolution to apply the mined potential evolution correlation between entities to the high-order message passing of the convolution layer, to realize the joint learning of structure and temporal evolution, thereby fully capturing the structural characteristics of the temporal knowledge graph; the historical information module assists knowledge reasoning by modeling the relevance between queries and historical entities, which can effectively improve the prediction performance of the model.

[0042] In summary, as shown in Figure 1 A temporal knowledge graph reasoning model construction method is provided, the method comprising: step S1, obtaining a historical temporal knowledge graph and constructing a plurality of queries based thereon; wherein the historical temporal knowledge graph comprises a plurality of fact quadruples, each fact quadruple comprising a head entity, a relationship, a tail entity and a timestamp; each query is a missing tail entity or head entity missing fact quadruple, and each query is provided with an entity label corresponding to the missing tail entity or head entity thereof; step S2, constructing an initial model comprising a global historical evolution module, an adaptive entity attention module, a historical information module and a scoring module; step S3, taking the historical temporal knowledge graph and the plurality of queries as input, the target score of each query being the prediction output, and performing multiple rounds of iterative training on the initial model according to a preset target function until the model converges to obtain a temporal knowledge graph reasoning model.

[0043] In order to better understand the present application, each step will be described in detail below in conjunction with specific embodiments.

[0044] I. Step S1

[0045] In the step S1, a historical time-series knowledge graph is acquired and a plurality of queries are constructed based on the historical time-series knowledge graph.

[0046] The historical time-series knowledge graph is a directed multi-relation graph with time stamp information composed of fact quadruples. A fact quadruple is composed of a head entity, a tail entity, a relation between the head entity and the tail entity, and a time stamp. The time stamp indicates the time when the relation between the head entity and the tail entity occurs. The time stamp can be a time node (e.g., 2010) or a time interval (e.g., 2010-2013). Specifically, a historical time-series knowledge graph can be represented as wherein, denotes a set of entities, denotes a set of relations, denotes a set of time stamps, denotes a set of facts. A fact quadruple can be represented as wherein, denotes a head entity, denotes a relation, denotes a tail entity, denotes a time stamp. If the fact quadruples in the historical time-series knowledge graph are sorted in ascending order of time stamps, and the fact quadruples at the same time stamp are integrated into a time subgraph, the complete historical time-series knowledge graph can be represented as a sequence composed of multiple time subgraphs , denotes a time subgraph at a time stamp , denotes a time subgraph at a time stamp , denotes a time subgraph at a time stamp .

[0047] A query is a missing fact quadruple with a missing tail entity or head entity, which can be represented as or or wherein, denotes an entity (a head entity or a tail entity) in the query, denotes a missing entity (a head entity or a tail entity) in the query.

[0048] II. Step S2,

[0049] In the step S2, an initial model is constructed. As shown in Figure 2 , the initial model includes a global historical evolution module, an adaptive entity attention module, a historical information module, and a scoring module. The components of the initial model will be introduced one by one in combination with the internal structure of the initial model shown in Figure 3 .

[0050] 2.1 Global historical evolution module

[0051] The global historical evolution module is used to extract semantic information of the historical time sequence knowledge graph to obtain initial entity representation and relationship representation of the historical time sequence knowledge graph containing historical evolution information. The global historical evolution module is set because, if the missing entity in the query needs to be predicted The global information of the knowledge graph needs to be considered, which includes all adjacent entity information (structural dependency information) of the entity appearing in the same time subgraph, and the evolution mode (time sequence evolution mode) of the fact where the entity is located in the cross-time subgraph.

[0052] In order to enable the global historical evolution module to extract the structural dependency information and the time sequence evolution mode, the inventors propose that a structural dependency sub-module and a time sequence evolution sub-module can be set in the global historical evolution module; wherein the structural dependency sub-module is used to explore the structural features of the time subgraph; and the time sequence evolution sub-module is used to model the time sequence evolution features of the entity in other time subgraphs. Based on this, the global historical evolution module can learn the neighborhood information of the time subgraph after message passing and the evolution information across the time subgraphs contained in the historical time sequence knowledge graph.

[0053] According to one embodiment of the present application, the global historical evolution module comprises a composite graph convolution network and a time sequence evolution sub-module, wherein: the composite graph convolution network is used to extract the structural features of each time subgraph to obtain the entity representation of each time subgraph; the time sequence evolution sub-module is used to extract the relationship representation of each time subgraph based on the entity representation of each time subgraph and the static relationship representation configured by itself according to a predetermined extraction manner; and is further used to fuse the entity representation of the time subgraph of the previous time stamp and the entity representation of the time subgraph of the current time stamp to obtain the initial entity representation of the historical time sequence knowledge graph; and fuse the relationship representation of the time subgraph of the previous time stamp and the relationship representation of the time subgraph of the current time stamp to obtain the relationship representation of the historical time sequence knowledge graph.

[0054] ​The composite graph convolution network explores the internal structural features of each time subgraph as a structural dependency submodule. Specifically, the composite graph convolution network extracts the entity representation of each time subgraph according to the multiple network layers set by itself in the following manner: initial embedding: the first network layer randomly initializes the initial embedding representation of each entity and each relationship in the time subgraph; message construction: in the second to last network layer, each network layer constructs the embedding representation of the message composed of each pair of connected head entity and tail entity according to the embedding representation of each entity and the embedding representation of each relationship transmitted by the previous network layer; message aggregation: in the second to last network layer, each network layer aggregates all messages pointing to the same tail entity according to the embedding representation of all messages constructed in an additive manner to obtain the embedding representation of multiple comprehensive messages; embedding update: in the second to last network layer, each network layer updates the embedding representation of each entity in the time subgraph based on the embedding representation of the multiple comprehensive messages obtained by aggregation processing and passes it to the next network layer. The entity representation of each time subgraph is the output of the last network layer of the composite graph convolution network.

[0055] It should be noted that there may be entities without edge relationship in the time subgraph. For such entities without edge relationship, the composite graph convolution network will give a self-loop edge to make the entity evolve and update itself, which can be formalized as:

[0056]

[0057] wherein, represents the embedding representation of the tail entity of the time subgraph in the first network layer; represents the embedding representation of the head entity of the time subgraph in the first network layer; represents the embedding representation of the relationship between the head entity and the tail entity of the time subgraph in the first network layer; represents the ReLU activation function; is a regularization constant, and its numerical value is equal to the in-degree of the tail entity; represents the fact set of the time subgraph; represents the first weight matrix of the first network layer; represents the embedding representation of the head entity of the time subgraph in the second network layer; represents the embedding representation of the relationship between the head entity and the tail entity of the time subgraph in the second network layer; represents the second weight matrix of the second network layer, ​​​​​​​​​​​Indicates the first Timestamps in Layered Networks Tail entity of the next time subgraph Embedded representation.

[0058] The temporal evolution submodule is used to learn cross-timestamp temporal evolution patterns based on the entity representations of each temporal subgraph obtained from the composite graph convolutional network. This is to fully learn information reflecting the evolutionary trend of each entity, such as... Figure 3 As shown, the temporal evolution submodule adopts an RNN-based structure for temporal evolution pattern learning. The temporal evolution submodule includes multiple gating units, with each gating unit retaining only the update gate. The temporal evolution submodule extracts the relation representation of each temporal subgraph based on its entity representation and its own configured static relation representation according to a preset extraction method. Furthermore, it uses the update gate to retain entity representations of temporal subgraphs with adjacent timestamps to varying degrees to obtain the initial entity representation of the historical temporal knowledge graph, and also uses the update gate to retain relation representations of temporal subgraphs with adjacent timestamps to varying degrees to obtain the relation representation of the historical temporal knowledge graph.

[0059] According to one embodiment of the present invention, the preset extraction method is as follows:

[0060]

[0061] in, Indicates timestamp as Relational representation of time subgraphs, This represents a vector concatenation operation. This represents the average pooling function; The timestamp after merging is indicated as Entity representation of the time subgraph; Indicates timestamp as The set of entities that form fact quadruples with each relation in the time subgraph; This represents a static relation. It should be noted that if there are no entities in the temporal subgraph that form a fact quadruple with the relation, the relation can be initialized as a zero vector.

[0062] To better understand the working principle of the temporal evolution submodule, the following will combine... Figure 3 The content displayed in the global historical evolution module will be explained.

[0063] like Figure 3 As shown, the temporal evolution submodule performs evolutionary modeling of entities in the temporal subgraph by preserving entity representations of adjacent timestamps to varying degrees through update gates (gating units). Figure 3The evolution modeling process is briefly described in the context of the entity evolution modeling process in Figure 3 , denotes the entity representation of the time subgraph after the updated gate fusion process at time stamp , denotes the entity representation of the time subgraph after the updated gate fusion process at time stamp , denotes the entity representation of the time subgraph after the updated gate fusion process at time stamp . Wherein, , and The evolution modeling process can be represented as:

[0064]

[0065]

[0066] wherein, denotes the updated gate parameters at time stamp , denotes the dot product operation, denotes the entity representation of the time subgraph at time stamp (computed by the composite graph convolution network process), denotes the entity representation of the time subgraph after the updated gate fusion process at time stamp , denotes the parameter matrix constructed for the update, denotes the bias matrix, denotes the sigmoid function.

[0067]

[0068]

[0069] wherein, denotes the updated gate parameters at time stamp , denotes the entity representation of the time subgraph at time stamp .

[0070]

[0071]

[0072] wherein, denotes the updated gate parameters at time stamp , denotes the entity representation of the time subgraph at time stamp entity representations of the time sub-graphs at the adjacent timestamps by the update gate to different degrees.

[0073] It should be noted that the initial entity representations of the historical time sequence knowledge graph containing historical evolution information can be finally obtained by the update gate to different degrees of preservation of the entity representations of the time sub-graphs at the adjacent timestamps, and the initial entity representations of the historical time sequence knowledge graph are the entity representations of the time sub-graphs at the last timestamp after the fusion processing by the update gate.

[0074] As shown in Figure 3 , the time sequence evolution sub-module also extracts the relationship representations of each time sub-graph based on the entity representations of each time sub-graph and the static relationship representations configured by itself according to a preset extraction manner, and evolves the relationships in the time sub-graphs by evolving modeling the relationships representations of the time sub-graphs at the adjacent timestamps by the update gate to different degrees. Still taking the content shown in Figure 3 to briefly explain the relationship evolution modeling process, in Figure 3 , represents the relationship representations of the time sub-graphs at the timestamp after the fusion processing by the update gate, represents the relationship representations of the time sub-graphs at the timestamp after the fusion processing by the update gate, represents the relationship representations of the time sub-graphs at the timestamp after the fusion processing by the update gate. Wherein, , and the evolution modeling process can be represented as:

[0075]

[0076]

[0077]

[0078] Wherein, represents the information fusion processing by the update gate, represents the relationship representations of the time sub-graphs at the timestamp after the fusion processing by the update gate, represents the relationship representations of the time sub-graphs at the timestamp , represents the entity set formed by all entities in the time sub-graphs at the timestamp and each relationship to form a fact quadruple, represents the fact set of the time sub-graphs at the timestamp .

[0079]

[0080]

[0081]

[0082] in, Represents timestamp Relationship representation of the time subgraph below. Represents timestamp The entity set in the time subgraph below, consisting of all entities that form fact quadruples with each relation. Represents timestamp The fact set of the next time subgraph.

[0083]

[0084]

[0085]

[0086] in, Represents timestamp Relationship representation of the time subgraph below. Represents timestamp The entity set in the time subgraph below, consisting of all entities that form fact quadruples with each relation. Represents timestamp The fact set of the next time subgraph.

[0087] It should be noted that by using update gates to preserve the relational representations of adjacent time-stamp subgraphs to varying degrees, the final relational representation of the historical time-series knowledge graph, containing historical evolutionary information, can be obtained. Furthermore, the relational representation of the historical time-series knowledge graph is the relational representation of the time-series subgraph corresponding to the last time-stamp after update gate fusion processing. It should also be noted that some relations in the historical time-series knowledge graph have weak time-varying characteristics, such as "(A, CapitalOf, B)", which will not change for a long time in the future. For these relations, a static representation can be set. Based on this, in the above relation evolution modeling process, a static relational representation is set for all relations. Furthermore, it uses static relation representations for relation evolution learning, ensuring that it can continuously learn dynamically updated relation information during the learning process, while keeping those relations that do not frequently disappear or reappear unchanged.

[0088] 2.2 Adaptive Entity Interest Module

[0089] After the global historical evolution module processes the historical time sequence knowledge graph, global structure information (explicit association between entities) and evolution information of entities in the historical time sequence knowledge graph are obtained. However, there may be few edge relationships between some similar attribute entities in the historical time sequence knowledge graph. For example, a country entity A has few edge relationships with other entities. In order to predict the future behavior of the country entity A, the behavior of other country entities with rich information can be referred to. Therefore, it is not enough to only learn the explicit association between entities. The implicit (potential) association between entities in the historical time sequence knowledge graph also needs to be learned. Based on this, the adaptive entity attention module is set to learn the potential association between entities.

[0090] The adaptive entity attention module is used to extract the potential dependency relationship between each entity in the historical time sequence knowledge graph in the form of adaptive graph and diffusion convolution based on the initial entity representation, so as to obtain the target entity representation corresponding to the historical time sequence knowledge graph.

[0091] Among them, the adaptive graph calculates the similarity between each entity in the historical time sequence knowledge graph through dot product similarity to measure the correlation between entities, and constructs a new edge relationship by taking the similarity between each entity as the weight of the edge, to obtain a new historical time sequence knowledge graph as shown in the following formula: Figure 3 The adaptive entity attention module displays a new historical time sequence knowledge graph example (the dashed part in the example is the constructed new edge relationship). The diffusion convolution utilizes the adjacency relationship of each entity on the new historical time sequence knowledge graph to propagate the feature information of the entity to other neighbor entities, so as to obtain the diffusion entity representation, so as to further capture the potential association between each entity on the historical time sequence knowledge graph. It should be noted that since the adaptive graph and the diffusion convolution are two mature technologies, therefore, in the present application, only the basic principles of them are briefly outlined, and no more details are given.

[0092] 2.3 Historical information module

[0093] Based on the relationship representation and the target entity representation of the historical time sequence knowledge graph obtained after processing by the global historical evolution module and the adaptive entity attention module, the prediction probability of each entity in the historical time sequence knowledge graph as a candidate entity and the query to form a complete fact quadruple can be predicted. However, the most important prior information for the prediction process is actually the repeated facts in the past. For example, to predict the champion of basketball game A at time t (basketball game, champion,?, t), the team that has won the championship at time t-1, t-2 and other past times is more likely to win the championship. Based on this, the historical information module is introduced, which gives important known entities a higher attention degree by using the important prior information of the repetition frequency, to assist the final prediction.

[0094] The historical information module is configured to calculate a historical auxiliary score of the query based on the target entity representation, the relationship representation, and historical information of the query according to a preset evaluation mode; the historical information of the query refers to entity information of an existing fact quadruple constituted by the query at different timestamps; and the historical auxiliary score of the query refers to a prediction probability of each entity in a historical time sequence knowledge graph as a candidate entity and the query constituting a complete fact quadruple, which is calculated based on the target entity representation, the relationship representation, and the historical information of the query.

[0095] According to an embodiment of the present application, the preset evaluation mode is:

[0096]

[0097] wherein,

[0098]

[0099]

[0100] wherein,

[0101]

[0102] wherein, represents the historical auxiliary score, represents a candidate entity set constituted by all candidate entities, represents an entity, represents a relationship, represents entity information of an existing fact quadruple constituted by the query at different timestamps, represents a normalization function, a historical indicator vector, represents a guide vector, represents an activation function, represents a weight matrix, represents a representation of an entity in the target entity representation, represents a representation of a relationship in the relationship representation, represents a cumulative frequency of other entities and entities and relationships in the query forming a fact quadruple in all time subgraphs between a timestamp and a timestamp , represents a frequency of other entities and entities and relationships in the query forming a fact quadruple in a time subgraph of a timestamp , represents a frequency of other entities and entities other entities in the time subgraph and the entities in the query and relations the frequency of forming fact quadruples, denotes a timestamp other entities in the time subgraph and the entities in the query and relations the indication vector corresponding to the fact quadruples formed; denotes a constant, denotes a multiplication symbol, other entities not in the query and relations the indication vector corresponding to the fact quadruples formed, other entities in the query and relations the indication vector corresponding to the fact quadruples formed. Wherein, denotes the prediction (estimation) probability of each entity in the candidate entity set , relation and entity information as a candidate entity and entity and relation constitute a fact under the condition that entity . For example, the candidate entity set includes , , , , , , , , and , and the entity information includes , , and , then denotes the prediction (estimation) probability of , , and , , , , , , , , , , , and as a candidate entity and entity and relation The prediction (estimate) probability of the fact. Wherein, The corresponding indication vector and The number of elements contained in the corresponding indication vector is consistent, assuming there are five entities , , , and , and and do not form a fact quadruple with entity and relationship , The corresponding indication vector can be represented as [1, 1, 0, 0, 0], The corresponding indication vector can be represented as [0, 0, 1, 1, 1], otherwise The corresponding indication vector and The corresponding indication vector is determined according to the foregoing, and the present application will not be described in more detail.

[0103] It should be noted that the principle of the preset evaluation method can be understood as follows: the estimated probability of the entity forming a known fact with the query is raised, and the estimated probability of the entity not forming a known fact with the query is reduced. For example, for the query , the candidate entities include , , , , , , , , and , wherein, , , and are entities that have formed known facts with the query at the time , when calculating the historical auxiliary score, , , and , the estimated probability of the candidate entity and the query forming a complete fact quadruple is higher than that of other candidate entities. Furthermore, if the frequency of entity forming a known fact with the query before the time is higher than the frequency of , and forming a known fact with the query, the estimated probability of entity as a candidate entity and the query forming a complete fact quadruple is also higher than that of , and an estimated probability that the candidate entity and the query constitute a complete fact quadruple, wherein the higher the estimated probability that the candidate entity and the query constitute a complete fact quadruple, the higher the probability that the candidate entity is predicted to be the correct entity.

[0104] It should be further noted that the historical information module gives a higher estimated probability to the candidate entity that has formed a fact with the query by using prior information, which can greatly reduce the candidate space of the predicted candidate entity and achieve more accurate entity prediction.

[0105] 2.4 Scoring module

[0106] The scoring module is configured to calculate a score of each entity in the historical time sequence knowledge graph as a candidate entity and the query constituting a complete fact quadruple based on the target entity representation and the relationship representation according to a preset scoring method, to obtain a global score of the query; and calculate the global score of the query and the historical auxiliary score of the query according to a preset calculation method to obtain a target score of the query; wherein the target score of the query is a predicted probability that each entity in the historical time sequence knowledge graph as a candidate entity and the query constitutes a complete fact quadruple.

[0107] According to an embodiment of the present application, the preset scoring method is:

[0108]

[0109] wherein, represents the global score, represents the target entity representation, represents the relationship representation, represents the scoring function. Wherein, Other scoring functions such as Euclidean distance, Manhattan distance, cosine similarity, etc. can also be replaced, and the present application does not make special limitations.

[0110] According to an embodiment of the present application, the preset calculation method is:

[0111]

[0112] wherein, represents the target score, represents the hyperparameter. It should be noted that the value of the hyperparameter is adjusted according to the actual situation, and the present application does not make special limitations.

[0113] It should be noted that the scoring module combines the historical auxiliary score output by the historical information module to predict the predicted probability that each candidate entity and the query constitute a complete fact quadruple, which can achieve more accurate entity prediction.

[0114] III. Step S3,

[0115] In step S3, the historical time-series knowledge graph and a plurality of queries are taken as inputs, the target score of each query is taken as a prediction output, and the initial model is trained in multiple rounds of iterations according to a preset target function until the model converges to obtain a time-series knowledge graph reasoning model.

[0116] According to an embodiment of the present application, the preset target function is:

[0117]

[0118] wherein,

[0119]

[0120]

[0121] wherein, denotes the target function; denotes the entity prediction loss; denotes the historical score loss; denotes the hyperparameter; denotes the number of time subgraphs of the historical time-series knowledge graph; denotes a fact quadruple; denotes the head entity; denotes the tail entity; denotes the timestamp; denotes the number of entities of the historical time-series knowledge graph; denotes the value of whether the entity is consistent with the entity label of the query, if the entity is consistent with the entity label, then , otherwise ; denotes the logarithmic function; denotes the prediction probability that the entity and the entity and the relation in the target score constitute a complete fact quadruple, denotes the prediction probability that the entity and the entity and the relation in the historical auxiliary score constitute a complete fact quadruple. The hyperparameter may be set to 1, or adjusted to other numerical values according to actual conditions, which is not specially limited in the present application.

[0122] The time sequence knowledge graph reasoning model constructed by the foregoing embodiment can realize query prediction, and based on this working characteristic, the application further provides a time sequence knowledge graph reasoning method, which comprises the following steps: step T1, obtaining a historical time sequence knowledge graph to be processed and constructing one or more queries based on the historical time sequence knowledge graph to be processed; step T2, using the time sequence knowledge graph reasoning model constructed by the foregoing embodiment to perform knowledge reasoning on all the queries based on the historical time sequence knowledge graph to be processed, so as to obtain a target score of each query; and step T3, selecting a candidate entity corresponding to a prediction probability meeting a preset requirement and a query to form a complete fact quadruple as a knowledge reasoning result from the target score of each query.

[0123] The application has the following beneficial effects: (1) the global historical evolution module is arranged to extract the historical evolution mode of the facts in the time sequence knowledge graph with time change, so that the time sequence dependency between the facts can be accurately captured, and the prediction accuracy can be improved; (2) the adaptive entity attention module is arranged to mine the potential association information between the entities in the time sequence knowledge graph, so that the structural features of the time sequence knowledge graph can be comprehensively captured, and the prediction accuracy can be improved; and (3) the historical information module is arranged to reduce the candidate space of the prediction candidate entity, and the prediction accuracy can be improved.

[0124] It should be noted that although the above describes the steps in a specific order, it does not mean that the steps must be performed in the above specific order, and in fact, some of the steps can be performed concurrently or even in a changed order, as long as the required functions can be achieved.

[0125] The application can be a system, a method and / or a computer program product. The computer program product can include a computer readable storage medium having computer readable program instructions stored therein for causing a processor to implement various aspects of the application.

[0126] The computer readable storage medium can be a tangible device that stores and / or reads instructions used by a processing device. The computer readable storage medium may, for example, include, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the above. More specific examples (non-exhaustive list) of the computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device, such as a punch card or a hole in a groove structure, having instructions stored thereon, and any suitable combination of the above.

[0127] Having described various embodiments of the application, it is to be understood that the above description is meant not to limit and not to encompass all of the possible embodiments covered by the claims. Many modifications and variations of this application can be apparent to those of ordinary skill in the art without departing from the spirit and scope of the described embodiments. It is intended that the scope of the application should only be limited by the appended claims.

Claims

1. A method for constructing a temporal knowledge graph reasoning model, characterized in that, The method includes: Step S1: Obtain the historical time-series knowledge graph and construct multiple queries based on it; wherein, the historical time-series knowledge graph includes multiple fact quadruples, each fact quadruple includes a head entity, relation, tail entity and timestamp; each query is a missing fact quadruple with a missing tail entity or head entity, and each query is set with an entity label corresponding to its missing tail entity or head entity; Step S2: Construct the initial model, which includes a global historical evolution module, an adaptive entity attention module, a historical information module, and a scoring module, wherein: The global historical evolution module is used to extract semantic information from the historical temporal knowledge graph in order to obtain the initial entity representation and relation representation containing historical evolution information corresponding to the historical temporal knowledge graph. The adaptive entity attention module is used to extract the potential dependencies between entities in the historical time-series knowledge graph based on the initial entity representation, and then extract the target entity representation corresponding to the historical time-series knowledge graph by using an adaptive graph and diffusion convolution. The historical information module is used to calculate the historical auxiliary score of the query based on the target entity representation, relation representation and query historical information according to a preset evaluation method; wherein, the query historical information is the entity information of the existing fact quadruple formed with the query at different timestamps; the query historical auxiliary score is the predicted probability of each entity in the historical time series knowledge graph calculated based on the target entity representation, relation representation and query historical information forming a complete fact quadruple with the query as a candidate entity. The scoring module is used to calculate the score of each entity in the historical time-series knowledge graph as a candidate entity and the query forming a complete fact quadruple based on the target entity representation and relation representation according to a preset scoring method, so as to obtain the global score of the query; and to calculate the global score of the query and the historical auxiliary score of the query according to a preset calculation method to obtain the target score of the query; wherein, the target score of the query is the predicted probability of each entity in the historical time-series knowledge graph as a candidate entity and the query forming a complete fact quadruple based on the global score and the historical auxiliary score. Step S3: Using the historical time-series knowledge graph and multiple queries as inputs, and the target score of each query as the predicted output, the initial model is trained iteratively in multiple rounds according to the preset objective function until the model converges to obtain the time-series knowledge graph reasoning model.

2. The method according to claim 1, characterized in that, In the historical time-series knowledge graph, fact quadruples at the same timestamp constitute a time subgraph. The global historical evolution module includes a composite graph convolutional network and a time-series evolution submodule, wherein: The composite graph convolutional network is used to extract the structural features of each temporal subgraph to obtain the entity representation of each temporal subgraph; The temporal evolution submodule is used to extract the relation representation of each temporal subgraph according to a preset extraction method based on the entity representation of each temporal subgraph and the static relation representation configured by itself; it is also used to fuse the entity representation of the temporal subgraph with the timestamp of the previous moment with the entity representation of the temporal subgraph with the timestamp of the current moment to obtain the initial entity representation of the historical temporal knowledge graph; and to fuse the relation representation of the temporal subgraph with the timestamp of the previous moment with the relation representation of the temporal subgraph with the timestamp of the current moment to obtain the relation representation of the historical temporal knowledge graph.

3. The method according to claim 2, characterized in that, The preset extraction method is as follows: in, Indicates timestamp as Relational representation of time subgraphs, This represents a vector concatenation operation. This represents the average pooling function; The timestamp after merging is indicated as Entity representation of the time subgraph; Indicates timestamp as The set of entities that form fact quadruples with each relation in the time subgraph; This represents a static relationship.

4. The method according to claim 3, characterized in that, The preset evaluation method is as follows: in, in, in, Indicates historical auxiliary scores, This represents the candidate entity set consisting of all candidate entities. Represents an entity, Indicates a relationship. This represents the entity information of the existing fact quadruple formed by the query at different timestamps. Represents the normalization function. Historical indicator vector, Represents the guiding vector. This represents the activation function. Represents the weight matrix. In the representation of the target entity, the entity is represented. The representation, Representation of relations in relation representation The representation, Represents timestamp to timestamp Other entities in the time subgraph between the two time periods and the entities in the query and relationships The cumulative frequency of forming fact quadruplets Represents timestamp Other entities in the time subgraph and entities in the query and relationships The frequency of forming fact quadruplets Represents timestamp Other entities in the time subgraph and entities in the query and relationships The frequency of forming fact quadruplets Represents timestamp Other entities in the time subgraph and entities in the query and relationships The frequency of forming fact quadruplets; Represent a constant. Indicates the product symbol. Indicates other entities not found in the query. and relationships Form the indicator vector corresponding to the fact quadruple. Indicates other entities related to the query. and relationships The indicator vector corresponding to the fact quadruple is formed.

5. The method according to claim 4, characterized in that, The preset scoring method is as follows: in, This represents the overall score. Represents the target entity. Representing the relational characterization, This represents the scoring function.

6. The method according to claim 5, characterized in that, The preset calculation method is as follows: in, Indicates the target score. This represents hyperparameters.

7. As described in claim 6, characterized in that, The preset objective function is: in, in, Represent the objective function; Indicates the entity's predicted loss; Indicates historical score loss; Indicates hyperparameters; The number of temporal subgraphs in a historical time-series knowledge graph; Represent a fact quadruple; Indicates the head entity; Indicates the tail entity; Represents a timestamp; Represents the number of entities in a historical time-series knowledge graph; Represents the entities among all candidate entities Whether the value matches the entity label in the query. If the entity... If consistent with the entity label Conversely ; Represents a logarithmic function; Indicates that the target score is based on entities Entities in the query and relationships The predicted probability of forming a complete fact quadruple. Indicating that the historical auxiliary rating is based on entities Entities in the query and relationships The predicted probability that constitutes a complete fact quadruple.

8. A temporal knowledge graph reasoning method, characterized in that, The method includes: Step T1: Obtain the historical time-series knowledge graph to be processed and construct one or more queries based on it; Step T2: The time-series knowledge graph reasoning model constructed using any one of the methods described in claims 1-7 performs knowledge reasoning on all queries based on the historical time-series knowledge graph to be processed, in order to obtain the target score for each query; Step T3: Select candidate entities that meet the preset requirements and whose predicted probabilities are selected from the target scores of each query to form a complete fact quadruple as the knowledge reasoning result.

9. A computer-readable storage medium, characterized in that, It stores a computer program that can be executed by a processor to implement the steps of the method according to any one of claims 1-8.

10. An electronic device, characterized in that, include: One or more processors, and Memory, wherein the memory is used to store executable instructions; The one or more processors are configured to implement the steps of the method of any one of claims 1-8 by executing the executable instructions.

Images