Explainable event prediction method based on graph attention network and Hawkes process

Abstract

The present application relates to an interpretable event prediction method based on a graph attention network and a Hawkes process. The method adopts the graph attention mechanism and the Hawkes process in the event prediction timing point process, constructs the time prediction process as a process of reasoning subgraph expansion, takes the node to be predicted as the subgraph base point, combines the attention value with the conditional intensity function value obtained by the Hawkes process as the basis for subgraph search expansion and node embedding vector update, infers the context of event development, and uses this as a forward condition to reason about future events. This method not only pays attention to the relative time interval information between events, further improving the accuracy of the prediction results, but also can give a clear reasoning path, making the prediction of events interpretable, improving the prediction accuracy of urban safety events, and reducing the adverse effects of safety events.

Description

Technical Field

[0001] The present application relates to the field of information technology, and in particular to an explainable event prediction method based on graph attention network and Hawkes process. Background Art

[0002] With the acceleration of urbanization, urban safety issues are becoming increasingly prominent. If urban emergencies can be accurately predicted, the city's ability to respond to emergencies can be improved, ensuring that the city's lifeline runs more safely.

[0003] The knowledge graph is essentially a semantic network that reveals the relationship between different entities. It contains entities, attributes, and various semantic relationships between them. It is not only a set of artificial intelligence technology systems, but also a mode of knowledge organization and expression, and a structured multi-related knowledge base. The knowledge graph has extremely strong expressive power and modeling flexibility. It is a human-recognizable and machine-friendly knowledge representation. It focuses on the relationship between events and can support the semantic intercommunication of massive multi-source heterogeneous and heterogeneous data, which makes computer reasoning possible.

[0004] Temporal knowledge graph prediction is to predict the missing event elements in the future based on historical event tuples. With the development of deep learning, it has shown a good development trend in recent years. However, most models focus on optimizing entity representation and time embedding to improve the accuracy of predictions, and there is less exploration of explainable predictions with clear reasoning paths.

[0005] According to the development history and characterization methods of knowledge graph prediction, the time series knowledge graph prediction methods can be divided into several types, including prediction methods based on translation ideas, prediction methods based on tensor decomposition, prediction methods based on time point processes, and prediction methods based on graph neural networks.

[0006] The idea of ​​migration is to embed the entities and relations of multi-relational data into a low-dimensional vector space, where the vector sum of the head entity vector and the relation vector is close to the tail entity vector. The idea of ​​tensor decomposition is to believe that multiple binary relations can be directly represented by a three-dimensional tensor. The idea of ​​the sequential point process is to regard the appearance of entities and relations in the sequential knowledge graph as a probabilistic event, which is represented as a conditional strength function With the survival function Results , historical evolution characteristics, time intervals, etc. can all affect its probability. The idea of ​​graph neural network is to use graph convolutional networks, graph attention networks and other graph neural network methods to extract knowledge graph snapshots. Based on the structural features of the graph, neural networks such as recurrent neural network (RNN), attention mechanism, long short-term memory network (LSTM) are used to learn the evolutionary features between each graph snapshot, and then predict events. Although the above methods have good accuracy in predicting the results of entities and relationships, the predictive ability is poor. In recent years, interpretable knowledge graph reasoning methods have been continuously updated. Rajarshi Das et al. used reinforcement learning methods to find the reasoning path by random walk, and then realized event prediction. Xiang Wang et al. proposed the KPRN network, which generates path representation by combining the semantics of entities and relationships, and uses the sequential dependency relationship within the path to predict events by reasoning the path. These methods mainly focus on static knowledge graphs. For time series knowledge graphs, the XERT model proposed by Zhen Han is interpretable, but its model ignores the influence of the relative time interval between events on the probability of event occurrence during the reasoning process. Summary of the invention

[0007] Based on this, it is necessary to provide an explainable event prediction method based on graph attention network and Hawkes process to address the above technical problems.

[0008] An explainable event prediction method based on graph attention network and Hawkes process, the method comprising:

[0009] Obtain the original time series knowledge graph and the event quadruple to be predicted; the original time series knowledge graph quadruple data is text data; the original time series knowledge graph includes several event quadruple, event quadruple: head entity, relationship predicate, tail entity, timestamp; the original time series knowledge graph is a time series knowledge graph constructed based on a series of events in the security warning process.

[0010] The event quadruple to be predicted is used as the base point of the reasoning subgraph, the one-hop prior neighbor node of the base point is used as the first-order extension node of the reasoning subgraph, and the relationship predicate is used as the attribute of the edge between two nodes.

[0011] Embed the entities and relation predicates of the base node and its first-order expansion nodes.

[0012] The embedding vector of the base point and its first-order extended node and the embedding vector of the relationship predicate of the node edge are concatenated as the input of the graph attention mechanism to calculate the attention value between nodes regarding the event to be inferred.

[0013] The entity embedding vectors of the base point and its extended nodes are concatenated with their relation predicate embedding vectors respectively, which are used as the input of the Hawkes process to calculate the conditional strength values ​​between the nodes regarding the event to be inferred.

[0014] The sum of the attention value and the conditional strength value is used as the likelihood value of the expanded node.

[0015] The node embedding is updated according to the neighborhood information aggregated according to the possibility value, and the first-order expansion node is expanded to obtain the second-order expansion node, and then enter the next round of node expansion. After multiple rounds of node expansion, the node entity with the largest possibility value about the base point is selected as the entity of the quadruple of the event to be predicted, and the prediction of the event to be predicted is completed.

[0016] The above-mentioned interpretable event prediction method based on graph attention network and Hawkes process adopts the graph attention mechanism and the Hawkes process in the event prediction time sequence point process, constructs the time prediction process as the process of reasoning subgraph expansion, takes the node to be predicted as the subgraph base point, combines the attention value and the conditional intensity function value obtained by the Hawkes process as the basis for subgraph search expansion and node embedding vector update, infers the context of event development, and uses this as a forward condition to reason about future events. This method not only pays attention to the relative time interval information between events, further improving the accuracy of the prediction results, but also can give a clear reasoning path, making the prediction of events interpretable, improving the prediction accuracy of urban safety events, and reducing the adverse effects of safety events. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 1 is a flowchart of an explainable event prediction method based on a graph attention network and Hawkes process in one embodiment;

[0018] Figure 2 A schematic diagram of a reasoning subgraph expansion process in another embodiment;

[0019] Figure 3 A model framework diagram of an explainable event prediction method based on a graph attention network and Hawkes process in another embodiment;

[0020] Figure 4 A schematic diagram showing a comparison between an explainable event prediction method based on a graph attention network and Hawkes process and other models in another embodiment;

[0021] Figure 5 This is a schematic diagram of the comparison effect between AHP-H-only and AHP on various data sets in another embodiment;

[0022] Figure 6 FIG. 4 is a schematic diagram showing the comparative effects of AHP-A-only and AHP on various data sets in another embodiment. DETAILED DESCRIPTION

[0023] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0024] The temporal knowledge graph can be described as a collection of event quads with timestamps, namely (head entity, relation predicate, tail entity, timestamp). Events can be formalized , used to indicate events In timestamp t The moment occurs (or is effective). s , r ∈E, o ∈R, t ∈T={0,1,2,…}represents the head entity respectively s , tail entity o , relational predicate r , Timestamp t , and E, R, T represent finite sets of entities, relation predicates, and timestamps, respectively. The temporal knowledge graph can be represented as G, where each event quadruple represents an edge of the temporal knowledge graph.

[0025] The task of event prediction is to predict the missing entities or relations of event quadruple at future timestamps based on the events observed in the past. That is, for a given event quadruple to be predicted , given a set of event quads G Under the premise of G Prior knowledge before time t Infer the missing entities in the event quad to be predicted o The specific implementation is to regard all entities as possible prediction objects, combine them with the event quadruple to be predicted, sort the combined event quadruple according to prior knowledge, and select the entity with the highest possibility of event quadruple as the result to be predicted.

[0026] This application proposes an interpretable event prediction method based on graph attention network and Hawkes process to predict events of the form . The predicted event is used as the starting point of the reasoning subgraph; the attention score between events and the conditional strength function value of the event occurrence are combined as the probability value of time occurrence, and the edges that may contain the predicted entity are iteratively sampled to expand the subgraph, and the probability value of the event occurrence is propagated along the extended edge. Through multiple rounds of expansion, the missing objects are predicted from the extended nodes. Therefore, the reasoning subgraph can be regarded as a path explanation of entity prediction. In order to make the subgraph expand the edges of interest, this method designs a graph attention mechanism and a Hawkes process to jointly enhance and expand the edges of interest from different dimensions. Specifically, we use time encoding to encode entity and relationship predicates, select related entities and relationships before the predicted timestamp as extended edges, use a graph attention network to calculate the attention score of the extended edge layer, and use the Hawkes process to calculate the conditional strength value of the impact of different time intervals on the occurrence of events. The conditional strength value and attention score are combined to predict future knowledge. The next-level extended graph of the reasoning subgraph is used as prior knowledge, and the nodes and encodings are updated by aggregating prior knowledge. This method focuses on the impact of the relative time interval between events on the occurrence of events, and objectively shows the temporal impact of events that have occurred on events that have not occurred. It can give a clear reasoning path for the predicted events, so that the reasoning results can be traced back to the reasoning path to find the events that have a greater impact on the predicted results.

[0027] In one embodiment, Figure 1 As shown, an explainable event prediction method based on graph attention network and Hawkes process is provided, and the method includes the following steps:

[0028] Step 100: Obtain the original time series knowledge graph and the event quadruple to be predicted; the original time series knowledge graph quadruple data is text data; the original time series knowledge graph includes several event quadruple, event quadruple: head entity, relationship predicate, tail entity, timestamp; the original time series knowledge graph is a time series knowledge graph constructed based on a series of events in the security warning process.

[0029] Specifically, a series of prior events will appear before an urban security incident occurs. Based on the learning of the laws of prior events, possible urban security incidents can be predicted. This method can take into account the impact of the correlation between time intervals and events themselves on the occurrence of events, and can provide a reasoning path for predicting events, making the prediction of social security incidents more explainable.

[0030] For example, in order to predict the safety risks of important infrastructure such as drainage, water supply, gas, heating, bridges, buildings, tunnels, integrated pipelines, rivers and lakes in a city, the urban safety incidents that have occurred in the region in recent years are summarized and analyzed to form a time-series knowledge graph of urban safety incidents in the region. The method of this application can be used to predict the types of events that may occur at a certain time in the future (such as gas pipeline leakage and explosion, water supply and drainage network leakage, collapse of old houses, reservoir dam collapse, urban flooding, waterlogging, etc.), or the location of the event, as well as the location of the event, so that decision makers can prepare response plans in advance. At the same time, this method can also provide a clear event reasoning path, so that decision makers can clearly understand the context of the event, and also enable decision makers to take action in advance to reduce the negative impact of the event.

[0031] An example of a city safety event quadruple is as follows: ("City A", "Gas pipeline leakage and explosion", "Section B of a gas company's pipeline", "2024-12-20") indicates that a gas pipeline leakage and explosion occurred in Section B of a gas company's pipeline in City A on December 20, 2024.

[0032] Step 102: Use the event quadruple to be predicted as the base point of the reasoning subgraph, use the one-hop prior neighbor node of the base point as the first-order extension node of the reasoning subgraph, and use the relationship predicate as the attribute of the edge between two nodes.

[0033] Specifically, this method constructs the reasoning process for event prediction as a process of expanding the reasoning subgraph.

[0034] For a given event to be predicted , build an inference subgraph to infer and predict events. Let E represent the set of all entities. e represents an entity in the entity set E. It is represented by a combination of entity and time, and the node is represented as , the node's one-hop priori neighbor set is The node’s one-hop a posteriori neighbor set is .

[0035] Align the entity of the event to be predicted As the base point of the reasoning subgraph, As the extension point of the node, a directed graph is constructed, with the direction from the node with an earlier timestamp to the node with a later timestamp. As the attribute of the edge in the directed graph, by expanding the node, the node entity with the largest probability value about the base point is found as the prediction object.

[0036] Use the quadruple to be predicted as the basis point of the inference subgraph , its one-hop prior neighbor node( ,in As the base point The set of one-hop prior neighbors of the inference subgraph is used as the next-level expansion node, and the relationship predicate is used as the attribute of the edge between the two nodes. For the base point, its relationship predicate As the attribute of the base point's own unique edge, calculate the likelihood value of the expanded node, expand the node according to its likelihood value, and also aggregate the neighborhood information according to its likelihood value to update the node embedding. The node entity with the largest probability value is taken as the prediction result of the model. The schematic diagram of the reasoning subgraph expansion process is as follows: Figure 2 shown.

[0037] Step 104: embed the entities and relationship predicates of the base point and its first-order extended nodes.

[0038] Specifically, in order to fully mine the information contained in the knowledge graph, it is necessary to embed entities and relationship predicates. Entity and relationship predicate embedding is to represent entities and relationship predicates as low-dimensional vectors that are suitable for them.

[0039] Step 106: The embedding vectors of the base point and its first-order extended nodes and the embedding vectors of the relationship predicates of the node edges are concatenated as the input of the graph attention mechanism to calculate the attention values ​​between the nodes regarding the events to be inferred.

[0040] Specifically, the graph attention mechanism combines the attention mechanism with the graph structure, and uses node vector embedding to calculate the attention value between two nodes to indicate the importance of a node to another node. This method uses the graph attention mechanism to calculate the attention value between a node and its one-hop prior neighbor to indicate the influence of the one-hop prior neighbor on the occurrence of node events.

[0041] Step 108: The entity embedding vectors of the base point and its extended nodes are concatenated with their relation predicate embedding vectors respectively, and used as the input of the Hawkes process to calculate the conditional strength values ​​between the nodes regarding the event to be inferred.

[0042] Specifically, the Temporal Point Process (TPP) believes that events have evolutionary characteristics. What happens now may depend on what happened in the past, rather than what will happen in the future. It is a mathematical method for modeling in the time domain, and the Hawkes process is a typical temporal point process method used to statistically model the occurrence of events over time. The basic mathematical expression is .in Indicates that the event occurred at the time t The “intensity” or “rate” of an event at a certain point in time, given all past events that have occurred up to that point.t The expected value of events occurring within a small time interval nearby; is the "background rate" of the Hawkes process, that is, the rate at which events occur without any triggering or cascading effects, indicating the certain probability of an event occurring when it is not affected by other factors; It is a "triggering kernel", also known as a memory kernel. It describes how past events affect future events. It is a function that depends on the time when each past event occurred. It is used to measure the impact of each past event on any given time at present (the impact on the intensity of the current event).

[0043] Step 110: The attention value and the conditional strength value are summed as the likelihood value of the expanded node.

[0044] Step 112: Update node embedding according to the neighborhood information aggregated by the probability value, perform node expansion on the first-order expansion node, obtain the second-order expansion node, and enter the next round of node expansion. After multiple rounds of node expansion, select the node entity with the largest probability value about the base point as the entity of the event quadruple to be predicted, and complete the prediction of the event to be predicted.

[0045] Specifically, the predicted event is used as the starting point of the subgraph, the attention score between events and the intensity function of the event occurrence are combined as the probability value of the time occurrence, the iterative sampling may contain the edge of the predicted entity to expand the subgraph, and the probability value of the event occurrence is propagated along the expanded edge. After several rounds of expansion, the missing objects are predicted from the expanded nodes. Therefore, the reasoning subgraph can be regarded as a path explanation of entity prediction.

[0046] The model framework of the explainable event prediction method based on graph attention network and Hawkes process is shown in the figure Figure 3 shown.

[0047] The above-mentioned interpretable event prediction method based on graph attention network and Hawkes process adopts the graph attention mechanism and the Hawkes process in the event prediction time sequence point process, constructs the time prediction process as the process of reasoning subgraph expansion, takes the node to be predicted as the subgraph base point, combines the attention value and the conditional intensity function value obtained by the Hawkes process as the basis for subgraph search expansion and node embedding vector update, infers the context of event development, and uses this as a forward condition to reason about future events. This method not only pays attention to the relative time interval information between events, further improving the accuracy of the prediction results, but also can give a clear reasoning path, making the prediction of events interpretable, improving the prediction accuracy of urban safety events, and reducing the adverse effects of safety events.

[0048] In one embodiment, step 104 includes: using the Embedding module to initialize static entity and relationship predicate vectors for the entities and relationship predicates of the base point and the first-order extension node, respectively, to obtain static embeddings of the entities and relationship predicates; using a time encoding method based on the Bochner theorem to process the timestamps of the base point and the first-order extension node, integrating the dynamic information of the temporal knowledge graph into the changes of the entity, and obtaining an entity embedding vector with time information in the temporal knowledge graph; and using the static embedding of the relationship predicate as the relationship predicate embedding vector of the temporal knowledge graph.

[0049] Specifically, this method uses the Embedding module in pytorch to initialize static entity and relationship predicate vectors. In the time series knowledge graph, the graph is no longer static, but as time changes, the structure and content of the graph will also change. In order to represent the dynamic properties of the graph, time also needs to be embedded. For the embedding of time, a time encoding method based on Bochner's theorem is used. This time encoding method is usually used to process sequence data and has been widely used in graph networks, and has been proven to be an encoding method that can discover time patterns. The characteristics of an entity change over time, and we integrate the dynamic information of the time series knowledge graph into the changes of the entity. Use It represents the concatenation of static entity embedding and time embedding, which is used to represent entity embedding with time information in the temporal knowledge graph, namely node embedding. represents the embedding of a static entity, Indicates timestamp t The embedding of relational predicates is done using static relational predicate embedding. He said that time has little impact on it.

[0050] In one embodiment, step 106 includes: concatenating the entity embedding vector of the base point with the predicate embedding vector of the node edge, and then concatenating it with the entity and its relationship predicate embedding vector of the event to be inferred, to obtain the input of the graph attention mechanism; calculating the attention value between nodes about the event to be inferred as:

[0051] (1)

[0052] in, are two linear transformations whose parameters are shared by all nodes and are used to capture the dependencies between nodes and the events to be predicted; It is the first-order expansion node With the second stage node The relation predicate between the edges, is the predicate in the event to be predicted, Respectively represent the middle nodes in the nth reasoning step and The hidden representation of .

[0053] Specifically, in the graph attention layer, the embedding vector of the node is concatenated with the predicate embedding vector of the node edge. In order to pay more attention to the relevant information of the event to be reasoned during the expansion of the reasoning subgraph, it is further concatenated with the entity of the event to be reasoned and its predicate embedding vector to form the input of the graph attention mechanism. Formula (1) is used to calculate the attention value between nodes about the event to be reasoned.

[0054] When a node is at the first inference step, the hidden representation is the original embedding representation of the node. is the node edge in the inference subgraph About the events to be predicted The attention value indicates the event to be predicted Under the influence of The vector feature pair of nodes The importance of.

[0055] In one embodiment, step 108 includes: in the reasoning subgraph, between the base point and the first-order extended node, under the influence of the event to be predicted, the entity embedding vector of the base point and the entity embedding of the first-order extended node are respectively concatenated with their relation predicate vectors as inputs of the Hawkes process, and the inputs of the Hawkes process are:

[0056] (2)

[0057] The conditional strength value between the computing nodes regarding the event to be inferred is:

[0058] (3)

[0059] (4)

[0060] (5)

[0061] (6)

[0062] in, is the attention value of the prior edge between the node and its extended node, is the activation function, is the activation function used to decay the parameter, , , are the three parameters of the intensity function, is the history hidden vector, , and is the nonlinear transformation corresponding to the three parameters of the intensity function.

[0063] Specifically, in the reasoning subgraph, given an infinitesimal time window , the conditional intensity function is defined as a In the event of a Before, the event b The probability of occurrence. Qiang Zhang et al. used the embedding vector of the event node as input and constructed the Hawkes process to predict the event. Similarly, in our reasoning subgraph, the node With Node Events to be predicted between Under the influence of Embedding and its expansion nodes The embedding of is concatenated with its predicate vector and as the input of the Hawkes process to capture the influence of relative time interval on the occurrence of events. The Hawkes process input vector is defined as shown in formula (2).

[0064] Since the intensity function of the Hawkes process depends on the history, the following three nonlinear transformations are performed based on the historical hidden vector The three parameters of the intensity function are calculated using the formulas (4) to (6). Cannot be a negative number, so use softplus The function is used to attenuate the parameter, ensuring that it is non-negative. Finally, the intensity function is expressed as formula (3), where softplus The function ensures that the strength function value is positive.

[0065] In one embodiment, the step of reasoning about subgraph expansion specifically includes: taking the sum of the attention value and the conditional strength value of the prior edge formed by the node and its extended node as the likelihood value of the extended node:

[0066] (7)

[0067] in, is the probability value of the expanded node, is the attention value of the prior edge between the node and its extended node, It is the strength value of the prior edge condition formed by the node and its extended nodes.

[0068] The edges with higher probability values ​​among the prior neighbor edges are selected as the extended edges, the edges with lower probability are discarded, and the nodes on the edges are used as extended nodes.

[0069] Specifically, in the reasoning subgraph, in order to infer the entities that match the event to be predicted and learn as much hidden information as possible in the temporal knowledge graph, the reasoning subgraph needs to be expanded. We use the one-hop prior neighbor edge of the node to expand the node. However, if all the prior neighbors of the node are expanded as the nodes to be expanded, after a few reasoning steps, almost all of the temporal knowledge graph is traversed, resulting in too much computation. In order to reduce the amount of computation and improve the efficiency of prediction. The attention value of the prior edge formed by the node and its extended node is calculated. and conditional strength values The sum of is taken as the possibility value of the expanded node, and its expression is shown in formula (7).

[0070] The edges with higher probability values ​​among the prior neighbor edges are selected as the extended edges, and the edges with lower probability are discarded. The nodes on the edges are used as extended nodes. In this method, a fixed maximum number of extended edges will be selected, that is, each node can expand a fixed number of nodes at most in one reasoning step.

[0071] In one embodiment, step 112 includes: using a softmax function to normalize the likelihood values ​​of the extended nodes of the same node; the normalization result of the likelihood values ​​of the extended nodes of the node is:

[0072] (8)

[0073] in, is the normalized result of the likelihood value of the node’s expansion node, A set of relationship predicates representing the relationship between a node and its extended nodes, For Node The set of one-hop a priori neighbors of is the probability value of expanding the node.

[0074] All nodes With Node The sum of the likelihood values ​​of the connected edges is taken as the node The probability value of

[0075] (9)

[0076] in, Representative Node and its prior neighbor nodes The probability value between .

[0077] Compute Node The probability value of the entity that is the event to be predicted is:

[0078] (10)

[0079] According to the probability value between the node and its extended node, the extended node vector of the node is aggregated, and a linear layer is added to update the aggregated node vector; the update formula is:

[0080] (11)

[0081] (12)

[0082] in, Indicates that after the nth reasoning, the nodes are expanded and aggregated using the probability value as the ratio The vector obtained by the prior neighbor nodes, Represents the node after the nth reasoning expansion node The embedding vector of It is the ratio of the original node embedding vector and the embedding vector containing prior knowledge obtained by aggregating neighbor nodes in the updated node vector when updating the node.

[0083] In the new reasoning step n, after adding a new node, the node vector aggregation operation is performed on the newly added n layers to update the node vector of the previous level n-1 layer, and then the node vector of the n-1 layer is aggregated to update the node vector of the n-2 layer, and so on, until the base point of the reasoning subgraph is updated. .

[0084] Specifically, in order to enable event prediction to be performed based on prior knowledge. When the reasoning subgraph performs event reasoning, it is necessary to update the node embedding of the subgraph according to the expanded nodes. In the traditional graph convolutional neural network, in order to obtain more information about the graph structure, the nodes of the graph will aggregate the embedding vectors of their neighboring nodes to update their own embedding vectors. In the graph attention network, the nodes will be updated according to the attention values ​​of the neighboring nodes as the ratio of the aggregated neighboring node embedding vectors. In this embodiment, the sum of the attention value and the conditional strength value between the nodes is used as the ratio of the aggregated neighboring node embedding vector to update the node vector. On the one hand, it can notice the different importance of different prior events to the occurrence of the event, and on the other hand, it can capture the influence of the relative time interval of the prior neighbor events on the occurrence of the event. In order to make the obtained possibility value better represent the importance of different extended nodes to the upper-level node in the same reasoning step. The attention value of the extended node of the same node is normalized using the softmax function, and its normalization is shown in formula (8).

[0085] For nodes , which may be connected to the prior neighbor nodes by more than one edge, that is, for the node , there are multiple , so that it is connected to the node To this end, all With Node The sum of the likelihood values ​​of the connected edges is taken as the node The probability value is calculated as shown in formula (9).

[0086] Formula (10) is used to calculate the node is the probability value of the entity of the event to be predicted.

[0087] After obtaining the likelihood value between a node and its extended node, the extended node vector of the node is aggregated, and a linear layer is added to update the aggregated node vector. The update formulas are shown in Formula (11) and Formula (12).

[0088] In the new reasoning step n, after adding a new node, the node vector aggregation operation is performed on the newly added n layers to update the node vector of the previous level n-1 layer, and then the node vector of the n-1 layer is aggregated to update the node vector of the n-2 layer, and so on, until the base point of the reasoning subgraph is updated. .

[0089] In one embodiment, a binary cross entropy function is used as a loss function; the loss function is:

[0090] (13)

[0091] in, represents the number of sets of all training quads, The set of entities in the inference subgraph representing the event to be predicted, is the label that predicts whether entity e is the correct answer to the quadruple to be predicted, Represents the likelihood score at the last reasoning step.

[0092] Specifically, the inference subgraph prediction problem is essentially a multi-classification problem applied to graph neural networks. Like most multi-classification problems, we use the binary cross entropy function as our loss function. When the predicted probability value is close to the actual label, the loss is small; when the predicted probability value is far from the actual label, the loss is large. This helps to better train our event prediction model. The loss function is shown in formula (13).

[0093] It should be understood that although Figure 1 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 1At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0094] In a verification embodiment, this embodiment uses an experimental dataset commonly used in time series knowledge graph prediction. Specifically, three datasets constructed based on the Integrated Crisis Early Warning System (ICEWS), ICEWS14, ICEWS18 and ICEWS05-15, and the YOGO dataset constructed based on YAGO and the Wikipedia knowledge base WIKI dataset are selected. ICEWS14 focuses on data from 2014. It is often used as a benchmark for link prediction tasks. It contains 6869 entities and 230 entity relationships. ICEWS18 contains event data of event data between January 1 and October 31, 2018 in ICEWS, including approximately 23033 entities and 256 entity relationships. ICEWS05-15 contains international events and event data from 2005 to 2015 in ICEWS. It contains approximately 10094 entities, 251 entity relationships, and 4017 time points such as event type, participants, time, and location. YAGO (Yet Another Great Ontology) is a large-scale semantic knowledge base. It aims to provide a rich and accurate knowledge base for artificial intelligence and natural language processing. It integrates multiple data sources such as Wikipedia, WordNet, WikiData, and GeoNames, and contains more than 17 million entities and about 150 million factual information. The YAGO dataset derived from the YAGO knowledge base contains 10,623 entities and 10 entity relationships. The WIKI dataset is a project of the Wikimedia Foundation, which aims to build an open and editable knowledge graph and is widely used in intelligent assistants, semantic search, question-and-answer systems, and other fields. The dataset of partial knowledge construction used in this embodiment contains 12,554 entities and 24 entity relationships.

[0095] (1) Baseline Model

[0096] During the experiment, this method is compared with the traditional static knowledge graph model and the typical temporal knowledge graph model on the data set selected in this embodiment. The static knowledge graph model uses DistMult, ComplEx, R-GCN, ConvE, ConvTransE, and RotatE. HyTE, TTransE, TA-DistMult, RGCRN, CyGNet, RE-Net, RE-GCN, and CENET as the baseline temporal knowledge graph.

[0097] (2) Experimental settings and evaluation indicators

[0098] The experiment was conducted on the Ubuntu platform with the system version number of 18.04.06. The experimental development framework used the pytorch framework and the GPU was TITANRTX 24G. Adam was used as the optimizer. The learning rate was set to 0.0002. Each data set was divided in chronological order and divided into training set, validation set, and test set at a ratio of 80%, 10%, and 10%. The evaluation indicators used two categories of indicators: MRR (Mean Reciprocal Rank) and Hits@K. Hits@K means that in the prediction task, the predicted result is the average proportion of quadruplets whose ranking of the correct result is less than the top K. The three indicators used are Hits @1, Hits@3, and Hits@10; the calculation formula is as follows:

[0099] ;

[0100] in, S is a set of event quads, | S | is the number of quaternions, and ranki is the i Link prediction ranking of quadruple. For example, for the test set S The predicted results of the entity are scored and sorted from high to low. If the correct result is in the top K items, the ranking is increased by 1.

[0101] The full name of MRR is Mean Reciprocal Ranking. The calculation formula of MRR is as follows:

[0102] ;

[0103] in, It is the reciprocal of the correct answer's ranking in the prediction results. The position of the correct tail entity in the prediction ranking is the rank obtained by the model. MRR is a measure of the overall performance of the system, especially when evaluating multiple queries. It calculates the average of the reciprocal ranking of the first correct answer obtained for all queries. The value range is [0, 1]. The higher the value, the better the performance of the model.

[0104] (3) Experimental results

[0105] The results of the entity prediction task are shown in Tables 1 and 2. In general, the overall performance of the model on the ICEWS14, ICEWS18, and ICEWS05-15 datasets is better than other baseline models except the CENET model, which shows the effectiveness of the model proposed in this paper. Specifically, because the model takes into account the timeliness of knowledge, the performance is significantly better than the baseline model of static knowledge graph prediction. Compared with the typical temporal knowledge graph model, the prediction accuracy is also significantly improved. In addition, the reasoning path of the prediction result can be directly found by backtracking. Therefore, the method of this application is also interpretable. On the WIKI dataset, the model proposed in this paper improves the MRR indicator by 3.11%, and improves the Hits@3 and Hits@10 indicators by 7.08% and 8% respectively. On the YAGO dataset, the MRR indicator is improved by 2.42%, and the Hits@3 and Hits@10 indicators are improved by 4.79% and 4.8% respectively. After analyzing the WIKI and YAGO datasets, we found that the reason for their high prediction accuracy is that the WIKI and YAGO datasets have more entities, but the predicate relationship is relatively simple, and their time granularity spans 1 year, while the time granularity span of the ICEWS dataset is 24 hours. This also shows that as the amount of data increases, the longer the time span of event records, the more features the method in this application can learn than other typical models, which can further improve the accuracy of prediction.

[0106] Table 1 Results of entity prediction task (I)

[0107]

[0108] Table 2 Results of entity prediction task (II)

[0109]

[0110] In order to more intuitively demonstrate the comparative effect of the present application method with other models, the overall performance index MRR of the model is selected to draw the experimental results and draw a bar chart. Figure 4 As shown, Figure 4Among them, comparison method 1 is DistMult, comparison method 2 is ComplEx, comparison method 3 is R-GCN, comparison method 4 is ConvE, comparison method 5 is ConvTransE, comparison method 6 is RotatE, comparison method 7 is HyTE, comparison method 8 is TTransE, comparison method 9 is TA-DistMult, comparison method 10 is RGCRN, comparison method 11 is CyGNet, comparison method 12 is RE-Net, comparison method 13 is RE-GCN, and comparison method 14 is CENET; dataset 1 is ICEWS14-filtered, dataset 2 is ICEWS05-15-fltered, dataset 3 is ICEWS18-fltered, dataset 4 is WIKI, and dataset 5 is YAGO.

[0111] (3) Ablation experiment

[0112] In order to show the contribution of each module to the model, ablation experiments were performed on four datasets. Variants of the model adopted by the method of this application were created by shielding relevant model components, and their performance on the four datasets was compared.

[0113] 1) The impact of graph attention networks

[0114] The model used in the present invention is recorded as: AHP model. AHP-H-only is the model when the AHP model only uses the Hawkes process. The graph attention network module is blocked, and only the time point characteristics of the event development are focused on. The experimental results are shown in Tables 3 and 4.

[0115] Table 3 Experimental results of the influence of attention network (I)

[0116]

[0117] Table 4 Experimental results of the influence of attention network (II)

[0118]

[0119] In order to show the experimental results more intuitively, a histogram is used for comparison, such as Figure 5 Show, among which, Figure 5 Dataset 1 is ICEWS14-filtered, Dataset 2 is ICEWS05-15-filtered, Dataset 3 is ICEWS18-filtered, Dataset 4 is WIKI, and Dataset 5 is YAGO. Figure 5It can be clearly seen that if only the Hawkes process is used to infer the inference subgraph, compared with AHP, it is slightly worse in all performance indicators. The gap ranges from 1% to 5%. However, the prediction accuracy has not dropped significantly. This shows that the impact of the occurrence of some prior events on the occurrence of other events always exists and will not dissipate with the length of the time interval.

[0120] 2) Hawkes process influence

[0121] AHP-A-only is the model when the AHP model only uses the graph attention network, shielding the Hawkes process network module. Similar to the XERT model, it only focuses on the degree of mutual importance between events. The experimental results are shown in Tables 5 and 6.

[0122] Table 5 Experimental results of the influence of Hawkes process (I)

[0123]

[0124] Table 6 Experimental results of the influence of Hawkes process (II)

[0125]

[0126] In order to show the experimental results more intuitively, a histogram is used for comparison, such as Figure 6 Show, among which, Figure 6 Dataset 1 is ICEWS14-filtered, Dataset 2 is ICEWS05-15-filtered, Dataset 3 is ICEWS18-filtered, Dataset 4 is WIKI, and Dataset 5 is YAGO. Figure 6 It can be clearly seen that in the case of only attention network, although the model performance is very close to the AHP model, there is still a gap of about 1% from AHP in overall performance. Its performance on the ICEWS dataset is far better than the AHP-H-only model, but on datasets such as WIKI and YAGO with long data duration and large time granularity, the effect is not much different from AHP-H-only. Even on datasets such as YAGO with larger data volume and longer time span, AHP-H-only performs better than AHP and AHP-A-only. This shows that the relative time interval between prior events is also an information dimension worthy of attention for predicting events, and has an important impact on the accuracy of event prediction. Generally speaking, for datasets with shorter duration and smaller time granularity, the degree of mutual attention between events is more important than the degree of time accumulation incentive between events. For datasets with longer duration, more data and larger time granularity, the degree of time accumulation incentive influence between events may catch up with or exceed the degree of attention between events as the data becomes larger and the time granularity is refined.

[0127] This method uses the attention mechanism to capture the degree of attention between events, uses the Hawkes process to learn the influence of the relative time interval between events on the occurrence of events, and relies on the cross entropy function to train and learn the parameters in the attention mechanism and the parameters in the Hawkes process as well as the vector embedding of entities and relations. Then, the entity of the event is predicted. Compared with advanced models on multiple data sets, the overall prediction accuracy has been improved to varying degrees.

[0128] The technical features of the above embodiments may be arbitrarily combined. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0129] The above-described embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An explainable event prediction method based on graph attention network and Hawkes process, characterized in that: The method comprises: Obtaining an original time series knowledge graph and a quadruple of events to be predicted; the quadruple data of the original time series knowledge graph is text data; the original time series knowledge graph includes several event quadruple, and the event quadruple includes: a head entity, a relationship predicate, a tail entity, and a timestamp; the original time series knowledge graph is a time series knowledge graph constructed based on a series of events in the security warning process; The event quadruple to be predicted is used as a base point of the reasoning subgraph, the one-hop priori neighbor node of the base point is used as a first-order extension node of the reasoning subgraph, and the relationship predicate is used as an attribute of the edge between two nodes; Embed the entity and relationship predicates of the base point and its first-order extended nodes; The embedding vector of the base point and its first-order extended node and the embedding vector of the relationship predicate of the node edge are concatenated as the input of the graph attention mechanism to calculate the attention value between the nodes about the event to be inferred; The entity embedding vectors of the base point and its extended nodes are concatenated with their relation predicate embedding vectors respectively, which are used as the input of the Hawkes process to calculate the conditional strength values ​​between the nodes regarding the event to be inferred; The attention value and the condition strength value are summed as the possibility value of the expanded node; The node embedding is updated according to the neighborhood information aggregated according to the possibility value, and the first-order expansion node is expanded to obtain the second-order expansion node, and then enter the next round of node expansion. After multiple rounds of node expansion, the node entity with the largest possibility value about the base point is selected as the entity of the quadruple of the event to be predicted, and the prediction of the event to be predicted is completed.

2. The interpretable event prediction method based on graph attention network and Hawkes process according to claim 1, characterized in that: Embed the entities and relationship predicates of the base point and its first-order extended nodes, including: Initializing static entity and relationship predicate vectors of the entities and relationship predicates of the base point and the first-order extended node respectively using an Embedding module to obtain static embeddings of the entities and relationship predicates; The timestamps of the base point and the first-order extended node are processed using a time encoding method based on the Bochner theorem, the dynamic information of the time series knowledge graph is integrated into the change of the entity, and an entity embedding vector with time information in the time series knowledge graph is obtained; The static embedding of relation predicates is used as the relation predicate embedding vector of the temporal knowledge graph.

3. The interpretable event prediction method based on graph attention network and Hawkes process according to claim 2, characterized in that: The embedding vector of the base point and its first-order extended node and the embedding vector of the relationship predicate of the node edge are concatenated as the input of the graph attention mechanism, and the attention value between the nodes about the event to be inferred is calculated, including: The entity embedding vector of the base point is concatenated with the predicate embedding vector of the node edge, and then concatenated with the entity of the event to be inferred and its relationship predicate embedding vector to obtain the input of the graph attention mechanism; The attention value between the calculation nodes about the event to be inferred is: in, are two linear transformations whose parameters are shared by all nodes and are used to capture the dependencies between nodes and the events to be predicted; It is the first-order expansion node With the second stage node The relation predicate between the edges, is the predicate in the event to be predicted, Respectively represent the middle nodes in the nth reasoning step and The hidden representation of .

4. The interpretable event prediction method based on graph attention network and Hawkes process according to claim 2, characterized in that: The entity embedding vectors of the base point and its extended nodes are concatenated with their relation predicate embedding vectors respectively, which are used as the input of the Hawkes process to calculate the conditional strength values ​​between the nodes regarding the event to be inferred, including: In the reasoning subgraph, under the influence of the event to be predicted, the entity embedding vector of the base point and the entity embedding of the first-order extended node are concatenated with their relation predicate vectors respectively as the input of the Hawkes process. The input of the Hawkes process is: The conditional strength value between the computing nodes regarding the event to be inferred is: in, is the attention value of the prior edge between the node and its extended node, is the activation function, is the activation function used to decay the parameter, , , are the three parameters of the intensity function, is the history hidden vector, , and is the nonlinear transformation corresponding to the three parameters of the intensity function.

5. The interpretable event prediction method based on graph attention network and Hawkes process according to claim 1, characterized in that: The steps of reasoning subgraph expansion specifically include: The sum of the attention value and conditional strength value of the prior edge between the node and its extended node is used as the likelihood value of the extended node: in, is the probability value of the expanded node, is the attention value of the prior edge between the node and its extended node, is the strength value of the prior edge condition formed by the node and its extended nodes; The edges with higher probability values ​​among the prior neighbor edges are selected as the extended edges, the edges with lower probability are discarded, and the nodes on the edges are used as extended nodes.

6. The interpretable event prediction method based on graph attention network and Hawkes process according to claim 1, characterized in that: Aggregating neighborhood information according to the likelihood value and updating node embeddings include: The softmax function is used to normalize the likelihood values ​​of the extended nodes of the same node; in, is the normalized result of the likelihood value of the node’s expansion node, A set of relationship predicates representing the relationship between a node and its extended nodes, For Node The set of one-hop a priori neighbors of is the possibility value of the expanded node; All nodes With Node The sum of the likelihood values ​​of the connected edges is taken as the node The probability value of in, Representative Node and its prior neighbor nodes The probability value between Compute Node The probability value of the entity that is the event to be predicted is: According to the probability value between the node and its extended node, the extended node vector of the node is aggregated, and a linear layer is added to update the aggregated node vector; in, Indicates that after the nth reasoning, the nodes are expanded and aggregated using the probability value as the ratio The vector obtained by the prior neighbor nodes, Represents the node after the nth reasoning expansion node The embedding vector of is the ratio of the original node embedding vector and the embedding vector containing prior knowledge obtained by aggregating neighboring nodes in the updated node vector when updating the node; In the new reasoning step n, after adding a new node, the node vector aggregation operation is performed on the newly added n layers to update the node vector of the previous level n-1 layer, and then the node vector of the n-1 layer is aggregated to update the node vector of the n-2 layer, and so on, until the base point of the reasoning subgraph is updated.

7. The interpretable event prediction method based on graph attention network and Hawkes process according to claim 1, characterized in that: The binary cross entropy function is used as the loss function; the loss function is: in, represents the number of sets of all training quads, The set of entities in the inference subgraph representing the event to be predicted, is the label that predicts whether entity e is the correct answer to the quadruple to be predicted, Represents the likelihood score at the last reasoning step.

Images