Time knowledge graph question and answer method based on multi-granularity implicit tense perception

Abstract

The invention provides a time knowledge graph question answering method based on multi-granularity implicit tense perception, which comprises the following steps of: S1, acquiring a question text, and identifying entity information and time information in the question text; s2, the extracted information is retrieved in a time knowledge graph, and head entity embedding es, tail entity embedding eo and time embedding t tau are obtained; s3, inputting the entity information into a problem presentation layer to obtain a relation embedding vector r; S4, inputting the time information and the time embedding t [tau] into a time sequence fusion network to obtain a new time embedding t; and S5, inputting the relation embedding vector r and the new time embedding t into an integrated learning layer to obtain a score, and taking a result with the highest score as an answer. According to the method, the representation and prediction capabilities of time sequence data are improved, the graph attention network of multiple attention mechanisms is integrated, tense information implied in a problem is effectively captured, and the capability of capturing complex time dependence is enhanced through the synergistic effect of the convolutional neural network and the gating circulation unit.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and in particular to a temporal knowledge graph question-answering method based on multi-granularity implicit temporal perception. Background Art

[0002] In practical applications, factual knowledge often evolves over time, such as the change of host countries for the World Cup. This dynamic change of knowledge over time places higher demands on the management of temporal knowledge, so one of the current research trends is to use temporal knowledge graphs (TKGs) to capture and represent the evolution of information in the temporal dimension. Temporal Knowledge Graph Question Answering (TKGQA) is an important branch of the Knowledge Graph Question Answering (KGQA) task, which aims to extract entities or timestamps from the temporal knowledge graph to answer questions involving temporal reasoning.

[0003] The mainstream methods for TKGQA tasks can be divided into two categories: rule-based methods and embedding-based methods. Rule-based methods, such as decomposing questions into non-temporal sub-questions and temporal sub-questions through preset templates, or learning temporal logic rules directly from temporal knowledge bases through logical reasoning, rely on manually formulated rules or knowledge base-based extraction, and have weak generalization capabilities for complex problems. Embedding-based methods, on the other hand, match questions and answers by encoding questions and knowledge base subgraphs into a common embedding space. For example, CRONKGQA is the first model designed specifically for temporal knowledge graph tasks. It directly combines a pre-trained language model with TKG to generate answers, but performs poorly in complex temporal reasoning tasks. The TempoQR model proposes a reasoning framework for solving time constraint problems, which improves reasoning performance by re-encoding questions. However, its encoding of time only embeds static timestamps, ignoring the dynamic dependencies between time. In general, current methods focus on the processing of single-granularity temporal problems, ignoring the complexity of multi-granularity temporal information in practical applications, resulting in the existing methods not being able to perform well on problems with multi-level temporal dependencies and complex temporal relationships. In addition, existing methods do not pay enough attention to complex implicit multi-granularity temporal information.

[0004] One of the key points of the TKGQA task is to assist reasoning through the temporal information in the question, so it is necessary to integrate the temporal features with the representation of the question entities. For example, when dealing with the question "Who made the final 3-pointer before the end of the 2005 NBA Finals?", existing methods such as TempoOR (see Mavromatis C, Subramanyam PL, Ioannidis VN, et al. Tempoqr: temporal question reasoning overknowledge graphs [C] / / Proceedings of the AAAI conference on artificial intelligence. 2022, 36 (5): 5825-5833.) mainly rely on the question embedding representation of the pre-trained language model, directly taking "before the end of" in the question as the query, ignoring the time constraint of "2005", which causes the reasoning to be overly dependent on the entity information in the question, which may introduce temporal noise, thereby affecting the accuracy of reasoning. Although the GRONKGQA method (see Saxena A, Chakrabarti S, Talukdar P. Question answering over temporal knowledge graphs[J]. arXiv preprint arXiv:2106.01515, 2021.) can pay attention to the time information "2005", it has certain limitations in processing implicit time information and cannot fully capture the implicit constraints of time.

[0005] In the field of TKGQA, there are usually two types of solutions for temporal knowledge graph question answering tasks. One solution is to decompose the question into several non-temporal questions and temporal constraints, then use the question answering model designed for traditional knowledge graphs to answer the questions, and then select the answer through the temporal constraints, such as TEQUILA. However, this method relies on hand-crafted rules and is difficult to cope with the diversity of complex questions. The second method is to calculate the semantic similarity of the answers by obtaining the embedding of the temporal knowledge graph. For example, CronKGQA introduces a learnable reasoning process that performs well on simple questions, but cannot handle implicit temporal information well, and the effect on complex temporal reasoning is not ideal. The TempoQR model enhances the semantic representation of questions by introducing the time range information of each question and combining it with the EaE method. However, the TempoQR method relies on the question embedding representation of the pre-trained language model, so its reasoning is overly dependent on the entity information in the question, and is still insufficient when dealing with implicit temporal questions such as "Before", "After", "First" and "Last". The subsequent MultiQA proposed the concept of multi-granularity temporality and designed a multi-granularity temporal aggregation module MultiQA to enrich the temporal attributes. However, the reasoning ability of MultiQA mainly relies on the complementary reasoning ability obtained during the TKG embedding pre-training process, so it still has limitations, especially when dealing with complex constraints such as "First" and "Before".

[0006] In the field of TKGQA, positional encoding is an important method to capture the order and position features in time series information. Traditional positional encoding methods are mainly divided into two categories: fixed positional encoding and learnable positional encoding. Fixed positional encoding was widely used in early research, such as the positional encoding generated by sine and cosine functions used in the Transformer model. This method generates positional encodings through predefined functions. Although it is computationally efficient, it lacks adaptability to data features and has limited performance when processing complex and variable time series data. Subsequently, researchers proposed learnable positional encoding, which allows the model to learn the best encoding method during training to adapt to different data features. However, the effectiveness of this method is still limited when faced with multi-granularity temporal data. Summary of the invention

[0007] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a temporal knowledge graph question-answering method based on multi-granularity implicit temporal perception.

[0008] In order to achieve the above-mentioned object of the present invention, the present invention provides a time knowledge graph question answering method based on multi-granularity implicit temporal awareness, comprising the following steps:

[0009] S1, obtain the question text and identify the entity information and time information in the question text; wherein the entity information is the head entity or the tail entity in the time knowledge graph quadruple, and the relationship is the head entity and / or the tail entity.

[0010] S2, using an embedding-based representation learning method, retrieves the extracted information in the temporal knowledge graph (TKG) to obtain the head entity embedding e s , tail entity embedding o , and the time embedding t τ ;

[0011] The temporal knowledge graph is defined as a directed graph with timestamps G = (E, R, T), where E is an entity set, including |E| different entities, R is a relationship set, including |R| different relationships; T is a timestamp information set, including |T| different timestamps. The temporal knowledge graph can also be regarded as a set of static knowledge graphs at each timestamp, that is, G = {G1, G2, ..., G T}. The knowledge of each timestamp τ can be represented as a four-tuple (s, r, o, τ), where: s, o∈E represent the main entity and the guest entity respectively; r∈R represents the relationship, and τ∈T represents the timestamp. For the convenience of representation, the main entity s, the relationship r, the guest entity o and the timestamp τ can be represented by embeddings respectively. Another modeling form can also be used here, that is, (h, r, t, τ) is used to represent the four-tuple, where h is the head entity and t is the tail entity.

[0012] In the process of building a temporal knowledge graph (TKG), the first task is to extract entities, relationships, and time information from a variety of temporal data. These data sources include structured, unstructured, and semi-structured data. After the extraction of fact triples, knowledge fusion is required.

[0013] Knowledge fusion aims to integrate temporal knowledge graphs from different data sources or time points to form an accurate and consistent dataset. This process not only focuses on the fusion of entities, relationships, and attributes, but also ensures the consistency of these elements in the time dimension.

[0014] The construction of TKG is not only a simple integration and representation of time series data, but more importantly, it is to conduct in-depth analysis and reasoning on the information in the graph through knowledge computing, thereby mining potential information and improving the completeness and accuracy of knowledge. The core of knowledge computing lies in knowledge reasoning, which derives new knowledge from existing facts and continuously improves and enriches the content of the graph.

[0015] S3, input the entity information into the question representation layer to obtain the relation embedding vector r:

[0016] S4, embed the time information and its time into tτ Input the time series fusion network to obtain a new time embedding t;

[0017] S5, input the relationship embedding vector r and the new time embedding t into the integrated learning layer to obtain the scoring score, and take the result with the highest score as the answer.

[0018] Preferably, step S3 comprises:

[0019] The pre-trained RoBERTa model is used to encode the question text and generate the initial semantic matrix Q of the question:

[0020] Q = W R RoBERTa(q text ) (1)

[0021] Where Q = [q CLS ,q1,q2,...,q n ];

[0022] q CLS is a global representation of the question text;

[0023] q1,q2,q n is the contextual representation of the 1st, 2nd, and nth words in the question;

[0024] S is the number of markers;

[0025] W R is the projection matrix of the question text;

[0026] q text is the question text;

[0027] Get the final problem representation q, q = q CLS ;

[0028] Then the obtained global problem representation q is passed through the fully connected layer to obtain the relation embedding vector r:

[0029] r = W r q + b r (2)

[0030] Where W r is a learnable weight matrix;

[0031] q is the global problem representation;

[0032] b r The dimension is D r Learnable bias vector.

[0033] Preferably, S4 comprises the following steps:

[0034] S4-1, time information and its time embedding t τ Construct the time semantic matrix T d , used to represent the monthly granularity time; then the question text is encoded using the encoder to obtain the encoded position embedding Y; and then the Transformer encoder is used to obtain the time series representation T of the question m :

[0035] T m = Transformer(T d ′) (5)

[0036] Among them, T d ′=T d +Y;

[0037] t CLS represents the global representation of the time series, represents the embedding representation of the 1st, 2nd, ..., nth day in the time series;

[0038] At this time, the time embedding t τ =t CLS ;

[0039] S4-2, embed the obtained time into t τ Input the multi-head attention mechanism, the results of multiple attention heads are concatenated into a single attention head to get the final attention output;

[0040] S4-3, the final attention output is input to the graph attention network (GAT), at this time, t τ After linear transformation, we get the eigenvector h i :

[0041] h i = t τ W i (8)

[0042] Where W i is a weight matrix, which maps the time embedding to the feature space of the node.

[0043] The graph attention network calculates the attention coefficient a between node i and its neighbor node j ij , through the LeakyReLU activation function and node feature h i With h j Splicing, normalize the attention scores between the obtained nodes through the softmax function:

[0044]

[0045] Among them, a ijis the attention coefficient between node i and its neighbor node j;

[0046] exp() is the exponential function of the natural constant e;

[0047] for a i The transpose of

[0048] LeakyReLU() is the activation function;

[0049] k represents the neighbor node of node i;

[0050] a i is the learnable weight vector of attention;

[0051] N(i) represents the set of neighbor nodes of node i;

[0052] || represents the concatenation operation of vectors;

[0053] h i ′ represents the updated feature of node i;

[0054] S4-4, use a one-dimensional convolutional layer to process the updated node features and generate a new time embedding t:

[0055]

[0056] Where Conv represents the convolution operation;

[0057] σ represents the activation function;

[0058] C in Indicates the number of channels of input data;

[0059] h′ c is the cth channel in the input feature map;

[0060] W c is the corresponding convolution kernel weight;

[0061] b is the bias term;

[0062] * is the convolution operator.

[0063] Preferably, the encoder is a self-learning encoder, and the encoded position embedding Y is obtained by encoding with the self-learning encoder, including:

[0064] Y = X b,l,d + PE L,D (4)

[0065] Where X b,l,d Representing the relevant embeddings of the question;

[0066] PE is a self-learning parameter matrix, which represents the position encoding. When the model is initialized, the position encoding vector is initialized to a random value PE, and then the model is gradually trained to learn the best encoding representation for each position. L,D It means extracting the encoding of the first L positions from the position encoding PE tensor with an embedding dimension of D, and expanding it in the first dimension to make its size compatible with the input tensor X, so that it can be added element-by-element with the input embedding tensor.

[0067] Preferably, S5 comprises the following steps:

[0068] S5-1, input the relation embedding vector r and the new time embedding t into the gated recurrent unit (GRU) respectively to obtain the new relation representation R and the updated time embedding t′;

[0069] S5-2, and then use the scoring formula to get the score of the candidate answer:

[0070]

[0071] Among them, Score represents the score of the candidate answer;

[0072] max means taking the maximum value;

[0073] φ( ) represents the TComplEx scoring function;

[0074] e ∈ is the candidate answer entity, which is all entities in the temporal knowledge graph;

[0075] e s 、e o Head entity embedding and tail entity embedding respectively;

[0076] R represents the relational representation of the final entity semantic unit;

[0077] t′ represents the final time representation;

[0078] t τ represents the time representation at τ.

[0079] Preferably, it includes: using a named entity recognition tool to identify entity information in the question text, and using a time extraction tool to obtain time information.

[0080] Preferably, the loss function of the ensemble learning layer is cross entropy loss.

[0081] In summary, due to the adoption of the above technical solutions, the present invention improves the representation and prediction capabilities of time series data: the graph attention network (GAT) that integrates a multi-head attention mechanism effectively captures the temporal information implicit in the problem, and through the synergy of the convolutional neural network (CNN) and the gated recurrent unit (GRU), the ability to capture complex time dependencies is enhanced. At the same time, a self-learning encoding strategy is designed that can dynamically adapt to data of different time granularities. When processing time tensors, an integrated learning strategy is adopted to retain multiple time segments as different "weak learners" and select the optimal time representation through decision-making. The specific advantages are as follows:

[0082] (1) It can dynamically generate position encodings based on the input time granularity and data features, efficiently capture the multi-granularity temporal information implicit in the problem, and use ensemble learning methods to make the reasoning results robust. It is particularly suitable for reasoning about complex temporal problems containing implicit temporal information.

[0083] (2) We design an implicit temporal information capturer MGC, which introduces a multi-head attention mechanism and a graph attention network (GAT) to extract temporal dependencies at different levels, and introduces a CNN to extract local dependency structures in time series, so that it can accurately parse key patterns in complex temporal relationships.

[0084] (3) An ensemble learning strategy is proposed. A GRU-based implicit temporal information integration unit is constructed to integrate the implicit temporal information in the problem and the relational information of the fused entity semantics. An ensemble learning strategy that retains multiple time segments as "weak learners" is proposed, which improves the adaptability and accuracy of MGIT in handling complex temporal problems and enables it to reason robustly.

[0085] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0086] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0087] Figure 1 It is a schematic diagram of the framework structure of MGIT of the present invention.

[0088] Figure 2 This is a schematic diagram of the attention head number analysis experiment results.

[0089] Figure 3 It is a schematic diagram of the Dropout numerical analysis experimental results.

[0090] Figure 4 Figure 2 is a schematic diagram of the model performance of MGIT and other baselines for different problem types. DETAILED DESCRIPTION

[0091] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0092] The present invention proposes a temporal fusion network that can perceive multi-granularity time information, and makes implicit time visible. For example, "before the end of the 2005" in the background technology example sentence is displayed as 2005-01 to 2005-12. The time feature is then fused and encoded with the problem entity representation, thereby effectively capturing the entity information contained in the implicit time period and improving the accuracy and robustness of reasoning.

[0093] Introducing a time series fusion network may expand the solution space and allow the model to discover answers contained in the implicit time, but it will also introduce noise. To this end, a decision-making method based on ensemble learning is also proposed. Multiple time segments are retained as different weak learners for decision-making, thereby improving the stability of the model's reasoning performance. For example, for the question "Who was the United States Energy Secretary after Hazel R.O'Leary?", the correct answer is Bill Richardson, but after adding the time series fusion network, the prediction result becomes Spencer Abraham. Through ensemble learning, we retain Bill Richardson and Spencer Abraham as candidate answers, and finally select Bill Richardson with the highest score as the answer.

[0094] In view of the shortcomings of the two types of solutions usually used in the Temporal Knowledge Graph Question Answering (TKGQA) task, this paper proposes a method based on multi-granularity time perception. To better capture implicit temporal features, we propose a temporal fusion network, use Transformer to extract relational representations in complex questions, use a temporal fusion network to enhance and fuse temporal features and relational representations, and propose an integrated learning method to eliminate the noise introduced by fusing multi-granularity time.

[0095] In view of the shortcomings of positional encoding of temporal knowledge graphs, this paper proposes a self-learning positional encoding strategy to improve the performance of positional encoding. Unlike traditional methods, self-learning positional encoding can not only be adjusted during the training process, but also dynamically generate the optimal positional encoding according to the input time granularity and data features.

[0096] Therefore, a TKGQA reasoning framework MGIT based on multi-granular implicit temporal awareness is proposed to improve the reasoning ability of complex TKGQA tasks by enhancing temporal encoding. Figure 1 As shown in the figure, it consists of an information retrieval layer, a question representation layer, a TFN layer, and an integrated learning layer. The information retrieval layer includes temporal information retrieval and entity retrieval. The validity of the retrieval is ensured by matching the time and entity in the question. The TFN layer introduces a self-learning encoding strategy to adapt to data of different time granularities. It combines the multi-head attention mechanism and GAT to capture the complex dependencies between temporal information and entities, and enhances its ability to handle complex problems through CNN. The integrated learning layer consists of a fusion layer and an answer scoring module. It uses GRU to process relational features and temporal features, and finally uses a decision ensemble learning strategy to select the optimal entity and its score.

[0097] 1. Problem Representation Layer

[0098] In order to accurately obtain the entity and time information in the question, we used an efficient entity and time information extraction process for the functional problem that the ICEWS dataset does not have built-in entity linking. First, the entity names in the question are identified using the named entity recognition (NER) tool, and then the time information is obtained through the time extraction tool. In this step, we eliminated the entity names that are not related to the knowledge graph to ensure that the extracted entity and time information are highly accurate. Subsequently, we retrieved the extracted information with the entities and times in the temporal knowledge graph through an embedding-based representation learning method. Next, we used the pre-trained RoBERTa model to encode the question text and generate the initial semantic matrix Q of the question, which contains the semantic representation of the words in the question.

[0099] Q = W R RoBERTa(q text ) (1)

[0100] Where Q = [q CLS ,q1,q2,...,q n ] is a D×S embedding matrix;

[0101] q CLS is a global representation of the question text;

[0102] q n is the contextual representation of the nth word in the question;

[0103] D is the dimension of Temporal KG embedding;

[0104] S is the number of markers;

[0105] W Ris the projection matrix of the question text, with dimension D×D roberta ;

[0106] q text is the question text;

[0107] D roberta is the embedding dimension, and the final problem is represented by q=q CLS .

[0108] We obtain the global problem representation q through a fully connected layer with a learnable weight matrix W r and the learnable bias term b r Combined, generate the relation embedding vector r. The formula is as follows:

[0109] r = W r q + b r (2)

[0110] Where W r Yes D r ×D learnable weight matrix;

[0111] q is the global problem representation;

[0112] b r The dimension is D r Learnable bias vector.

[0113] 2 Time Series Fusion Network

[0114] 2.1 Time Aggregation

[0115] Traditional temporal knowledge graph (Temporal KG) embedding training usually focuses on semantic information of a single granularity (such as daily granularity), which limits the effectiveness of the model in processing multi-granularity time, such as day, month, year, etc. To this end, we innovatively proposed a temporal fusion network (TFN). TFN can extract and integrate key information from different time granularities, improving the accuracy and robustness of the model when dealing with multi-granularity time reasoning problems.

[0116] Specifically, for multi-granularity problems, Figure 1 Taking the monthly granularity time m in the question as an example, we first extract all the included daily timestamps d1, d2, d3...d n and its temporal embedding t τ , Where n is the number of relevant days. The time semantic matrix T is constructed from this d To express monthly granularity time:

[0117]

[0118] Where T d ∈R N×D is a matrix containing the embeddings of the entire day of the mth month;

[0119] It is the daily timestamp dn obtained by embedding it into the temporal knowledge graph (TKG).

[0120] Time is sequential, similar to the position of words in a text. In order to capture the sequential relationship of temporal information, we propose a self-learning encoding method that can d The representation of time features is adjusted at different positions in the model to ensure the adaptability of the model to different time granularities, making the relevant positions more accurate. The calculation formula for self-learning encoding is as follows:

[0121] Y=X b,l,d +PE L,D (4)

[0122] Where Y is the embedded tensor after adding the positional encoding, which contains the weighted positional encoding information for further time aggregation calculations.

[0123] X b,l,d Represents the relevant embedding of the problem, of size (b,l,d), where b is the batch size, l is the sequence length, d is the embedding dimension, PE is the parameter matrix, which means extracting the encoding of the first L positions from the position encoding tensor and the embedding dimension is D.

[0124] PE is the self-learning parameter matrix, which represents the position encoding; PE L,D It means extracting the encoding of the first L positions from the position encoding PE tensor with an embedding dimension of D, and expanding it in the first dimension to make its size compatible with the input tensor X, so that it can be added element-by-element with the input embedding tensor.

[0125] 2.2MGC Unit

[0126] Mainstream methods in the field of TKGQA often focus on extracting representations of question encodings and their associated relationships, often ignoring the importance of temporal conditions in question reasoning. To this end, we constructed an implicit temporal information capturer MGC, which consists of a multi-head attention mechanism layer, a graph attention network, and a one-dimensional convolutional neural network.

[0127] We use an information fusion layer to fuse multi-dimensional information into a time representation t mIn , this layer consists of a dedicated learnable encoder Transformer (·), which consists of two Transformer encoding layers. This encoding layer can promote the interaction and attention between time tags, effectively embedding and fusing different temporal features into a unified representation, thereby achieving deep integration and refinement of information. We d Add the position embedding Y to get T d ′. T m is the time tag embedding matrix, and the calculation formula is as follows:

[0128] T m = Transformer(T d ′) (5)

[0129] The time series of the problem is expressed as t CLS represents the global representation of the time series, represents the embedding representation of the 1st, 2nd, ..., nth day in the time series. Finally, it is represented as t m =t CLS Repeat the aggregation for each monthly granularity to get the annual granularity of t τ The time representation is, at this time t v =t m .

[0130] The temporal representation t of the question at τ in the multi-head attention mechanism τ It is mapped to multiple subspaces, each of which corresponds to an attention head. By aggregating the features of different subspaces, information on different dimensions can be effectively captured. The results of multiple attention heads are concatenated with a single attention head to obtain the final attention output. The calculation of each attention head can be expressed as:

[0131]

[0132] in and They are the linear transformation matrices used to generate query, key, and value respectively;

[0133] h represents h attention heads;

[0134] Q h , K h 、V h Represents the query (Query), key (Key), and value (Value) of the h-th attention head;

[0135] Attention() represents the attention mechanism;

[0136] · Tis the transpose of the matrix;

[0137] d k is the dimension of the key vector;

[0138] Softmax is used to normalize the calculated attention weights.

[0139] Based on multi-head attention, the graph attention network GAT further introduces this into the graph structure and uses the adjacency matrix to determine the connection relationship between nodes. Specifically, GAT uses time to represent t τ After linear transformation, we get the eigenvector h i . It can be simply expressed as:

[0140] h i = t τ W i (8)

[0141] Where W i is a weight matrix, which maps the temporal embedding to the feature space of the node. Next, GAT calculates the attention coefficient a between node i and its neighbor node j ij . Through the LeakyReLU activation function and node feature h i With h j Splicing, the attention scores between the obtained nodes are normalized by the softmax function. The formula is as follows:

[0142]

[0143] Among them, a ij is the attention coefficient between node i and its neighbor node j;

[0144] exp() is the exponential function of the natural constant e;

[0145] for a i The transpose of

[0146] LeakyReLU() is the activation function;

[0147] k represents the neighbor node of node i;

[0148] a i is the learnable weight vector of attention;

[0149] N(i) represents the set of neighbor nodes of node i;

[0150] || represents the concatenation operation of vectors;

[0151] h i ′ represents the updated feature of node i.

[0152] Finally, we use a one-dimensional convolutional layer to process the updated node features and generate a new time embedding t. The time-related local features are extracted through the convolution operation and the activation function σ. The formula is as follows:

[0153]

[0154] Where Conv represents the convolution operation;

[0155] σ represents the activation function;

[0156] C in Indicates the number of channels of input data;

[0157] h′ c is the cth channel in the input feature map;

[0158] W c is the corresponding convolution kernel weight;

[0159] b is the bias term;

[0160] The symbol * is the convolution operator.

[0161] 3 Ensemble Learning Layer

[0162] 3.1 Gated Recurrent Unit

[0163] In order to enhance the model's ability to extract implicit features when dealing with complex reasoning problems, we input the relational representation r of the entity semantic unit and the time embedding t extracted from the time semantic unit into the GRU (Gated Recurrent Unit) model to obtain a new relational representation R and an updated time embedding t'. Through this processing, the model can more effectively improve its ability to capture time and relations. The formula is as follows:

[0164] R = GRU(r) t ′ = GRU(t) (12)

[0165] 3.2 Integration Layer

[0166] The integration layer aims to score candidate answers through a relevance scoring function and use an integrated learning method to obtain the final answer. Specifically, this module combines the global question representation q, the head entity embedding e s , tail entity embedding o In the decision-making process, the time embedding t is retained. τ And the final time representation t′, in order to evaluate the authenticity of the candidate answer, and finally get the score of the candidate answer by maximizing the evaluation score. The formula is as follows:

[0167]

[0168] Among them, Score represents the score of the candidate answer;

[0169] max means taking the maximum value;

[0170] φ( ) represents the TComplEx scoring function;

[0171] e ∈ is the candidate answer entity, which is all entities in TKG;

[0172] e s 、e o Head entity embedding and tail entity embedding respectively;

[0173] R represents the relational representation of the final entity semantic unit;

[0174] t′ represents the final time representation;

[0175] t τ represents the time representation at τ.

[0176] A scoring function is used to calculate the relevance of the candidate entity to the question. In order to ensure that the entity s and tail entity embedding o With the same weight, we adopt a symmetrical perspective to deal with e s With e o , that is, the scores remain the same when swapping two entities. In addition, through the max(·) function, we effectively exclude the possibility that in some cases, due to the candidate entity e s or e o Possible error scores when the entity is a virtual entity.

[0177] During the training process, the model converts the combined score vector into a probability distribution through softmax, and uses the cross entropy loss function to optimize the prediction results, thereby improving the classification performance of the model.

[0178] 4 Experimental results and analysis

[0179] In order to verify the effect of the MGIT model, a solid foundation is laid for the performance evaluation of the model by explaining the data set selection, experimental environment configuration, experimental parameter setting and evaluation criteria involved in subsequent experiments. In addition, a baseline model for comparison will be introduced to clearly demonstrate the advanced nature of the MGIT model. By analyzing a representative complex problem case, this chapter will intuitively demonstrate the significant advantages of the MGIT model over the baseline model in solving the problem. In order to further explore the internal mechanism of the model, an ablation experiment was also implemented to verify the stability of the key modules, and the analysis and comparison were conducted based on the experimental results.

[0180] 4.1 Experimental Dataset

[0181] Our experimental data uses the MULTITQ dataset, which focuses more on the multi-granularity level of temporal reasoning. The TempQuestions dataset TKGQA was proposed by Jia et al. It contains 1,271 questions and introduces the temporal dimension into the knowledge graph question answering task for the first time.

[0182] To overcome the bottleneck of small data size, Saxena et al. (2021) released CRONQUESTIONS, a large-scale temporal knowledge graph question answering dataset containing 410,000 questions extracted from Wikidata.

[0183] Existing studies often ignore the complex requirements of TKGQA for temporal granularity in practical applications, that is, questions may involve a mixture of multiple temporal granularities (such as a combination of days and months). To address this challenge, Ziyang Chen et al. (2023) proposed the concept of multi-granularity temporal question answering and developed a large-scale multi-granularity TKGQA dataset MULTITQ. MULTITQ contains 500k question-answer pairs, a significant increase compared to other temporal question-answering datasets. Secondly, the knowledge graph ICEWS on which MULTITQ relies has extremely high semantic complexity, which means that it can more accurately simulate complex temporal relationships in the real world. In addition, MULTITQ supports question and answering at multiple temporal granularities and is no longer limited to simple question and answering tasks under a single time dimension. The experiments in this paper are conducted on the MULTITQ dataset, and the question categories of its dataset are shown in Table 1.

[0184] Table 1 Statistics of question categories in MULTITQ

[0185]

[0186] Multiple question: This type of question contains multiple time constraints and can be more diverse. For example, “Which country first visited China in 2024?” This type of question needs to satisfy multiple time constraints, namely Equal and First. In the MULTITQ dataset, multiple questions are further divided into three categories: Equal multi, After first, and Before Last.

[0187] Single question: This type of question contains a time constraint, and the answer can be an entity or a time. For example, the question "Who visited the China in 2024?" only needs to satisfy a single time constraint, namely Equal. In the MULTITQ dataset, single questions are further divided into three categories: Equal, Before / After, and First / Last.

[0188] 4.2 Evaluation indicators and experimental parameters

[0189] 4.2.1 Evaluation Metrics

[0190] The model evaluation index adopts the Hits@n accuracy rate commonly used in the TKGQA field, which mainly records whether the four-tuple ranking of the head and tail entities to be detected is in the top n, and then accumulates the number of resource groups that meet the requirements, calculates the proportion of the total number, and obtains the Hits@n score. The specific definition is as follows:

[0191]

[0192] Where S quad is the number of quadruplets to be predicted, rank i Record the predicted quadruple ranking of the i-th head and tail entity. II() is an indicator function, which is 1 if the condition is true, otherwise 0. This paper uses the most commonly used Hits@1 and Hits@10 as evaluation indicators.

[0193] 4.2.2 Experimental parameters

[0194] The model is implemented in PyTorch. We use TComplEx as our TKG embedding with dimension D = 512. We set the number of transformer layers of the encoder TTransformer to 2, with 4 heads per layer. The model parameters are updated by Adam with a learning rate of 0.0001. All experiments are conducted on a server with an Intel(R) Core(TM) i9-11900K@3.50GHz CPU and a 24GB NVIDIA GeForce RTX 3090 GPU.

[0195] Changing the number of attention mechanisms will have different effects on the experiment, such as Figure 2 As shown in the figure, after many experiments, the model effect is best when the number of attention heads is 16.

[0196] In the GAT model, alpha is a trainable (learnable) parameter that accurately quantifies the attention strength between each node and its neighboring nodes. The dropout technology is cleverly integrated into the attention calculation mechanism and the output layer of the model to improve the generalization ability of the model and avoid overfitting.

[0197] Our experiments show that when the dropout ratio is set to 0.5 and the alpha parameter is set to 0, the model exhibits the best prediction effect, such as Figure 3 This experimental result not only verifies the effectiveness of dropout in preventing the model from over-relying on specific features, but also reveals that in some scenarios, such as in social graphs, nodes represent users and edges represent interactions between users. In this case, directly relying on the structural information of the social network graph to determine the attention weights without additional weight parameters can more effectively model the strength of the relationship between users.

[0198] 4.3 Experimental Results and Analysis

[0199] 4.3.1 Baseline model comparison experiment

[0200] This section selects a representative TKGQA model and compares it with the MGIT model on the MULTITQ dataset:

[0201] BERT uses a self-attention mechanism to capture contextual information. To adapt to multi-granularity temporal information, embeddings of different temporal granularities can be added to the input, allowing BERT to recognize multi-level temporal information in text representation.

[0202] DistillBERT is a streamlined version of BERT. It processes multi-granularity temporal information through lightweight temporal embedding, reducing the amount of computation while retaining the temporal context. It is suitable for obtaining efficient temporal semantics at different granularities.

[0203] ALBERT achieves more efficient representation by sharing parameters. It strengthens the capture of temporal information at multiple levels by adding temporal embeddings of different granularities, thereby obtaining temporal semantics from fine-grained to coarse-grained.

[0204] EmbedKGQA is designed with static KGs. To handle multiple time granularities, timestamps are ignored during pre-training and random time embeddings are used.

[0205] CronKGQA is designed for a single time granularity. To handle multiple granularities, time embeddings at year / month granularity are randomly extracted from the corresponding day embeddings.

[0206] MULTIQA is designed to handle multi-granularity time, using TKG embedding to obtain the semantic information of KG, and obtaining temporal semantic embeddings of different granularities through a multi-granularity aggregation module.

[0207] To evaluate BERT, DistillBERT, and ALBERT 19] For these three pre-trained models, we first generated their question embeddings based on the language model (LM) and concatenated them with the entity embedding and time embedding. Subsequently, the concatenated vectors were input into a learnable projection layer for processing, and the final generated embedding vectors were scored and compared with all candidate entities and timestamps through dot product operations. Table 2 shows the Hist@1 and Hist@10 performance indicators of the MGIT model and various baseline models for different question types and answer types. Table 3 shows the Hits@1 results of different models in multi-granularity time dimensions. The experiments were tested based on different time granularities (days, months, years) and different question types (Equal, Before / After, Equal Multi). All results were obtained by averaging five consecutive experiments to ensure that the results are reliable and stable.

[0208] Table 2 Comparison results between MGIT method and baseline methods

[0209]

[0210] As can be seen from Table 2, our MGIT method surpasses all baseline models in various evaluation indicators of Hist@1 and Hist@10, showing excellent performance. First, by comparing EmbedKGQA with pre-trained LMs (BERT, DistillBERT, AlBERT), we can see that even without providing any temporal information, introducing KG representation with score function can significantly improve the reasoning ability of the model, while BERT, DistillBERT and ALBERT are not ideal in dealing with temporal issues due to the lack of a dedicated temporal information encoding mechanism and KG representation with score function.

[0211] Since EmbedKGQA has non-temporal embeddings, its performance on questions whose answers are temporal is very low. By comparing CronKGQA and EmbedKGQA, we find that introducing a pre-trained temporal representation helps a lot in answering temporal questions. In this case, CronKGQA improves Hits@1 by 7.3 percentage points and Hits@10 by 14.9 percentage points relative to EmbedKGQA. But paradoxically, while CronKGQA outperforms EmbedKGQA in overall results, it is not as effective at multi-granularity temporal reasoning. The single-granularity temporal representation introduced by CronKGQA improves reasoning at the corresponding temporal granularity, but is misleading at other granularities, causing the results of multi-granularity temporal knowledge graph question answering (TKGQA) to deteriorate instead. This also highlights that previous models lack the ability to reason about multi-granularity temporal question answering.

[0212] In addition, due to the use of a multi-granularity aggregation module, MultiQA has improved Hits@1 and Hits@10 by 1.4 percentage points and 2.7 percentage points respectively compared to CronKGQA. However, MultiQA did not pay much attention to the effective fusion of time and entity, which may cause the model to retrieve the wrong relationship between time and entity, thus affecting the accuracy of reasoning. The MGIT model effectively integrates the relationship between time and entity through an integrated learning layer, thereby improving the accuracy of answer selection and eliminating the corresponding noise. And the synergy of the multi-layer structure of the information retrieval layer, TFN layer and self-learning encoding strategy. Compared with the MultiQA method, MGIT has improved Hits@1 by 8.7 percentage points and Hits@10 by 2.6 percentage points, indicating the superiority and effectiveness of the MGIT model in dealing with time-sensitive problems.

[0213] Table 3 Experimental results of multi-particle time on Hits@1

[0214]

[0215]

[0216] In order to further observe the effect of this method on implicit time problems, we conducted a classification experiment. The results are shown in Table 3. Referring to the method of the baseline model, the implicit time uses the before / after problem. It can be seen that this method has achieved significant improvements on the before / after problem at the day, month, and year granularity, and has achieved significant improvements on the Equal and Equal Multi problems at the day and year granularity, with Hits@1 reaching 0.691% and 0.328% respectively. This shows that the MGIT model can better handle implicit time problems.

[0217] Figure 4 The performance comparison of MGIT and other baseline models on various question types is further visualized. It can be seen that for the equal and before_after and before_last and after_first question types, MGIT's Hits@1 score is much higher than CronKGQA and MultiQA, showing a clear advantage. Especially in the before_after question type, MGIT's Hits@1 is as high as 0.606, which is at least 18.6% higher than the other two baseline models. This shows that the MGIT model performs better in dealing with complex time problems, such as implicit time problems.

[0218] 4.3.2 Ablation Experiment

[0219] In order to evaluate the effectiveness of each component of the MGIT model, we conducted an ablation experiment, and the results are shown in Table 4. The experiment gradually removed different modules in the model, including GRU, CNN, GAT, multi-head attention mechanism, decision ensemble learning strategy, and self-learning encoding module, and observed the impact of each module on the overall performance.

[0220] Table 4 Ablation experiment results

[0221]

[0222]

[0223] The experimental results show that the complete MGIT model achieves the best performance in all tasks, especially in multiple-choice questions and time-type questions, with Hits@1 reaching 0.380% and Hits@10 reaching 0.661%. In comparison, when GRU is removed (w / o GRU), the model performance Hits@1 drops by 4.7% and Hits@10 drops by 1%. We believe that the removal of GRU weakens the ability to process complex temporal information. When CNN is removed, the model performance Hits@1 drops by 0.8% and Hits@10 drops by 0.7%. We believe that this is because the removal of CNN affects the extraction of multi-level information. When GAT is removed, the model performance Hits@1 drops by 2.4% and Hits@10 drops by 1.8%. We believe that the absence of GAT limits the model's capture of implicit information in the graph structure. This shows that the MGC module is crucial for capturing temporal dependencies and temporal information in questions.

[0224] In addition, after removing the multi-head attention mechanism (w / o), ensemble learning strategy (w / o), and self-learning encoding (w / o) modules, the model performance has decreased to varying degrees. After removing the ensemble learning strategy, Hits@1 decreased by 1.5% and Hits@10 decreased by 0.4%, indicating that the ensemble learning strategy in this paper is conducive to robust reasoning of the model.

[0225] In summary, this paper designs a multi-granularity temporal knowledge graph reasoning framework MGIT, which consists of an information retrieval layer, a question representation layer, a TFN layer, and an integrated learning layer. The effectiveness of the design of each module is verified by comparative experiments and ablation experiments. The model adopts a temporal knowledge graph in the form of a quadruple, integrates a graph attention network (GAT) with a multi-head attention mechanism, effectively captures the implicit temporal information in the question, and enhances the ability to capture complex temporal dependencies through the synergy of a convolutional neural network (CNN) and a gated recurrent unit (GRU). In addition, the model designs a self-learning encoding strategy that enables it to dynamically adapt to data of different temporal granularities. When processing time tensors, an integrated learning strategy is adopted to retain multiple time segments as "weak learners" and select the optimal time representation through decision making. Finally, the performance of the MGIT model on the MULTITQ dataset demonstrates its superior performance in temporal reasoning tasks.

[0226] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A temporal knowledge graph question answering method based on multi-granularity implicit temporal awareness, characterized in that: The following steps are involved: S1, obtain the question text and identify the entity information and time information in the question text; S2, using an embedding-based representation learning method, retrieves the extracted information in the temporal knowledge graph and obtains the head entity embedding e s , tail entity embedding o , and the time embedding t τ ; S3, input the entity information into the question representation layer to obtain the relation embedding vector r: S4, embed the time information and its time into t τ Input the time series fusion network to obtain a new time embedding t; S5, input the relationship embedding vector r and the new time embedding t into the integrated learning layer to obtain the scoring score, and take the result with the highest score as the answer.

2. According to claim 1, a multi-granularity implicit temporal-aware temporal knowledge graph question answering method is characterized in that: Step S3 includes: The pre-trained RoBERTa model is used to encode the question text and generate the initial semantic matrix Q of the question: Q = W R RoBERTa(q text ) (1) Where Q = [q CLS ,q1,q2,...,q n ]; q CLS is a global representation of the question text; q1,q2,q n is the contextual representation of the 1st, 2nd, and nth words in the question; W R is the projection matrix of the question text; q text is the question text; Get the final problem representation q, q = q CLS ; Then the obtained global problem representation q is passed through the fully connected layer to obtain the relation embedding vector r: r = W r q + b r (2) Where W r is a learnable weight matrix; q is the global problem representation; b r is a learnable bias vector.

3. According to claim 1, a multi-granularity implicit temporal-aware temporal knowledge graph question answering method is characterized in that: S4 includes the following steps: S4-1, time information and its time embedding t τ Construct the time semantic matrix T d , then use the encoder to encode the question text to get the encoded position embedding Y; then use the Transformer encoder to get the time series representation T of the question m : T m = Transformer(T d ′ ) (5) Among them, T d ′ =T d +Y; t CLS represents the global representation of the time series, represents the embedding representation of the 1st, 2nd, ..., nth day in the time series; At this time, the time embedding t τ =t CLS ; S4-2, embed the obtained time into t τ Input the multi-head attention mechanism, and the results of multiple attention heads are concatenated into a single attention head to obtain the final attention output; S4-3, the final attention output is input into the graph attention network. At this time, t τ After linear transformation, we get the eigenvector h i : h i = t τ W i (8) Where W i is the weight matrix; The graph attention network calculates the attention coefficient a between node i and its neighbor node j ij , through the LeakyReLU activation function and node feature h i With h j Splicing, normalize the attention scores between the obtained nodes through the softmax function: Among them, a ij is the attention coefficient between node i and its neighbor node j; exp() is the exponential function of the natural constant e; for a i The transpose of a i is the learnable weight vector of attention; LeakyReLU() is the activation function; k represents the neighbor node of node i; N(i) represents the set of neighbor nodes of node i; || represents the concatenation operation of vectors; h i ′ represents the updated features of node i; S4-4, use a one-dimensional convolutional layer to process the updated node features and generate a new time embedding t: Where Conv represents the convolution operation; σ represents the activation function; C in Indicates the number of channels of input data; h ′ c is the cth channel in the input feature map; W c is the corresponding convolution kernel weight; b is the bias term; * is the convolution operator.

4. According to claim 1, a multi-granularity implicit temporal-aware temporal knowledge graph question answering method is characterized in that: The encoder is a self-learning encoder, and the encoded position embedding Y is obtained by encoding with the self-learning encoder, including: Y = X b,l,d + PE L,D (4) Where X b,l,d Representing the relevant embeddings of the question; PE L,D It means extracting the encoding of the first L positions from the position encoding PE tensor and the embedding dimension is D.

5. According to claim 1, a multi-granularity implicit temporal-aware temporal knowledge graph question answering method is characterized in that: S5 includes the following steps: S5-1, input the relation embedding vector r and the new time embedding t into the gated recurrent unit respectively to obtain the new relation representation R and the updated time embedding t ′ ; S5-2, and then use the scoring formula to get the score of the candidate answer: Score=max(φ(e s ,R,e ∈ ,t ′ ),φ(e o ,R,e ∈ ,t ′ ),φ(e s ,R,e ∈ ,t τ ),φ(e o ,R,e ∈ ,t τ )) ⊕max(φ(e s ,Bid ∈ ,t ′ ),φ(e s ,Bid ∈ ,t τ ))(13) Among them, Score represents the score of the candidate answer; max means taking the maximum value; φ() represents the TComplEx scoring function; e ∈ is the candidate answer entity, which is all entities in the temporal knowledge graph; e s 、e o Head entity embedding and tail entity embedding respectively; R represents the relational representation of the final entity semantic unit; t ′ Indicates the final time representation.

6. According to claim 1, a multi-granularity implicit temporal-aware temporal knowledge graph question answering method is characterized in that: include: Use named entity recognition tools to identify entity information in question text, and use time extraction tools to obtain time information.

7. According to claim 1, a multi-granularity implicit temporal-aware temporal knowledge graph question answering method is characterized in that: The loss function of the ensemble learning layer is cross entropy loss.

Images