A question-answering method for multi-granularity temporal knowledge graphs

Abstract

The present invention discloses a question-answering method for a multi-granularity temporal knowledge graph, comprising the following steps: step 1, preprocessing a data set to obtain entities, relationships, event entities and time; step 2, using the data set to train an entity extraction model and a relationship extraction model to obtain more accurate entities and relationships; step 3, using the obtained entities, relationships, event entities and time to perform a graph query to obtain a final candidate answer set; the question-answering method for a multi-granularity temporal knowledge graph in the present invention is adaptable to multi-granularity temporal knowledge graph question-answering scenarios in a dynamic environment, has high-precision question-answering performance under multi-granularity time and multi-temporal constraints, and the method ensures reasoning efficiency, has good generalization, and has good performance for different types of questions.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing and artificial intelligence technology, and in particular relates to a question-answering method for multi-granularity temporal knowledge graphs. Background Art

[0002] A knowledge graph is a structured graph database used to organize and represent knowledge. It contains various entities and the relationships between them. The knowledge graph aims to simulate the knowledge associations and semantic relationships in human thinking so that computers can better understand and process tasks such as human language, information retrieval, and question answering.

[0003] Knowledge graph-based question answering uses facts in the knowledge graph to answer natural language questions. By using information retrieval, semantic analysis, and other methods, the most appropriate answer can be found in the knowledge graph to achieve efficient answers to questions. Conventional knowledge graph question answering uses a static knowledge graph as a knowledge base to answer natural language questions. The knowledge and information in it are immutable, but in real life, knowledge is often updated dynamically, which is also one of the important characteristics of knowledge. In the face of this kind of dynamic knowledge, conventional methods are difficult to effectively reason and answer, and it is difficult to fully meet the needs of actual business applications.

[0004] The dynamic and open organizational environment requires faster information processing, more timely information updates, and more extensive information innovation. Information in the time dimension is becoming more and more important. The development of the time series knowledge graph question answering system is of great significance to deepening human-computer interaction and information acquisition. It can help users better understand and utilize time series data, provide more accurate and comprehensive answers, and support complex time analysis and prediction tasks.

[0005] The temporal knowledge graph question answering task has the following three characteristics: the knowledge graph contains time information; the answer is an entity or time; and complex temporal reasoning is required. However, the existing temporal knowledge graph system has very limited comprehensive processing capabilities for multi-granularity time and multi-temporal constraints, especially the lack of attention to multi-granularity time information, and there is no systematic method to deal with the impact of inconsistent multi-granularity time information on the performance of the question answering system.

[0006] In summary, the existing temporal knowledge graph question and answering has limited question and answer reasoning capabilities under multi-granularity time and multi-temporal constraints, and cannot meet the multi-granularity temporal knowledge question and answer scenarios in dynamic environments. Summary of the invention

[0007] (1) Technical issues to be solved

[0008] The present invention discloses a question-answering method for a multi-granularity temporal knowledge graph, aiming to solve the problem that the question-answering reasoning ability of temporal knowledge graph question-answering is limited under multi-granularity time and multi-temporal constraints, and cannot meet the multi-granularity temporal knowledge question-answering scenarios in a dynamic environment.

[0009] (2) Technical solution

[0010] The present invention discloses a question-answering method for a multi-granularity temporal knowledge graph, comprising the following steps:

[0011] Step 1: preprocess the data set to obtain entities, relations, event entities and time;

[0012] Step 2: Use the dataset to train the entity extraction model and the relationship extraction model to obtain more accurate entities and relationships;

[0013] Step 3: Use the obtained entities, relationships, event entities and time to perform graph query to obtain the final set of candidate answers.

[0014] Furthermore, the specific steps of step 1 are as follows:

[0015] Step 1.1, divide the data set into training set, validation set and test set;

[0016] Step 1.2: Perform time extraction, entity extraction, event extraction, and relationship extraction on each question in the training set, validation set, and test set;

[0017] The specific steps of step 2 are as follows:

[0018] Step 2.1, training the relationship extraction model;

[0019] Step 2.2, training entity extraction model;

[0020] In step 3, for each question, the entity, relationship, event entity, and time information have been obtained, and a graph query is performed based on these four key query information to obtain a candidate answer set containing entity location or time location. The present invention designs the following two query routes:

[0021] Step 3.1, entity query;

[0022] Step 3.2, time query.

[0023] Furthermore, in step 1.1, the training set data contains 350,000 question-answer pairs, and the validation set data contains about 25,000 question-answer pairs, wherein each question-answer pair contains three parameters: quid is the unique number of the question, question is the question text, and answers is the answer to the question. A question text corresponds to one or more answers. The test set data contains 25,000 question-answer pairs, and each question-answer pair contains only quid and question.

[0024] In step 1.2, the following steps are performed on each question text in the training set, the validation set, and the test set:

[0025] Step 1.2.1, time extraction;

[0026] Step 1.2.2, entity extraction;

[0027] Step 1.2.3, event entity extraction;

[0028] Step 1.2.4, relation extraction.

[0029] Furthermore, in step 1.2.1, the time is extracted by using a regular expression matching method, and the regular expression matching rule is as follows:

[0030] pattern=\d{2}.\d{2}.\d{4}|(?:\d{4}-\d{2}(?:-\d{2})?)|(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(? :y)? |Aug(?:ust)? |Oct(?:ober)? |Dec(?:ember)? )\s\d{1,2}|\b(?:st|nd|th)? (?: (?: (?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tembe r)? |Oct(?: ober)? |Dec(?: ember)? )|\d{1,2})\b\D{1,5}(?:\D{0,7}of\D{0,7})? \b\d{4}\b′

[0031] The time formats that can be matched include pure numbers, month information in English or English abbreviations, date information in ordinal numbers, and separated month and year information;

[0032] The specific method of step 1.2.2 is as follows:

[0033] Step 1.2.2.1, use the NLTK library to normalize each question in the dataset, including word form restoration, uppercase and lowercase conversion, Anglo-American word conversion, and deletion of stop words, so as to reduce the diversity of questions and reduce the difficulty of matching;

[0034] Step 1.2.2.2, create a string matching template for each entity in the knowledge graph. For an entity with n words, rearrange the words in the entity to generate n matching templates.

[0035] Step 1.2.2.3, using a matching algorithm to calculate the matching degree between the normalized question and each matching template and obtain the entities and entity positions in the question;

[0036] The specific steps of step 1.2.3 are as follows:

[0037] Step 1.2.3.1, normalize the question sentence;

[0038] Step 1.2.3.2, check whether event prompt words such as before / after, first / last appear in the question. If they do, mark their positions. If they do not, it means that the question does not have event time constraints.

[0039] Step 1.2.3.3, determine the positional relationship between the event prompt word and the entity. If there is an entity within 10 characters after the event prompt word, mark the entity as an event entity.

[0040] The step 1.2.4 is a relationship extraction method based on the bag-of-words model, which extracts the relationship with the highest similarity score to the question as the relationship of the question. The steps are as follows:

[0041] Step 1.2.4.1, using the extracted entities and the answers of the training set, obtain a list of all quadruple pairs in the graph that satisfy (entity, *, answer, *) and ((answer, *, entity, *), and group the relations that appear in the list into a candidate relation set;

[0042] Step 1.2.4.2, standardize all relations in the knowledge graph to initialize the bag-of-words model;

[0043] In step 1.2.4.3, the bag-of-words model is used to calculate the similarity scores between the relation and the question, and the relation with the highest score is taken as the relation of the question.

[0044] Furthermore, the specific steps of step 2.1 are as follows:

[0045] Step 2.1.1, input natural language question S = (s 1 ,s 2 ,…,s n ), s n Represented as words, the natural language question is segmented and embedded to obtain an embedding vector E = (e 1 ,e 2 ,…,e t ), e t is the element value at the tth position;

[0046] Step 2.1.2, the embedding vector is passed through the language model RoBERTa-large to extract contextual semantic features f = (c 1 ,c 2 ,…,c t ), c t is a characteristic element;

[0047] Step 2.1.3, sending the context semantic feature F to the Linear layer for classification to obtain the relationship category contained in the question;

[0048] The specific steps of step 2.2 are as follows:

[0049] Step 2.2.1, input natural language question S = (s 1 ,s 2 ,…,s n ), s n Represented as words, the natural language question is segmented and embedded to obtain an embedding vector E = (e 1 ,e 2 ,…,e t ), e t is the element value at the tth position;

[0050] Step 2.2.2: The embedding vector is passed through the language model RoBERTa-large to extract contextual semantic features f = (c 1 ,c 2 ,…,c t ), c t is a characteristic element;

[0051] Step 2.2.3, extracting the BIO-mark of the question sentence through the LSTM and Linear layers using the contextual semantic feature F;

[0052] Step 2.2.4, obtaining entity location information and event location information in the question sentence according to BIO-mark;

[0053] Step 2.2.5, according to the entity location information and event location information, F = (c 1 ,c 2 ,…,c t ) The semantic feature of the corresponding position is taken as the entity semantic feature h;

[0054] Step 2.2.6, link the entity semantic feature h to the existing entities in the graph through the entity linking module, so as to obtain the entity and the entity position.

[0055] Furthermore, the RoBERTa-large language model is a large pre-trained language model based on the Transformer architecture, which learns contextual information in text and extracts semantic features through masked language modeling tasks.

[0056] Furthermore, the BIO-mark in step 2.2.4 is used to mark entities or events in the text. This marking method has five types of markings, including entity start token, entity internal token, event entity start token, event entity internal token, and other types of tokens. Each token in the text is marked as one of the above five types. By using BIO-mark, entities or events can be accurately identified and located in the text, and used for training or evaluating models in subsequent processing.

[0057] The entity link module in step 2.2.6 is composed of a bidirectional LSTM module and a Linear layer, and the calculation formula is as follows:

[0058] h=(c 1 ,c 2 ,c 3 )

[0059] Q=LSTM(h)=LSTM(c 1 ,c 2 ,c 3 )

[0060] P=Linear(Q)

[0061] The LSTM module is used to aggregate entity semantic information features h of different lengths into an aggregate feature Q. The Linear layer maps the aggregate feature Q to a probability vector P representing the position number id of each entity, and takes the position number with the maximum value of the probability vector as its entity index.

[0062] Furthermore, the specific method of step 3.1 includes the following steps:

[0063] Step 3.1.1, based on the entities and relations extracted from the question, query all quadruple groups that meet the (entity, relation, *, *) format in the graph, where the first * represents any entity and the second * represents any time. All quadruple groups that meet the conditions are called initial candidate quadruple groups.

[0064] Step 3.1.2, determining the time constraint type of the question, including three types: specific time point, specific time period, and event time period; filtering the initial candidate quadruple according to different question types and obtaining filtered candidate quadruple;

[0065] Step 3.1.3, sorting the filtered candidate quadruplets in time;

[0066] Step 3.1.4, outputting the entity positions of the first ten sorted filtered candidate quadruplets as a candidate answer set;

[0067] The specific method of step 3.2 comprises the following steps:

[0068] Step 3.2.1, based on the entities and relations extracted from the question, search the graph for all quadruple groups that conform to the form of (entity 1, relation, entity 2, *), where * represents any time, and obtain all the initial candidate quadruple groups that meet the conditions;

[0069] Step 3.2.2, determine whether there is a first / last order constraint in the question sentence, and sort the initial candidate quadruple in forward order by time if there is a first; sort the initial candidate quadruple in reverse order if there is a last; and do not sort the initial candidate quadruple in time if there is no order constraint, and obtain sorted candidate quadruple;

[0070] Step 3.2.3, according to the granularity required in the question, the time information in the sorted candidate quadruple is subjected to granularity correction processing;

[0071] Step 3.2.4, output the time positions of the first ten sorted candidate quadruplets after granularity correction as the candidate answer set.

[0072] Furthermore, in step 3.1.2, the time constraint type of the question is determined by first defining the time constraint words as before / after, and if the time constraint words are followed by entities, the question is an event time period type question, if the time constraint words are followed by specific time, the question is a specific time period type question, and if there is no time constraint words, the question is a specific time point type question;

[0073] When the time constraint type of the question is an event time period type question, the time constraint words are derived from the initial candidate quadruple in the question, and the filtering steps are as follows:

[0074] Step 3.1.2.1, based on the entities, event entities and relations extracted from the question, search in the graph for all quadruple groups that meet the conditions of (entity, relation, event entity, *), where * represents any time, and obtain the initial candidate quadruple group;

[0075] Step 3.1.2.2, sort the times in all initial candidate quadruple groups that meet the query conditions;

[0076] Step 3.1.2.3, determine the specific time constraint according to the type of the time constraint word in the question sentence. When the time constraint word in the question sentence is a forward constraint word, extract the earliest time in the time sequence as the time constraint; when the time constraint word in the question sentence is a backward constraint word, extract the latest time in the time sequence as the time constraint;

[0077] Step 3.1.2.4, forward filtering or backward filtering the initial candidate quadruple using the determined time constraint to obtain a filtered candidate quadruple;

[0078] The forward filtering is to filter out the initial candidate quadruple occurring after the extraction time, and the backward filtering is to filter out the initial candidate quadruple occurring before the extraction time;

[0079] When the time constraint type of the question is a specific time period type question, it is necessary to obtain the time constraint based on the time constraint words in the question to filter the initial candidate quadruple; if the time constraint words are forward constraint words, the initial candidate quadruple occurring after the extraction time will be filtered out; if the time constraint words are backward constraint words, the initial candidate quadruple occurring before the extraction time will be filtered out to obtain the filtered candidate quadruple;

[0080] The forward time constraint words refer to words that push time forward, such as before and in front of, and the backward constraint words refer to words that push time backward, such as after;

[0081] When the time constraint type of the question is a specific time point type question, the time is explicitly present, and the initial candidate quadruple is filtered according to the extracted time, and the initial candidate quadruple occurring at the extracted time is retained to obtain the filtered candidate quadruple;

[0082] The method of step 3.1.3 is as follows:

[0083] Determine whether there is a first / last order constraint in the question sentence. If there is a first, the filter candidate quadruple is sorted in forward order by time. If there is a last, the filter candidate quadruple is sorted in reverse order. If there is no order constraint, the filter candidate quadruple is not sorted in time.

[0084] Furthermore, the particle size correction process in step 3.2.3 is specifically performed as follows:

[0085] First, the question is marked with granularity. The year in the question is marked as year granularity, the specific month in the question is marked as year-month granularity, and other cases are marked as year-month-day granularity. The time information of the quadruple defaults to year-month-day granularity. According to the granularity mark of the question, the time information of the sorted candidate quadruple is processed into the corresponding granular format.

[0086] Compared with the prior art, the present invention has the following beneficial effects:

[0087] The question-answering method for multi-granularity temporal knowledge graphs in the present invention is adaptable to multi-granularity temporal knowledge graph question-answering scenarios in dynamic environments, and has high-precision question-answering performance under multi-granularity time and multi-temporal constraints. The method ensures reasoning efficiency and has good generalization. According to the MRR score of its test set, the method has good performance for different types of questions. BRIEF DESCRIPTION OF THE DRAWINGS

[0088] Figure 1 A schematic flow chart of an implementation method of the present invention is shown;

[0089] Figure 2 A schematic diagram of the corresponding method flow of entity extraction in the present invention is shown;

[0090] Figure 3 A schematic diagram of the corresponding method flow of event extraction in the present invention is shown;

[0091] Figure 4 A schematic diagram of the corresponding method flow of the relationship extraction in the present invention is shown;

[0092] Figure 5 A schematic diagram of a relationship extraction model in the present invention is shown;

[0093] Figure 6 A schematic diagram of an entity extraction model in the present invention is shown;

[0094] Figure 7 A schematic diagram of the process of performing entity query on a graph in the present invention is shown;

[0095] Figure 8 A schematic diagram of the process of performing time query on a graph in the present invention is shown. DETAILED DESCRIPTION

[0096] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0097] For the purpose of reference and clarity, the technical terms, abbreviations or acronyms used below are summarized and explained as follows:

[0098] Transformer: A model structure that uses attention mechanism for feature extraction.

[0099] Linear: A fully connected layer in a neural network.

[0100] RoBERTa-large: A natural language processing model based on deep learning.

[0101] LSTM: is a variant of neural network specifically designed for processing sequence data.

[0102] BIO-mark: A tagging scheme for named entity recognition.

[0103] MRR: A metric for evaluating the performance of information retrieval systems.

[0104] ICEWS05-15: An open, multi-source data set of international conflicts and events.

[0105] NLTK Library: A Python library for natural language processing and text analysis.

[0106] Bag of Words model: refers to a simplified expression model in natural language processing and information retrieval, which uses the number of occurrences (frequency) of each word as the feature of the classifier.

[0107] AC algorithm: an efficient string matching algorithm.

[0108] Data augmentation: transforming the original data to generate new data samples.

[0109] Data synthesis: Generate completely new data samples through a model or algorithm.

[0110] Weak supervision: A machine learning or deep learning approach.

[0111] Word embedding: A text representation method for natural language processing that maps each word or phrase into a vector in the real number field.

[0112] Semantic features: Representations used to represent the semantic information of text.

[0113] Masked Language Modeling: A natural language processing technique commonly used in language generation tasks. When generating text, some words or tokens are masked or hidden, and then these masked words or tokens are predicted.

[0114] Entity: An entity usually refers to a named entity in a text, such as a person’s name, a place name, an organization name, and other specific objects or concepts.

[0115] Text normalization: converting texts of different forms and formats into a unified standard form.

[0116] Knowledge graph: A structured graph database used to organize and represent knowledge, which contains various entities and the relationships between them.

[0117] Dataset: refers to natural language questions and their answers.

[0118] Figure 1 A flow chart of an embodiment of the present invention is shown. A question-answering method for multi-granularity temporal knowledge graphs comprises the following steps:

[0119] Step 1: preprocess the data set to obtain entities, relations, event entities and time;

[0120] Step 2: Use the dataset to train the entity extraction model and the relationship extraction model to obtain more accurate entities and relationships;

[0121] Step 3: Use the obtained entities, relationships, event entities and time to perform graph query to obtain the final set of candidate answers.

[0122] It should be noted that the time series knowledge graph used in the present invention is ICEWS05-15, with a time range of 2005-2015. Each row of data in the time series knowledge graph is a four-tuple, and its format is (entity 1, relationship, entity 2, time). For example, a row of data in the graph is: Citizen_(Country name)Reject Country name 2005-01-01, where the time format is year-month-day format;

[0123] All questions in the dataset are built based on the temporal knowledge graph, where questions and answers contain information of multiple granularities, including yearly granularity, monthly granularity, and daily granularity. Multiple temporal logic constraints are designed in the questions, such as first, before, after, etc., and the answers to the questions are limited to the entities and time in the graph.

[0124] The following is a detailed explanation of the process of a question-answering method for multi-granularity temporal knowledge graphs using examples.

[0125] Execute step 1 to preprocess the data set to obtain entities, relationships, event entities, and time. The specific steps are as follows:

[0126] Step 1.1, divide the data set into training set, validation set and test set;

[0127] Step 1.2: Perform time extraction, entity extraction, event entity extraction, and relationship extraction on each question in the training set, validation set, and test set.

[0128] Specifically, in step 1.1, the data set is divided into a training set, a validation set, and a test set. The specific method is as follows:

[0129] The training set data contains 350,000 question-answer pairs, and the validation set data contains about 25,000 question-answer pairs, each of which contains three parameters: quid (unique question number), question (question text), and answers (answers to the question, which can be one or more). The test set data contains 25,000 question-answer pairs, each of which contains only quid and question.

[0130] Specifically, in step 1.2, the following steps are performed on each question text in the training set, the validation set, and the test set:

[0131] Step 1.2.1, time extraction;

[0132] Step 1.2.2, entity extraction;

[0133] Step 1.2.3, event entity extraction;

[0134] Step 1.2.4, relation extraction.

[0135] Furthermore, in step 1.2.1, time extraction, different forms of time information are extracted in a fixed format using a regular expression matching method, and the regular expression matching rules are as follows:

[0136] pattern=′\d{2}}.\d{2}.\d{4}|(?:\d{4}-\d{2}(?:-\d{2})?)|(?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul( ?: y)? |Aug(?: temper)? |Nov(?: ember)? (?: (?: (?:Jan(?:uary)?|Feb(?:ruary)?|Mar(?:ch)?|Apr(?:il)?|May|Jun(?:e)?|Jul(?:y)?|Aug(?:ust)?|Sep(?:tembe r)? |Oct(?: ober)? |Dec(?: ember)? )|\d{1,2})\b\D{1,5}(?:\D{0,7}of\D{0,7})? \b\d{4}\b′

[0137] The time formats that can be matched include pure numbers, month information in English or English abbreviations, date information in ordinal numbers, and separated month and year information.

[0138] Furthermore, the specific method of entity extraction in step 1.2.2 is as follows (eg Figure 2 ):

[0139] Step 1.2.2.1, use the NLTK library to normalize each question in the dataset, including word form restoration, uppercase and lowercase conversion, Anglo-American word conversion, and deletion of stop words, so as to reduce the diversity of questions and reduce the difficulty of matching;

[0140] Step 1.2.2.2, create a string matching template for each entity in the knowledge graph. For an entity with n words, rearrange the words in the entity to generate n matching templates.

[0141] For example, if the entity is citizen(AB), six matching templates will be created, namely citizenAB, citizen BA, A citizen B, A Bcitizen, B citizen A and BA citizen, where AB is the place name, i.e. the entity name;

[0142] Step 1.2.2.3, using a matching algorithm to calculate the matching degree between the normalized question and each matching template and obtain the entities and entity positions in the question;

[0143] For example, after entity matching for "question": "In which year did (C) first negotiate with (D)?", the result is "entities": [["C", 18, 22], ["D", 44, 65]], where C and D are entity names, (18, 22) and (44, 65) are corresponding entity positions;

[0144] Furthermore, the matching algorithm described in step 1.2.2.3 is an AC algorithm for multi-mode matching, which constructs all matching templates into a matching dictionary tree and then quickly matches the question to obtain the entities and entity positions in the question.

[0145] Furthermore, in step 1.2.3, event entity extraction, for example, for "question": "Before E, whovisted F?", the implied event conforms to the four-tuple of the form (E, Make a visit, F, *), where E and F are place names, i.e., entity names. Here, it is assumed that the entities and relations in the question have been correctly extracted. There is no time in the question, so * represents any time in the graph. At this time, it is necessary to determine whether the entity in the question should be used as an event constraint query space. It is found that there is a before event prompt word in the question, and there is also an entity E after it. The entity is marked as an event entity. The specific steps are as follows (such as Figure 3 ):

[0146] Step 1.2.3.1, normalize the question sentence;

[0147] Step 1.2.3.2, check whether event prompt words such as before / after, first / last appear in the question. If they do, mark their positions. If they do not, it means that the question does not have event time constraints.

[0148] Step 1.2.3.3, determine the positional relationship between the event prompt word and the entity. If there is an entity within 10 characters after the event prompt word, mark the entity as an event entity.

[0149] Furthermore, in step 1.2.4, relation extraction, based on the bag-of-words model, extracts the relation with the highest similarity score to the question as the relation of the question; for example, a question-answer pair in the training set is "question":"Before G, which country did H express his intention to negotiate with?","answers":["I","J"], where G, H, I and J are all entity names. According to the previously extracted entities and answers to the question, all quadruple groups satisfying (entity, *, answer, *) and (answer, *, entity, *) are searched in the graph, where the first * represents any relation and the second * represents any time. For example, a specific condition quadruple of the question can be expressed as (H, *, I, *); all relations appearing in the series of quadruple groups found are combined into a candidate relation set, and the bag-of-words model is used to calculate the similarity score between the relation and the question. The relation with the highest score is taken as the relation of the question. The relation with the highest similarity score in the example question is "Express intent to meet ornegotiate". The specific steps are as follows (such as Figure 4 ):

[0150] Step 1.2.4.1, using the extracted entities and the answers of the training set, obtain a list of all quadruple pairs in the graph that satisfy (entity, *, answer, *) and (answer, *, entity, *), and group the relations that appear in the list into a candidate relation set;

[0151] Step 1.2.4.2, standardize all relations in the knowledge graph to initialize the bag-of-words model;

[0152] In step 1.2.4.3, the bag-of-words model is used to calculate the similarity scores between the relation and the question, and the relation with the highest score is taken as the relation of the question.

[0153] It should be noted that based on the robustness considerations of the relationship and entity extraction methods, it is necessary to train the relationship extraction model and the entity extraction model separately to extract relationships and entities, so as to improve the accuracy and stability of relationship and entity extraction.

[0154] Execute step 2 and use the dataset to train the entity extraction model and the relationship extraction model to obtain more accurate entities and relationships. The specific steps are as follows:

[0155] Step 2.1, training the relationship extraction model;

[0156] Step 2.2, train the entity extraction model.

[0157] Specifically, the step 2.1, training the relationship extraction model, the specific steps are as follows (such as Figure 5 ):

[0158] Step 2.1.1, input natural language question S = (s 1 ,s 2 ,…,s n ), s n Represented as words, the natural language question is segmented and embedded to obtain an embedding vector E = (e 1 ,e 2 ,…,e t ), e t is the element value at the tth position;

[0159] Step 2.1.2, the embedding vector is passed through the language model RoBERTa-large to extract contextual semantic features F = (c 1 ,c 2 ,…,c t ), c t is a characteristic element;

[0160] Step 2.1.3, sending the context semantic feature F to the Linear layer for classification to obtain the relationship category contained in the question.

[0161] It should be noted that the relation extraction model can directly determine the relation in the question without relying on any additional information, and the extracted relation is used for reasoning of the test set.

[0162] Specifically, the step 2.2, training the entity extraction model, the specific steps are as follows (such as Figure 6 ):

[0163] Step 2.2.1, the input natural language question is S = (s 1 ,s 2 ,…,s n ), s n Represented as words, the natural language question is segmented and embedded to obtain an embedding vector E = (e 1 ,e 2 ,…,e t ), e t is the element value at the tth position;

[0164] Step 2.2.2: The embedding vector is passed through the language model RoBERTa-large to extract contextual semantic features F = (c 1 ,c 2 ,…,c t ), c t is a characteristic element;

[0165] Step 2.2.3, extracting the BIO-mark of the question sentence through the LSTM and Linear layers using the contextual semantic feature F;

[0166] Step 2.2.4, obtaining entity location information and event location information in the question sentence according to BIO-mark;

[0167] Step 2.2.5, according to the entity location information and event location information, F = (c 1 ,c 2 ,…,c t ) The semantic feature of the corresponding position is taken as the semantic feature h of the entity;

[0168] Step 2.2.6, linking the semantic features of the entity to the existing entities in the graph through the entity linking module, thereby obtaining the entity and the entity location.

[0169] It should be noted that the entity extraction model uses methods such as data enhancement and data synthesis to improve the entity recognition performance of diverse expressions.

[0170] Furthermore, the RoBERTa-large language model is a large pre-trained language model based on the Transformer architecture, which learns contextual information in text and extracts semantic features through masked language modeling tasks.

[0171] Furthermore, the BIO-mark in step 2.2.4 is used to mark entities or events in the text. The marking method has five types of markings, including entity start token, entity internal token, event entity start token, event entity internal token, and other type token. Each token in the text is marked as one of the types. For example, in the BIO-mark question, the first token is the entity start token, the second and third tokens are entity internal tokens, and the fourth token is other type token. Then the semantic feature of the entity is h = (c 1 ,c 2 ,c 3 ). By using BIO-mark, entities or events can be accurately identified and located in the text and used for training or evaluating models in subsequent processing.

[0172] Furthermore, the entity linking module in step 2.2.6 is composed of a bidirectional LSTM layer and a Linear layer, and the calculation formula is as follows:

[0173] h=(c 1 ,c 2 ,c 3 )

[0174] Q=LSTM(h)=LSTM(c 1 ,c 2 ,c 3 )

[0175] P=Linear(Q)

[0176] The LSTM module is used to aggregate entity semantic information features h of different lengths into an aggregate feature Q. The Linear layer maps the aggregate feature Q to a probability vector P representing the position number id of each entity, and takes the position number with the maximum value of the probability vector as its entity index.

[0177] Execute step 3, use the obtained entities, relationships, event entities and time to perform graph query to obtain the final candidate answer set. For each question, the four types of information, namely, entity, relationship, event entity and time, are obtained, and a graph query is performed based on these four key query information to obtain a candidate answer set containing entity location or time location. The present invention designs the following two query routes:

[0178] Step 3.1, entity query;

[0179] Step 3.2, time query.

[0180] Specifically, the entity query in step 3.1 includes the following steps (such as Figure 7 ):

[0181] Step 3.1.1, based on the entities and relations extracted from the question, query all quadruple groups that meet the (entity, relation, *, *) format in the graph, where the first * represents any entity and the second * represents any time. All quadruple groups that meet the conditions are called initial candidate quadruple groups.

[0182] Step 3.1.2, determine the time constraint type of the question, including three types: specific time point, specific time period, and event time period, filter the initial candidate quadruple according to different question types, and obtain filtered candidate quadruple;

[0183] Step 3.1.3, sort the filtered candidate quadruplets in time;

[0184] Step 3.1.4, output the entity locations of the first ten sorted filtered candidate quadruplets as the candidate answer set.

[0185] It should be noted that in step 3.1.2, the method for determining the time constraint type of the question is:

[0186] First, define the time constraint words as before / after. If the time constraint words are followed by entities, it is an event time period type problem. If the time constraint words are followed by specific time, it is a specific time period type problem. There is no time constraint word for a specific time point type problem.

[0187] Furthermore, in step 3.1.2, when the time constraint type of the question is an event time period question, the time constraint words are derived from the four-tuple event in the question, and the filtering steps are as follows:

[0188] Step 3.1.2.1, according to the entities, event entities and relations extracted from the question, search all quadruple groups that meet the conditions of (entity, relation, event entity, *) in the graph (* represents any time), and obtain the initial candidate quadruple group;

[0189] Step 3.1.2.2, sort the times in all initial candidate quadruple groups that meet the query conditions;

[0190] Step 3.1.2.3, determine the specific time constraint according to the type of time constraint words in the question.

[0191] When the time constraint word in the question is a forward constraint word, the earliest time in the time sequence is extracted as the time constraint;

[0192] When the time constraint word in the question is a backward constraint word, the latest time in the time sequence is extracted as the time constraint;

[0193] Step 3.1.2.4, forward filtering or backward filtering the initial candidate quadruple using the determined time constraint to obtain a filtered candidate quadruple;

[0194] The forward filtering is to filter out the initial candidate quadruple occurring after the extraction time, and the backward filtering is to filter out the initial candidate quadruple occurring before the extraction time;

[0195] Furthermore, in the step 3.1.2, when the time constraint type of the question is a specific time period type question, it is necessary to obtain the time constraint based on the time constraint words in the question for filtering the initial candidate quadruple; if the time constraint words are forward constraint words, the initial candidate quadruple occurring after the extraction time will be filtered out; if the time constraint words are backward constraint words, the initial candidate quadruple occurring before the extraction time will be filtered out to obtain filtered candidate quadruple.

[0196] Furthermore, the forward constraint words are words such as before and in front of that push the time forward; the backward constraint words are words such as after that push the time backward.

[0197] Furthermore, in step 3.1.2, when the time constraint type of the question is a specific time point type question, the time is explicitly present, and the initial candidate quadruple is filtered according to the extracted time, and the initial candidate quadruple occurring at the extraction time is retained to obtain the filtered candidate quadruple.

[0198] Further, the method of step 3.1.3 is as follows:

[0199] Determine whether there is a first / last order constraint in the question. If there is a first, sort the filter candidate quadruple in forward order by time. If there is a last, sort the filter candidate quadruple in reverse order. If there is no order constraint, do not sort the filter candidate quadruple in time.

[0200] Specifically, when querying the time in step 3.2, the specific method includes the following steps (such as Figure 8 ):

[0201] Step 3.2.1, based on the entities and relations extracted from the question, search the graph for all quadruple groups that meet the form of (entity 1, relation, entity 2, *) (* represents any time), and call all quadruple groups that meet the conditions the initial candidate quadruple groups;

[0202] Step 3.2.2, determine whether there is a first / last order constraint in the question sentence, sort the initial candidate quadruple in forward order by time if there is a first, sort the initial candidate quadruple in reverse order if there is a last, and do not sort the initial candidate quadruple in time if there is no order constraint, and obtain the sorted candidate quadruple;

[0203] Step 3.2.3, according to the granularity required in the question, for example, if "which year" or "what year" appears in the question, it is marked as year granularity; if "which month" or "what month" appears in the question, it is year-month granularity; if it is other cases, it is year-month-day granularity, and the time information in the sorting candidate quadruplets is corrected for granularity;

[0204] Step 3.2.4, output the time positions of the first ten sorted candidate quadruplets after granularity correction as the candidate answer set.

[0205] Furthermore, the particle size correction process in step 3.2.3 is specifically performed as follows:

[0206] First, the question is marked with granularity. The year in the question is marked as year granularity, the specific month in the question is marked as year-month granularity, and other cases are marked as year-month-day granularity. The time information of the quadruple defaults to year-month-day granularity. According to the granularity mark of the question, the time information of the sorted candidate quadruple is processed into the corresponding granular format.

[0207] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0208] The technical features of the above embodiments can be combined arbitrarily. At the same time, the number of each step is not used to constrain the order of the steps. As long as there is no strict order constraint between the steps, the order is allowed to be adjusted and changed. In order to make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0209] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A question-answering method for multi-granularity temporal knowledge graphs, It is characterized in that The following steps are involved: Step 1: preprocess the data set to obtain entities, relations, event entities and time; Step 2: Use the dataset to train the entity extraction model and the relationship extraction model to obtain more accurate entities and relationships; Step 3: Use the obtained entities, relations, event entities and time to perform graph query to obtain the final candidate answer set; The specific steps of step 1 are as follows: Step 1.1, divide the data set into training set, validation set and test set; Step 1.2: Perform time extraction, entity extraction, event extraction, and relationship extraction on each question in the training set, validation set, and test set; The specific steps of step 2 are as follows: Step 2.1, training the relationship extraction model; Step 2.2, training entity extraction model; In step 3, for each question, we have obtained the entity, relationship, event entity, and time information. We perform a graph query based on these four key query information to obtain a candidate answer set containing entity locations or time locations. We design the following two query routes: Step 3.1, entity query; Step 3.2, time query; In step 1.1, the training set data contains 350,000 question-answer pairs, and the validation set data contains 25,000 question-answer pairs, each of which contains three parameters: quid is the unique number of the question, question is the question text, and answers is the answer to the question, and one question text corresponds to one or more answers; the test set data contains 25,000 question-answer pairs, each of which contains only quid and question; In step 1.2, the following steps are performed on each question text in the training set, the validation set, and the test set: Step 1.2.1, time extraction; Step 1.2.2, entity extraction; Step 1.2.3, event entity extraction; Step 1.2.4, relation extraction; In step 1.2.1, time is extracted by using a regular expression matching method, and the regular expression matching rules are as follows: ； The time formats that can be matched include pure numbers, month information in English or English abbreviations, date information in ordinal numbers, and separated month and year information; The specific method of step 1.2.2 is as follows: Step 1.2.2.1, use the NLTK library to normalize each question in the dataset, including lemma restoration, case conversion, Anglo-American word conversion, and stop word removal; Step 1.2.2.2, create a string matching template for each entity in the knowledge graph, for a vocabulary of Entity, rearrange the words in the entity, generate Matching templates; Step 1.2.2.3, using a matching algorithm to calculate the matching degree between the normalized question and each matching template and obtain the entities and entity positions in the question; The specific steps of step 1.2.3 are as follows: Step 1.2.3.1, normalize the question sentence; Step 1.2.3.2, check whether before / after, first / last event prompt words appear in the question sentence. If they do, mark their positions. If they do not, it means that the question sentence does not have event time constraints. Step 1.2.3.3, determine the positional relationship between the event prompt word and the entity. If there is an entity within 10 characters after the event prompt word, mark the entity as an event entity. The step 1.2.4 is a relationship extraction method based on the bag-of-words model, which extracts the relationship with the highest similarity score to the question as the relationship of the question. The specific steps are as follows: Step 1.2.4.1, using the extracted entities and the answers of the training set, obtain a list of all quadruple pairs in the graph that satisfy (entity, *, answer, *) and (answer, *, entity, *), and group the relations that appear in the list into a candidate relation set; Step 1.2.4.2, standardize all relations in the knowledge graph to initialize the bag-of-words model; In step 1.2.4.3, the bag-of-words model is used to calculate the similarity scores between the relation and the question, and the relation with the highest score is taken as the relation of the question.

2. A question-answering method for multi-granularity temporal knowledge graph according to claim 1, It is characterized in that The specific steps of step 2.1 are as follows: Step 2.1.1, input natural language question , Represented as words, the natural language question is segmented and embedded to obtain an embedding vector , For the The element value at position Step 2.1.2: Use the language model RoBERTa-large to extract contextual semantic features from the embedding vector , is a characteristic element; Step 2.1.3: transform the contextual semantic features Send it to the Linear layer for classification to obtain the relationship categories contained in the question; The specific steps of step 2.2 are as follows: Step 2.2.1, input natural language question , Represented as words, the natural language question is segmented and embedded to obtain an embedding vector , For the The element value at position Step 2.2.2: Use the language model RoBERTa-large to extract contextual semantic features from the embedding vector , is a characteristic element; Step 2.2.3, the contextual semantic features Extract the BIO-mark of the question through LSTM and Linear layers; Step 2.2.4, obtaining entity location information and event location information in the question sentence according to BIO-mark; Step 2.2.5, according to the entity location information and event location information The semantic features of the corresponding positions are used as entity semantic features ; Step 2.2.6: transform the entity semantic features Through the entity linking module, we link to the existing entities in the graph to obtain the entities and their locations.

3. A question-answering method for multi-granularity temporal knowledge graph according to claim 2, It is characterized in that The RoBERTa-large language model is a large pre-trained language model based on the Transformer architecture, which learns contextual information in text and extracts semantic features through masked language modeling tasks.

4. A question-answering method for multi-granularity temporal knowledge graph according to claim 2, It is characterized in that The BIO-mark in step 2.2.4 is used to mark entities or events in the text. This marking method has five types of markings, including entity start token, entity internal token, event entity start token, event entity internal token, and other types of tokens. Each token in the text is marked as one of the above five types. By using BIO-mark, entities or events can be accurately identified and located in the text, and used for training or evaluating models in subsequent processing; The entity link module in step 2.2.6 is composed of a bidirectional LSTM module and a Linear layer, and the calculation formula is as follows: ； The LSTM module is used to transform entity semantic information features of varying lengths Aggregate into an aggregate feature In the linear layer, the aggregated features Mapped to the probability vector representing the position number id in the entity , take the position number of the maximum value of the probability vector as its entity index.

5. A question-answering method for multi-granularity temporal knowledge graph according to claim 1, It is characterized in that The specific method of step 3.1 comprises the following steps: Step 3.1.1, based on the entities and relations extracted from the question, query all quadruple groups that meet the (entity, relation, *, *) format in the graph, where the first * represents any entity and the second * represents any time. All quadruple groups that meet the conditions are called initial candidate quadruple groups. Step 3.1.2, determining the time constraint type of the question, including three types: specific time point, specific time period, and event time period; filtering the initial candidate quadruple according to different question types and obtaining filtered candidate quadruple; Step 3.1.3, sorting the filtered candidate quadruplets in time; Step 3.1.4, outputting the entity positions of the first ten sorted filtered candidate quadruplets as a candidate answer set; The specific method of step 3.2 comprises the following steps: Step 3.2.1, based on the entities and relations extracted from the question, search the graph for all quadruple groups that meet the form of (entity 1, relation, entity 2, *), where * represents any time, and obtain all the initial candidate quadruple groups that meet the conditions; Step 3.2.2, determine whether there is a first / last order constraint in the question sentence, and sort the initial candidate quadruple in forward order by time if there is a first; sort the initial candidate quadruple in reverse order if there is a last; and do not sort the initial candidate quadruple in time if there is no order constraint, and finally obtain sorted candidate quadruple; Step 3.2.3, according to the granularity required in the question, the time information in the sorted candidate quadruple is subjected to granularity correction processing; Step 3.2.4, output the time positions of the first ten sorted candidate quadruplets after granularity correction as the candidate answer set.

6. A question-answering method for multi-granularity temporal knowledge graph according to claim 5, It is characterized in that In step 3.1.2, the time constraint type of the question is determined by first defining the time constraint words as before / after. If the time constraint words are followed by entities, the question is an event time period type question. If the time constraint words are followed by specific times, the question is a specific time period type question. If there is no time constraint word, the question is a specific time point type question. When the time constraint type of the question is an event time period type question, the time constraint words are derived from the initial candidate quadruple in the question, and the filtering steps are as follows: Step 3.1.2.1, based on the entities, event entities and relations extracted from the question, search in the graph for all quadruple groups that meet the conditions of (entity, relation, event entity, *), where * represents any time, and obtain the initial candidate quadruple group; Step 3.1.2.2, sort the times in all initial candidate quadruple groups that meet the query conditions; Step 3.1.2.3, determine the specific time constraint according to the type of the time constraint word in the question sentence. When the time constraint word in the question sentence is a forward constraint word, extract the earliest time in the time sequence as the time constraint; when the time constraint word in the question sentence is a backward constraint word, extract the latest time in the time sequence as the time constraint; Step 3.1.2.4, forward filtering or backward filtering the initial candidate quadruple using the determined time constraint to obtain a filtered candidate quadruple; The forward filtering is to filter out the initial candidate quadruple occurring after the extraction time, and the backward filtering is to filter out the initial candidate quadruple occurring before the extraction time; When the time constraint type of the question is a specific time period type question, it is necessary to obtain the time constraint based on the time constraint words in the question to filter the initial candidate quadruple; if the time constraint words are forward constraint words, the initial candidate quadruple occurring after the extraction time will be filtered out; if the time constraint words are backward constraint words, the initial candidate quadruple occurring before the extraction time will be filtered out to obtain the filtered candidate quadruple; The forward constraint words are before and in front of, which are words that push the time forward; the backward constraint words are after, which are words that push the time backward; When the time constraint type of the question is a specific time point type question, the time is explicitly present, and the initial candidate quadruple is filtered according to the extracted time, and the initial candidate quadruple occurring at the extracted time is retained to obtain the filtered candidate quadruple; The method of step 3.1.3 is as follows: Determine whether there is a first / last order constraint in the question sentence. If there is a first, the filter candidate quadruple is sorted in forward order by time. If there is a last, the filter candidate quadruple is sorted in reverse order. If there is no order constraint, the filter candidate quadruple is not sorted in time.

7. A question-answering method for multi-granularity temporal knowledge graph according to claim 5, It is characterized in that The specific method of the particle size correction process in step 3.2.3 is as follows: First, the question is marked with granularity. The year in the question is marked as year granularity, the specific month in the question is marked as year-month granularity, and other cases are marked as year-month-day granularity. The time information of the quadruple defaults to year-month-day granularity. According to the granularity mark of the question, the time information of the sorted candidate quadruple is processed into the corresponding granular format.

Images