Joint event detection and extraction method and device based on end-to-end machine reading comprehension

Abstract

The application discloses a joint event detection and extraction method and device based on end-to-end machine reading comprehension, and belongs to the technical field of information extraction and natural language processing. The joint event detection and extraction method based on end-to-end machine reading comprehension comprises the following steps: S1, data preprocessing, analyzing a data set, traversing all argument roles contained in the data set to form an argument role set; different question templates are designed for different arguments; S2, generation of question and answer pairs, converting event extraction into machine reading comprehension, and generating questions according to the templates designed in the step S1; S3, inputting a given sentence and generated questions into a model of joint event detection and extraction based on end-to-end machine reading comprehension; and S4, heuristic post-processing. The application solves the technical problem that the existing event extraction method based on machine reading comprehension is mainly pipeline type, is prone to error transmission, and cannot effectively utilize the correlation between event types and arguments.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of information extraction and natural language processing, in particular to a joint event detection and extraction method and device based on end-to-end machine reading comprehension. BACKGROUND

[0002] Event extraction, as an important branch of information extraction tasks, includes two sub-tasks, namely event detection and corresponding argument extraction. In the past, event extraction was often defined as a classification task, which required full supervision, making it difficult to migrate to specific domains lacking high-quality labeled data.

[0003] To address the above problems, machine reading comprehension can be used to solve the event extraction task. Machine reading comprehension allows machines to answer questions based on given articles. The release of large-scale data sets and the development of deep neural networks have led to rapid development of machine reading comprehension technology in recent years. Event extraction based on machine reading comprehension can effectively utilize existing achievements in the field of machine reading comprehension. At the same time, for event extraction in low-resource scenarios, the use of machine reading comprehension models trained on labeled data sets to solve zero-shot or small-sample event extraction tasks has also been explored.

[0004] Although event extraction based on machine reading comprehension performs well, they treat event detection and argument extraction as two independent tasks, such as two independent question and answer processes, or use text entailment tasks to solve event detection and machine reading comprehension to solve argument extraction. This pipeline approach not only leads to error propagation, but also ignores the relationship between the two sub-tasks.

[0005] In related technologies, such as Chinese patent document (CN115114413A), a joint event extraction method and device based on reading comprehension are described, which extracts event types and event elements simultaneously. Compared to the two-stage event extraction method, although there is no cumulative error or error propagation, in the application, special symbols [AND] are used to concatenate event types and event elements as questions, and experts with domain knowledge are required to define the correspondence between event types and event elements. In the application, the machine reading comprehension model can only rely on determining whether the question for a specific event type has an answer to perform event detection.

[0006] In view of the problems in the related art, no effective solution has been proposed so far. SUMMARY

[0007] The main purpose of the present application is to provide an end-to-end machine reading comprehension based joint event detection and extraction method to solve the problem that the existing machine reading comprehension based event extraction method is mainly pipeline type, which is easy to cause error propagation, and cannot effectively utilize the association relationship between event types and arguments.

[0008] In order to achieve the above purpose, according to one aspect of the present application, an end-to-end machine reading comprehension based joint event detection and extraction method is provided.

[0009] The end-to-end machine reading comprehension based joint event detection and extraction method according to the present application comprises:

[0010] S1: data preprocessing, analyzing the data set, traversing all argument roles contained therein to form an argument role set; different question templates are designed for different arguments;

[0011] S2: generation of question and answer pairs, converting event extraction into machine reading comprehension, generating questions according to the templates designed in step S1;

[0012] S3: input the given sentence and the generated question into the model of the end-to-end machine reading comprehension based joint event detection and extraction; the model comprises four parts: encoding, event classifier, machine reader and joint training;

[0013] S4: heuristic post-processing.

[0014] Further, the step S2 further comprises traversing and asking all arguments in the data set, specifically: after a sentence is given, first determine whether it describes an event; if it describes an event, traverse and ask all arguments in the data set, generate questions according to the question templates; if the argument is mentioned in the given sentence, mark the argument as the answer to the question, otherwise the answer is empty; if the sentence does not describe an event, randomly select a number of pre-defined arguments to generate questions, and the answers to these questions are marked as a list of empty values.

[0015] Further, the step S3 specifically comprises the following steps:

[0016] S3.1: concatenate the sentence and the question input into the model, and perform embedded encoding;

[0017] S3.2: predict the event type of the given sentence through the event classifier;

[0018] S3.3: in the machine reader, calculate the association relationship between the event type and the argument role by using the attention mechanism from coarse granularity to fine granularity, predict the answer based on the calculation result of the association relationship, and then convert the answer into the extracted argument;

[0019] S3.4: Joint training of state-aware, i.e. training the event classifier and the machine reader jointly.

[0020] Further, in the step S3.1, the encoding refers to embedding encoding of mapping words in a high-dimensional discrete space to vectors in a low-dimensional continuous space, and the step S3.1 specifically comprises:

[0021] Given a question Q and a sentence S, spliced with special symbols [CLS] and [SEP], the input form is [CLS]Q[SEP]S[SEP];

[0022] The sum of the word vector, segment vector and position vector of each word in the sequence [CLS]Q[SEP]S[SEP] is input into the Transformer encoding block of BERT to obtain the semantic representation:

[0023]

[0024] Wherein, t refers to the number of hidden layers, and T refers to the number of Transformer encoding blocks.

[0025] Further, the step S3.2 specifically comprises:

[0026] The output of the encoder is input into the max-pooling layer and the dropout layer in turn, and then a linear classifier and a Softmax layer are used to calculate the probability of event type :

[0027]

[0028] Wherein, and are trainable parameters, is the output result of the linear layer, and e is the number of event types;

[0029] The negative logarithmic probability is used as the loss function, wherein θ represents the parameter set in the event classification module:

[0030]

[0031] Wherein, N is the number of training samples, is the labeled event type, y i is the predicted event type.

[0032] Further, the step S3.3 specifically comprises the following steps:

[0033] S3.3.1: Use the attention mechanism from coarse to fine to extract the association between event types and argument roles;

[0034] where the coarse-grained attention mechanism acts on the results outputted by the event classifier in step S3.2 to enable the model to focus on specific event types, the coarse-grained attention mechanism is implemented through a linear layer, a i As the output of the event classifier, the coarse-grained attention is calculated as follows:

[0035] a k = W c ·a i +b c

[0036] where W c and b c are trainable parameters;

[0037] Then, the output results a k of the coarse-grained attention and the BERT encoded semantic representation h t are input into the fine-grained attention mechanism to calculate the similarity between the event type and each word vector in the input sentence:

[0038]

[0039]

[0040] where W f is a trainable parameter, a j is a weight parameter calculated by the attention mechanism, M is the number of words in the sentence, and s is an activation function, is the word vector of the jth word encoded by the BERT encoder.

[0041] Further, the step S3.3 further comprises the following steps:

[0042] S3.3.2: After converting the event extraction into a machine reading comprehension task, the arguments in the sentence are extracted by answering questions, and the answer prediction method of the machine reading comprehension model is to predict the probability of the start and end positions of the answer, where the Softmax is used to calculate the final output, and the cross-entropy loss is selected as the loss function of the machine reader, and the loss calculation of the machine reading comprehension is as follows:

[0043]

[0044] where N is the number of training samples, and represent the correct start and end positions of the sample, represents the probability of the start position, represents the probability of the end position.

[0045] Further, in step S3.4, state-aware weights are respectively set in the event classifier and the machine reader, as follows:

[0046] When calculating the event classification loss, different weights are given to the following three cases in the data set: the weight is 1 when the sentence does not describe an event; the weight is 2 when the sentence describes an event but there is no answer to the question corresponding to the argument; and the weight is 3 when the sentence describes an event and there is an answer to the question corresponding to the argument;

[0047] When calculating the loss of the machine reader, the weight parameter of the cross-entropy loss function is set to

[0048] Let L e represent the loss of event classification, L mrc represent the loss of machine reading comprehension, then the joint loss is calculated as follows:

[0049] L t o tal =L e +γL mrc

[0050] Wherein, γ is used to control the weight of the machine reading comprehension loss.

[0051] In order to achieve the above purpose, according to another aspect of the present application, a joint event detection and extraction device based on end-to-end machine reading comprehension is provided.

[0052] The joint event detection and extraction device based on end-to-end machine reading comprehension according to the present application comprises:

[0053] A data preprocessing module for analyzing a data set, traversing all argument roles contained therein, forming an argument role set, and designing different question templates for different arguments;

[0054] A question and answer pair generation module for converting time extraction into machine reading comprehension and generating questions according to the designed question templates;

[0055] An input module for inputting the given sentence and the generated question into the model of the joint event detection and extraction based on end-to-end machine reading comprehension;

[0056] An analysis module for heuristic post-processing.

[0057] Further, the model of the joint event detection and extraction based on end-to-end machine reading comprehension comprises:

[0058] An encoder for concatenating and embedding the sentence and the question input into the model;

[0059] Event classifier: predict the event type of a given sentence;

[0060] Machine reader: calculate the association between event types and argument roles using attention mechanisms from coarse to fine-grained, predict the answer based on the calculation results of the association, and then convert the answer into extracted arguments;

[0061] Joint training module: jointly train the event classifier and the machine reader.

[0062] The present application has the advantages: a joint event detection and extraction method based on end-to-end machine reading comprehension, which introduces a BERT-based machine reading comprehension model. The model consists of two main modules, namely event classifier and machine reader with coarse-to-fine attention mechanism, which can recognize event types and extract arguments simultaneously through question and answer without labeling trigger words. Not only can it reduce error propagation, but also can take advantage of the characteristics of machine reading comprehension model to achieve good performance in data-scarce scenarios. BRIEF DESCRIPTION OF DRAWINGS

[0063] The accompanying drawings, which form a part of this application, are intended to provide further understanding of the application and are incorporated herein in their entirety. The schematic embodiment drawings of the present application and their descriptions serve to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0064] Figure 1 is a flowchart according to an embodiment of the present application;

[0065] Figure 2 is an example diagram of a problem generation algorithm according to an embodiment of the present application;

[0066] Figure 3 is an example diagram of a problem generation process according to an embodiment of the present application;

[0067] Figure 4 is a joint event detection and extraction model based on end-to-end machine reading comprehension according to an embodiment of the present application. DETAILED DESCRIPTION

[0068] In order to better understand the present application by those skilled in the art, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor should be within the scope of protection of the present application.

[0069] It should be noted that the terms "comprising" and "having" and any variations thereof in the present application are intended to cover a non-exclusive inclusion, for example, a process, method, system, product or apparatus that includes a list of steps or units not necessarily limited to those explicitly listed, but can include other steps or units not expressly listed or inherent to such processes, methods, products or apparatus.

[0070] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict.

[0071] As shown in Figure 1 The method comprises the following steps S1 to S4:

[0072] S1: First, data preprocessing is performed, the data set is analyzed, all argument roles contained therein are traversed, and an argument role set is formed.

[0073] In an embodiment of the present application, ACE 2005 data set is used, which is widely used for evaluation of event extraction task, and its data sources include broadcast dialogue, broadcast news, blogs, etc. Event mention, trigger word, argument and event coreference are carefully manually annotated. ACE 2005 includes 8 event categories (life, conflict, movement, justice, personnel, transaction, company and communication) 33 event subtypes, 35 argument roles. In the present fact example, in order to avoid confusion of the joint event detection and extraction model based on end-to-end machine reading comprehension, "time after", "time before", "time-within", "time-at-beginning" and "time-at-end" are combined to represent "time", so the argument role set constructed includes 31 argument roles. Different question templates are designed for different arguments, as shown in Table 1. The interrogative word changes according to different argument types, for example, "When" is used to ask for time, and "Who" is used to ask for victim.

[0074] Table 1: Examples of question templates for different types of arguments

[0075]

[0076] S2: Convert event extraction to machine reading comprehension, and the generation of question and answer pairs is a crucial step. Questions are generated according to the templates designed in step S1. By asking questions, the prior knowledge of argument roles can be introduced into the machine reading comprehension model as clues to improve the effect of event extraction.

[0077] All arguments in the data set are traversed and asked. As shown in Figure 2As shown in the algorithm, given a sentence, first determine whether it describes an event. If it describes an event, iterate through all the arguments in the dataset to ask questions according to the question templates. If the given sentence contains the argument, mark the argument mention as the answer to the question, otherwise the answer is empty. If the sentence does not describe an event, randomly select a predefined number of arguments to generate questions, and the answers to these questions are marked as a list of empty. In one specific example of the present application, as shown in Figure 3 the input sentence is "Banks was convicted of shooting and killing a 16-year-old at a park in 1980.", which describes a conflict attack event (Conflict.Attack), where Banks is the attacker, a 16-year-old is the target, a park is the location, and 1980 is the time. For the input sentence, 31 arguments in the argument role set constructed in S1 are iterated to ask questions, the question about the attacker "Who is the attacker?" has the answer "Banks", the question about the location "Where did the event take place?" has the answer "a park", the question about the target "Who is the target?" has the answer "a 16-year-old", the question about the time "When did the event take place?" has the answer "1980", and the questions about other argument roles have empty answer lists. In another specific example of the present application, the input sentence is "He was a career prosecutor.", which does not describe an event, so 4 arguments are randomly selected from the argument role set constructed in S1 to ask questions according to the question templates constructed in S1, and the answer lists of the questions are all empty.

[0078] S3: Input the given sentence and the generated question into the joint event detection and extraction model based on end-to-end machine reading comprehension. The model includes four parts: encoding, event classifier, machine reader, and joint training.

[0079] S3.1: Encoding refers to embedding coding of mapping words in a high-dimensional discrete space to low-dimensional continuous space vectors. In one embodiment of the present application, the process of encoding is to splice the input sentence and the question, and to perform embedding coding.

[0080] Specifically, given a question Q and a sentence S, the input form is [CLS]Q[SEP]S[SEP] by concatenating with special symbols [CLS] and [SEP];

[0081] The sum of the word vector, segment vector and position vector of each word in the sequence [CLS]Q[SEP]S[SEP] is input into the Transformer encoding block of BERT as an embedding code to obtain a semantic representation:

[0082]

[0083] Wherein, t refers to the number of hidden layers, and T refers to the number of Transformer encoding blocks.

[0084] It should be noted that in the embodiments of the present application, the automatic question answering model based on the pre-trained language model can use the BERT model published by Google for training on Chinese, that is, the pre-trained language model is the BERT model.

[0085] S3.2: The purpose of the event classifier is to predict the event type of a given sentence (None when the sentence is irrelevant to the event). In order to achieve this goal, the output of the BERT encoder in S3.1 is sequentially input into the max-pooling layer and the dropout layer, and then a linear classifier and a Softmax layer are used to calculate the probability of the event type:

[0086]

[0087] Wherein, and are trainable parameters (trainable parameters, i.e. the weights in the linear layer of the neural network, corresponding to the attributes of the kernel and the bias), is the output result of the linear layer, and e is the number of event types;

[0088] The negative logarithmic probability is used as the loss function, where θ represents the parameter set in the event classification module:

[0089]

[0090] Wherein, N is the number of training samples, is the labeled event type, y i is the predicted event type.

[0091] S3.3: In the machine reader, first, the attention mechanism from coarse to fine calculates the association between the event type and the argument role. Based on this result, the module predicts the answer, and then converts the answer into the extracted argument.

[0092] S3.3.1: In the event extraction task, the association between event type and argument role plays a crucial role. Event type will provide clues for identifying arguments, and in turn, event arguments can also assist event classification. In the joint event detection and extraction model based on end-to-end machine reading comprehension, a coarse-to-fine attention mechanism is used to extract the above-mentioned association. The coarse-grained attention mechanism acts on the results output by the event classifier in S3.2, allowing the model to focus on a specific event type. The coarse-grained attention mechanism is implemented through a linear layer, a i As the output of the event classifier, the coarse-grained attention is calculated as follows:

[0093] a k =W c ·a i +b c

[0094] where W c and b c are trainable parameters;

[0095] Next, the output of the coarse-grained attention a k and the BERT encoded semantic representation h t are input into the fine-grained attention mechanism to calculate the similarity between the event type and each word vector in the input sentence:

[0096]

[0097]

[0098] where W f is a trainable parameter, a j is a weight parameter calculated by the attention mechanism, M is the number of words in the sentence, and a is an activation function, is the word vector of the jth word encoded by the BERT encoder.

[0099] S3.3.2: After converting event extraction into a machine reading comprehension task, arguments in sentences are extracted by answering questions. The most common answer prediction method for machine reading comprehension is to predict the probability of the start and end positions of the answer. The answer predictor in this application uses Softmax to calculate the final output. Cross-entropy loss is selected as the loss function for machine reading. The loss calculation for machine reading comprehension is as follows:

[0100]

[0101] where N is the number of training samples, and represent the correct start and end positions of the sample, respectively, probabilities of start positions, probabilities of end positions.

[0102] S3.4: State-aware joint training. To avoid error propagation and effectively utilize the association between event types and argument roles, the event classifier and the machine reader are jointly trained.

[0103] Since all arguments in the argument role set are iteratively questioned in the S2 step, and only the arguments that appear in the given sentence have answers, this leads to the number of negative examples without answers in the training sample being much larger than that of positive examples with answers. In order to balance the distribution of training samples, state-aware weights are set in the event classifier and the machine reader respectively.

[0104] When calculating the event classification loss, different weights are given to the following three cases in the data set: when the sentence does not describe an event, the weight is 1; when the sentence describes an event, but the question corresponding to the argument has no answer, the weight is 2; when the sentence describes an event and the question corresponding to the argument has an answer, the weight is 3. The state-aware weight can make the joint event detection and extraction model based on end-to-end machine reading comprehension pay more attention to positive examples with answers.

[0105] When calculating the loss of the machine reader, the weight parameter of the cross-entropy loss function is set to (β mrc , which can take any value other than 0, to reduce the probability of the joint event detection and extraction model based on end-to-end machine reading comprehension predicting that the question has no answer.

[0106] Let L e represent the loss of event classification, L mrc represent the loss of machine reading comprehension, then the joint loss is calculated as follows:

[0107] L total = L e + γL mrc

[0108] where γ is used to control the weight of the machine reading comprehension loss.

[0109] S4: Heuristic post-processing.

[0110] Specifically, by analyzing the event extraction dataset, it can be found that the event parameters are divided into two categories, event type irrelevant and event type relevant. The event type irrelevant parameters usually play the role of place and time, while the event type relevant arguments are constrained by the event type, that is, some parameters only appear in the event sentence of a specific type. As an example in Table 2, in an embodiment of the present application, the event arguments attacker, victim and tool are more likely to occur in the conflict event. If the event type is movement, it contains arguments such as starting point, destination and vehicle.

[0111] Table 2: Event type and its corresponding argument example

[0112]

[0113] From the above description, it can be seen that the present application achieves the following technical effects: compared with the prior art, the present application does not require the corresponding relationship between the event type and the event element when constructing the question and answer pair, all the arguments in the dataset are traversed and asked, and the asking method is more in line with human language habits.

[0114] According to an embodiment of the present application, a device for implementing the above-mentioned joint event detection and extraction method based on end-to-end machine reading comprehension is also provided, which comprises: a data preprocessing module for analyzing the dataset, traversing all argument roles contained therein, forming an argument role set, and designing different question templates for different arguments; a question and answer pair generation module for converting time extraction into machine reading comprehension and generating questions according to the designed question templates; an input module for inputting the given sentence and the generated question into the model of joint event detection and extraction based on end-to-end machine reading comprehension; and an analysis module for heuristic post-processing. The machine reading comprehension model proposed in the present application comprises two modules, an event classifier and a machine reader with a coarse-to-fine attention mechanism, which realizes event detection and argument extraction by joint training of the two modules. Compared with the prior art, the model proposed in the present application can better utilize the association relationship between the event type and the event argument, achieving better event extraction effect.

[0115] Among them, the model of joint event detection and extraction based on end-to-end machine reading comprehension comprises: an encoder for splicing and embedding coding the sentence and question input into the model; an event classifier for predicting the event type of the given sentence; a machine reader for calculating the association relationship between the event type and the argument role by using the coarse-to-fine attention mechanism, predicting the answer based on the calculation result of the association relationship, and then converting the answer into the extracted argument; and a joint training module for jointly training the event classifier and the machine reader.

[0116] Specifically, the specific process in which each module in the device of the embodiment of the application implements its function can be referred to the related description in the method embodiment, which will not be described here again.

[0117] In one specific application example of the application:

[0118] The model parameters involved in the application are set according to Table 3:

[0119] Table 3 Parameter setting

[0120] Parameter Name Description Value Train_batch_size Training batch size 16 Learning_rate Initial learning rate for Adam 3e-5 Num_train_epochs Number of training iterations 5 Max_seq_length Maximum length of input sequence 384 Doc_stride Stride between document blocks 128 Logging_Steps Logging update step number 3000 Num_Event Number of event types 34 Max_query_length Maximum question length 64 Max_answer_length Maximum answer length 64

[0121] In order to prove the superiority of the model in the application, the model results in the application are compared with existing models, and Table 4 lists the evaluation results of different models under the full supervision condition of the ACE 2005 data set, including event classification, argument recognition and argument classification three tasks, and the evaluation indexes include accuracy, recall and F1 value, and the last column of Table 4 is the average value of F1 value of the three tasks.

[0122] Table 4 Full supervision experiment results of ACE 2005 data set

[0123]

[0124] As shown in Table 4, the first to fourth rows are the results of the event extraction model under the classification framework, the fifth to sixth rows are the results of the generative model, and the EEQA in the seventh row and the MQAEE in the eighth row define the event extraction model as a question answering task. Unlike EEQA and MQAEE, the application jointly trains event detection and argument extraction by using an end-to-end machine reading comprehension model. In the recall of the three tasks, the application is superior to other models. This is because the application iteratively asks questions about all argument roles in the data and is more likely to extract as many arguments as possible from the given sentence. Although the accuracy of the application is slightly lower than that of some models, the highest F1 score is obtained in the event classification and argument recognition tasks. The average score in the last column shows that by jointly training event classification and machine reading comprehension, the application performs more balanced in the three tasks.

[0125] The few-shot experiment results on the ACE 2005 data set are shown in Table 5. In order to simulate the data scarce scene, 1%, 5%, 10% and 20% of the samples in the data set are randomly selected as the training set. The third to sixth columns are the F1 scores of argument extraction of different training sets. The first to third rows are the results of the comparative models, the fourth and fifth rows are the results of the application, and the difference between the fourth and fifth rows is that the model in the fourth row uses BERT without additional data training for encoding, and the BERT in the fifth row is trained in advance on the SQuAD2.0 question answering data set.

[0126] Table 5. Results of few-shot experiments on ACE 2005 dataset

[0127]

[0128] The results in Table 5 show that the present application outperforms the CNN, RNN and BERT based event extraction methods in few-shot scenarios, for example, it achieves 41.0% F1 score on 20% training set, which is 12.4% higher than the BERT E. This improvement comes from the machine reading comprehension framework. By converting argument extraction to question answering, the present application can learn from the success cases in machine reading comprehension and reduce the dependence on labeled data. The results in the 4th and 5th rows show the effectiveness of transfer learning. In the scenario of extreme lack of data, directly using the BERT encoder trained on the additional data does not work well, for example, it only achieves 0.6% F1 on 5% training data and 1.7% F1 on 5% training data. In contrast, by pre-training the BERT encoder on the SQuAD2.0 dataset, it achieves 6.2% and 16.3% F1 on 1% and 5% training data, respectively. By training on the SQuAD2.0 dataset, the model can learn the question answering knowledge to assist event detection in data-scarce scenarios. This also proves that the effective transfer of knowledge learned by machine reading comprehension can improve the effect of event extraction in few-shot scenarios.

[0129] Obviously, those skilled in the art should understand that the modules or steps of the present application described above can be realized by general computing devices, which can be concentrated on a single computing device or distributed on a network composed of multiple computing devices, and can be realized by program codes executable by computing devices, so that they can be stored in storage devices and executed by computing devices, or they can be made into individual integrated circuit modules or a single integrated circuit module. Thus, the present application is not limited to any specific combination of hardware and software.

[0130] The above only describes the preferred embodiments of the present application and is not intended to limit the present application. Those skilled in the art can make various modifications and changes to the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A joint event detection and extraction method based on end-to-end machine reading comprehension, characterized in that: S1: data preprocessing, analyzing the data set, traversing all argument roles contained therein to form an argument role set; different question templates are designed for different arguments; S2: generation of question and answer pairs, converting event extraction into machine reading comprehension, and generating questions according to the templates designed in step S1; S3: inputting the given sentence and the generated question into the model of joint event detection and extraction based on end-to-end machine reading comprehension; the model includes four parts: encoding, event classifier, machine reader, and joint training; S4: heuristic post-processing; S3.1: concatenating the sentence and the question input into the model and performing embedded encoding; S3.2: predicting the event type of the given sentence through the event classifier; S3.3: in the machine reader, the association between the event type and the argument role is calculated using the attention mechanism from coarse to fine, the answer is predicted based on the calculation result of the association, and then the answer is converted into extracted arguments; S3.4: state-aware joint training, i.e., joint training of the event classifier and the machine reader; S3.3.1: using the attention mechanism from coarse to fine to extract the association between the event type and the argument role; S3.3.2: after converting event extraction into a machine reading comprehension task, the arguments in the sentence are extracted by answering the question, and the answer prediction method of the machine reading comprehension model is to predict the probability of the start and end positions of the answer, wherein the Softmax is used to calculate the final output, and the cross-entropy loss is selected as the loss function of the machine reader, and the loss calculation of the machine reading comprehension is as follows: S2 also includes traversing and asking questions for all arguments in the data set, specifically: After a sentence is given, first determine whether it describes an event; if it describes an event, traverse and ask questions for all arguments in the data set, and generate questions based on the question templates; wherein the coarse-grained attention mechanism acts on the result output by the event classifier in step S3.2 to enable the model to focus on a specific event type, the coarse-grained attention mechanism being implemented by a linear layer, The coarse-grained attention is computed as follows as output of the event classifier: wherein and are trainable parameters; Next, the output of the coarse-grained attention and the BERT-encoded semantic representation is input into the fine-grained attention mechanism, which computes the similarity of the event type and each word vector in the input sentence: wherein, is a trainable parameter, is a weight parameter computed by an attention mechanism, is the number of words in a sentence, is an activation function, is a word vector of the th word encoded by a BERT encoder If the argument is mentioned in the given sentence, the argument mention is marked as the answer to the question, otherwise the answer is empty; if the sentence does not describe an event, randomly select a pre-defined number of arguments to generate questions, and the answers to these questions are marked as a list of empty values. In step S3.1, encoding refers to embedding coding that maps words in a high-dimensional discrete space to vectors in a low-dimensional continuous space, and step S3.1 is specifically: wherein, N is the number of training samples, and represent the correct start and end positions of the sample, respectively, denotes the probability of the start position, denotes the probability of the end position.

2. The joint event detection and extraction method based on end-to-end machine reading comprehension according to claim 1, characterized in that: S3.2 is specifically: In step S3.4, state-aware weights are set in the event classifier and the machine reader, specifically as follows: When calculating the event classification loss, different weights are assigned to the following three cases in the data set: when the sentence does not describe an event, the weight is 1; when the sentence describes an event, but the question corresponding to the argument has no answer, the weight is 2; when the sentence describes an event and the question corresponding to the argument has an answer, the weight is 3; 3. The joint event detection and extraction method based on end-to-end machine reading comprehension according to claim 2, characterized in that: including: Given question Q and sentence S, concatenated with special tokens [CLS] and [SEP], input form is [CLS] Q [SEP] S [SEP]; The sum of the word vectors, segment vectors, and position vectors for each word in the sequence [CLS] Q [SEP] S [SEP] is input as an embedding encoding into the Transformer encoding blocks of BERT to obtain a semantic representation: wherein, refers to the number of hidden layers, T refers to the number of Transformer encoding blocks.

4. The joint event detection and extraction method based on end-to-end machine reading comprehension according to claim 3, characterized in that: ​ The output of the encoder is sequentially input into a max-pooling layer and a dropout layer, followed by a linear classifier and a Softmax layer to compute the probability of the event type : wherein, and are trainable parameters, is an output result of a linear layer, is a number of event types; Using the negative log probability as the loss function, where represents a set of parameters in the event classification module: wherein, N is the number of training samples, is the labeled event type, y i is the predicted event type.

5. The joint event detection and extraction method based on end-to-end machine reading comprehension according to claim 4, characterized in that: ​ ​ In computing the loss for the computer reader, the weight parameter of the cross-entropy loss function is set to ; With the loss representing event classification, the loss representing machine reading comprehension, the joint loss is calculated as follows: wherein, a weight for controlling the machine reading comprehension loss.

6. An apparatus for joint event detection and extraction based on end-to-end machine reading comprehension, configured to perform the method for joint event detection and extraction based on end-to-end machine reading comprehension according to any one of claims 1-5. ​ A data preprocessing module is configured to analyze a data set, traverse all argument roles contained in the data set, form an argument role set, and design different question templates for different arguments; A question and answer pair generation module is configured to convert time extraction into machine reading comprehension and generate questions according to the designed question templates; An input module is configured to input a given sentence and generated questions into a model of joint event detection and extraction based on end-to-end machine reading comprehension; An analysis module is configured to perform heuristic post-processing.

7. The joint event detection and extraction apparatus based on end-to-end machine reading comprehension according to claim 6, characterized in that: The model of joint event detection and extraction based on end-to-end machine reading comprehension comprises: An encoder configured to splice and perform embedded coding on a sentence and a question input into the model; An event classifier configured to predict an event type of a given sentence; A machine reader configured to calculate a correlation between the event type and an argument role by using an attention mechanism from coarse granularity to fine granularity, predict an answer based on a calculation result of the correlation, and then convert the answer into an extracted argument; A joint training module configured to jointly train the event classifier and the machine reader.

Images