A multi-domain oriented complex event element extraction method

Abstract

The application provides a multi-field-oriented complex event element extraction method, belongs to the technical field of information extraction, solves the problems of single application field of a traditional element extraction model and insufficient extraction capacity for complex events, and comprises a construction part of an element extraction model and an application part of the element extraction model; wherein the construction part forms an event element dataset according to an event element label system, iteratively trains a pre-trained language model using the event element dataset for an element extraction task, and obtains the element extraction model after the training is completed; the application part applies the trained element extraction model to multiple fields, performs event detection on multi-field text corpus, obtains event sentence data, performs element extraction work using the element extraction model, confirms the result of the element extraction work, and then carries out incremental optimization training of the element extraction model; and the application can realize cross-domain migration application of the model and element extraction of complex events.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of information extraction, and relates to a sequence labeling type event element extraction technology, in particular to a multi-field-oriented complex event element extraction method. BACKGROUND

[0002] Event extraction is an important but challenging task in information extraction. Events, as a special form of information, occur at a specific time and place, involve one or more participants, and can generally be described as a change of state. The event extraction task aims to extract such event information from unstructured text into a structured form, mainly describing when, where, why, how, and what happened in the real world, and who was involved. In terms of application, event extraction facilitates the retrieval of event information and the analysis of human behavior, and is often used as an upstream task for information retrieval, recommendation, intelligent question answering, knowledge graph construction and other applications.

[0003] Traditional event extraction methods generally use related field experts to manually write rules and specify template matching to achieve the extraction; with the explosive increase of network information, traditional methods have begun to fail to meet new demands, and new technologies such as statistical machine learning methods and deep learning models have emerged, greatly improving the effect of event extraction tasks.

[0004] In traditional event extraction methods, pattern matching-based event extraction methods generally require domain experts to manually construct rules and templates, which are usually matched in the form of dictionaries, regular expressions, and syntax trees. Typical event extraction expert systems (such as AutoSlog, PALKA) and subsequent systems that use partial statistical or learning methods to improve rules (such as CRYSTAL, AutoSlog-ST, etc.) are all based on this form to achieve extraction. Pattern matching-based methods usually include two steps of construction and extraction, i.e., rules are first discovered on the corpus, and then the rules are applied to new texts to be extracted for matching. Although the pattern matching-based method has been around for a long time and has many limitations, it has good interpretability and does not require a large amount of precision-annotated data, even in recent years there have been related researches to promote it, such as the GenPAM system. Compared with classic expert systems, these newer systems have the ability to automatically mine or generate corresponding patterns from general and domain corpora, to some extent, reducing the cost of manpower. However, in general, the accuracy of such methods is still limited.

[0005] Because the pattern matching based method usually needs a lot of human resources and is ineffective, especially when it needs to mine patterns again when migrating to new field data, the statistical machine learning based method gradually replaced the traditional pattern matching method in the late 20th century. Typical statistical machine learning methods include Maximum Entropy Model, Support Vector Machine, Conditional Random Field, etc. Generally speaking, the characteristics of such work are that the author will carefully select features (such as POS, bigram, etc.) according to the data set and the model, and regard the problem as a classification problem, for example, the event extraction method based on maximum entropy combines the features such as "demonstrative", POS, and verbs between two demonstratives, and sends them into the maximum entropy model for classification to obtain the event type. How to select or construct appropriate features, i.e. feature engineering, has a decisive influence on the effect of machine learning method; and statistical machine learning method usually needs large-scale fine-labeled corpus, and is easily affected by corpus class imbalance, long-tail data, etc.; and it is difficult to integrate external prior knowledge, so it is gradually replaced by deep learning technology in the wave of rapid development in recent years.

[0006] In recent years, with the joint development of computing power and data, deep learning has been widely studied and applied in natural language processing and other fields. The latest event extraction methods are mostly based on deep learning models. Deep learning based event extraction models are various, and more and more new methods are proposed with the development of deep learning models. For example, convolutional neural network (CNN) can be used to extract text features like TextCNN, and then sent to a classification model for classification or sequence labeling; or the chain network structure of long short term memory neural network (LSTM) can be used to model the context relationship of each word in the sentence to improve the effect. On the basis of traditional convolutional neural network model, a dynamic multi-pool mechanism is added to improve the experimental effect of event extraction task. This way divides the candidate trigger word and candidate entity into three parts according to the position, which can obtain more abstract and deep feature information than the maximum pool mechanism, and can help the model improve the recognition and classification of event arguments. Another Skip-gram convolutional neural network model based on traditional convolutional neural network can help the model obtain the feature information of non-continuous phrases, and thus improve the accuracy of event type discrimination. The limitation of deep learning based event element extraction method is that it depends on a large amount of training data, and the semantic information mining of text events is not sufficient from the global or local point of view.

[0007] In 2018, Google proposed the Transformer large-scale pre-training language model, which has a milestone significance in the field of natural language processing. The BERT pre-training model that appeared later has an absolute crushing advantage compared with RNN, CNN and other deep learning models in named entity recognition, text classification, machine question and answer and other tasks. Researchers also use pre-training language models to process event element extraction tasks, such as the two-stage event extraction model based on BERT proposed in 2019. First, the BERT sequence labeling method is used to classify each word in the sentence to obtain the possibility of each word as a trigger word. Then, each trigger word and the original sentence word are sent into the argument extraction model to perform binary classification on each word to obtain the probability of a single word as a specified trigger argument. In this way, the overlap problem of a word as multiple event arguments is solved. The pre-training language model uses the self-attention mechanism to better capture the semantic information of words, and due to the characteristics of the self-attention mechanism, long-distance semantic relationship mining can be realized. The pre-training language model is unsupervised trained by a large amount of text data, and the model itself has learned the deep semantic information of the text. When migrating to event extraction, text classification, entity recognition and other tasks, only a small amount of training corpus is needed for fine-tuning, which reduces the requirement of the model for a large amount of specific training data.

[0008] In summary, event extraction technology has made significant progress in both open and limited domains after years of research, but there are still the following problems:

[0009] 1. Since most event element extraction models are designed to solve problems in a specific field, the training corpus used by the model limits the application direction of the model, resulting in poor element extraction effect of the model in cross-domain event extraction.

[0010] 2. Most existing methods can well solve the element extraction and role definition of simple event sentences, but cannot effectively solve the element extraction of complex long event sentences, such as nested compound event sentences and parallel long event sentences with common elements. SUMMARY

[0011] In view of the poor model transfer ability, inaccurate element extraction of complex events and element extraction missing problems of the existing event element extraction method, the application discloses a multi-domain oriented complex event element extraction method, which realizes the cross-domain application of the model and the characteristics of accurate and comprehensive element extraction of complex events through the semantic feature mining ability of the pre-training model, the unique event element labeling method and the incremental training of the model after artificial correction.

[0012] The application adopts the following technical solutions to achieve the purpose:

[0013] A multi-field-oriented complex event element extraction method, comprising a construction part of an element extraction model and an application part of the element extraction model;

[0014] The construction part of the element extraction model forms an event element data set according to an event element label system, uses the event element data set to perform iterative training of a pre-trained language model on an element extraction task, and obtains the element extraction model after the training is completed;

[0015] The application part of the element extraction model applies the trained element extraction model to multiple fields, performs event detection on multi-field text corpus to obtain event sentence data, uses the element extraction model to perform element extraction on the event sentence data, and after confirming the result of the element extraction, performs incremental optimization training of the element extraction model.

[0016] Further, the element extraction model is a complex event element extraction model, and the construction process of the complex event element extraction model comprises the following steps:

[0017] S1, establishing an event element label system;

[0018] S2, obtaining a complex event sentence data set;

[0019] S3, according to the event element label system, manually annotating the complex event sentence data set to form a complex event element data set;

[0020] S4, using the complex event element data set to train a pre-trained language model to obtain an optimal element extraction model;

[0021] S5, testing the extraction result accuracy of the optimal element extraction model, and obtaining the trained element extraction model when the accuracy meets the index requirement, and completing the construction of the complex event element extraction model.

[0022] Further, in the step S1, the establishment of the event element label system defines each element part existing in the complex event sentence by referring to the BIO label definition method of the sequence labeling task;

[0023] The element part comprises a common element part and a plurality of sub-event element parts; the common element part comprises a common subject, a common object, a common time, a common place and a common trigger word, and the sub-event element part comprises a sub-event subject, a sub-event object, a sub-event event, a sub-event place and a sub-event trigger word, wherein for a plurality of sub-events, the sub-event element part uniformly numbers the element components using the same serial number.

[0024] Specifically, in step S3 of the construction process of the complex event element extraction model, manual annotation is performed, which involves manually labeling each element in each complex event sentence in the complex event sentence dataset as a common element part or a sub-event element part according to the event element labeling system, thereby ultimately forming the complex event element dataset.

[0025] Furthermore, in step S4, the complex event element dataset is divided into a training set, a validation set, and a test set; the pre-trained language model is trained using the training set, and the optimal element extraction model is selected from the trained pre-trained language model using the validation set; in step S5, the optimal element extraction model is tested for accuracy using the test set.

[0026] Furthermore, in step S4, training the pre-trained language model using the training set specifically includes the following steps:

[0027] S4-1. Divide the complex event sentence dataset in the training set before manual annotation into multiple event data groups of specific sizes;

[0028] S4-2. Input an event data set into the RoBERTa model to obtain the semantic feature vector matrix of each event sentence in the event data set;

[0029] S4-3. Input the semantic feature vector matrix into the linear classifier to obtain the element label probability corresponding to each element in each event sentence, forming an element label probability matrix;

[0030] S4-4. Input the feature label probability matrix into the CRF model to obtain the feature label corresponding to each feature in each event sentence in this event data group;

[0031] S4-5. Calculate the log-likelihood ratio loss between the feature labels and the manually labeled real labels. If the loss value is less than the preset threshold, determine whether the training of the current event data group is complete.

[0032] S4-6. After the current event group data training is completed, determine whether the training of all event data groups is completed. If not, switch to the next event data group and perform the operation starting from step S4-2. If the training of all event data groups is completed, the pre-trained language model is obtained.

[0033] Specifically, in step S4-5, if the loss value is greater than a preset threshold, the current event group data is iterated through backpropagation, and the operation starting from step S4-2 is performed until the result is obtained where the loss value is less than the preset threshold.

[0034] Specifically, the optimal element extraction model is tested for accuracy by the test set, the accuracy value is recorded as F1 value, the accuracy index requirement value is 85%, when the F1 value of the accuracy test result reaches the index requirement value, the trained complex event element extraction model is obtained, the construction of the complex event element extraction model is realized, and the complex event element extraction model is applied to multiple fields.

[0035] Further, in the accuracy test process, if the F1 value of the accuracy test result does not reach the index requirement value, the optimal element extraction model is manually corrected, and the same training process as the pre-training language model is carried out for incremental training.

[0036] Further, when the trained element extraction model is applied to multiple fields, the results of the element extraction work in different fields are manually checked and confirmed, incorrect element results are corrected, and the corresponding results are fed back to the construction part of the element extraction model. According to the text corpus of the field, the element extraction model is trained incrementally to obtain an element extraction model optimized and updated for the field and having practical application value.

[0037] To sum up, since the technical solution is adopted, the present application has the following advantages:

[0038] In view of the problems of the existing element extraction method, such as inaccurate complex event element extraction and missing elements, the present application proposes a complex event element extraction method based on sequence labeling, which can effectively realize accurate positioning and extraction of elements in nested event sentences, complex long event sentences and other complex sentences, and can also complete the supplement and improvement of missing elements in sub-event sentences in complex sentences, thereby effectively supporting subsequent event analysis tasks.

[0039] In view of the problems of the existing element extraction model, such as difficult cross-domain migration application and low efficiency, the present application establishes a complex event element extraction framework for multiple fields through model training, forms an element extraction mechanism with good universality and strong robustness through a sequence labeling event element label system definition method, and can effectively bridge the gap between different fields. At the same time, the present application establishes a matching mechanism of manual feedback correction and model incremental training, which can update the model while using it, quickly realize cross-domain migration application, and therefore the present application can effectively perform any work related to text event element extraction. BRIEF DESCRIPTION OF DRAWINGS

[0040] Figure 1 The present application is a method principle flowchart;

[0041] Figure 2 The present application is a reference sequence labeling task BIO label definition system details schematic diagram;

[0042] Figure 3 A schematic diagram for manually labeling a complex event element tag system;

[0043] Figure 4 A schematic diagram for the training process of a complex event element extraction model;

[0044] Figure 5 A schematic diagram for the practical application of a complex event element extraction model for multi-domain. DETAILED DESCRIPTION

[0045] To make the objects, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the drawings herein can be arranged and designed in various different configurations.

[0046] Therefore, the following detailed description of the embodiments of the present application provided in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by a person of ordinary skill in the art without creative labor based on the embodiments in the present application are within the scope of protection of the present application.

[0047] As shown in Figure 1 A multi-domain complex event element extraction method includes a construction part of an element extraction model and an application part of the element extraction model.

[0048] The construction part of the element extraction model forms an event element dataset according to an event element tag system, uses the event element dataset to perform iterative training of a pre-trained language model for an element extraction task, and obtains the element extraction model after the training is completed.

[0049] The application part of the element extraction model applies the trained element extraction model to multi-domains, performs event detection on multi-domain text corpus, obtains event sentence data, performs element extraction on the event sentence data using the element extraction model, and performs incremental optimization training of the element extraction model after confirming the result of the element extraction.

[0050] In the present embodiment, the element extraction model takes complex events as extraction objects, and therefore the model is a complex event element extraction model, and the construction process can specifically include the following steps.

[0051] S1, establishing an event element tag system;

[0052] S2, obtaining a complex event sentence dataset;

[0053] S3, manually labeling the complex event sentence data set according to the event element label system to form a complex event element data set;

[0054] S4, training the pre-trained language model using the complex event element data set to obtain an optimal element extraction model;

[0055] S5, testing the extraction result accuracy of the optimal element extraction model, and obtaining the trained element extraction model when the accuracy meets the index requirements, completing the construction of the complex event element extraction model.

[0056] Specifically, in the embodiment, the establishment of the event element label system in step S1 is described with reference to Figure 2 , by referring to the BIO label definition method of the sequence labeling task, the various element parts existing in the complex event sentence are defined; considering the complexity of the event sentence, the element part will include the common element part and multiple sub-event element parts.

[0057] The common element part in the complex event sentence is defined, for example, common subject (B_sub_c, I_sub_c), common object (B_obj_c, I_obj_c), common time (B_tim_c, I_tim_c), common place (B_loc_c, I_loc_c) and common trigger word (B_trig_c, I_trig_c);

[0058] The sub-event element part in the complex event sentence is defined, and for multiple sub-events, the sub-event element part is uniformly numbered by the same serial number for element composition; for example, event 1 subject (B_sub_0, I_sub_0), event 1 object (B_obj_0, I_obj_0), event 1 place (B_loc_0, I_loc_0) and event 1 trigger word (B_trig_0, I_trig_0), event 2 subject (B_sub_1, I_sub_1), event 2 object (B_obj_1, I_obj_1), event 2 place (B_loc_1, I_loc_1) and event 2 trigger word (B_trig_1, I_trig_1), and so on.

[0059] As shown in Figure 3 , in step S3 of the construction process of the complex event element extraction model, manual labeling is performed, specifically: according to the event element label system, each element in each complex event sentence in the complex event sentence data set is manually labeled as a common element part or a sub-event element part, thereby finally forming a complex event element data set.

[0060] For Figure 3In the example sentence, the common element part and the sub-event element part are labeled with corresponding element labels respectively, so as to obtain the real labels required for calculating the log-likelihood ratio loss in the subsequent model training process.

[0061] Next, in step S4 of the present embodiment, the complex event element dataset will be divided into a training set, a validation set and a test set; the pre-trained language model is trained through the training set, and the optimal element extraction model is selected from the trained pre-trained language model through the validation set; and in step S5, the optimal element extraction model is tested for accuracy through the test set.

[0062] As shown in Figure 4 The training of the pre-trained language model in step S4 uses the training set, which is specifically divided into the following steps:

[0063] S4-1, the complex event sentence dataset in the training set before manual labeling is divided into a plurality of event data groups of a specific size;

[0064] S4-2, input an event data group into the RoBERTa model to obtain a semantic feature vector matrix of each event sentence in the event data group;

[0065] S4-3, input the semantic feature vector matrix into a linear classifier to obtain the element label probability corresponding to each element in each event sentence, forming an element label probability matrix; in the present embodiment, the linear classifier uses a Sigmoid linear classifier;

[0066] S4-4, input the element label probability matrix into the CRF model to obtain the element label corresponding to each element in each event sentence in the event data group;

[0067] S4-5, calculate the log-likelihood ratio loss of the element label and the real label after manual labeling, and if the loss value is less than a preset threshold, determine whether the current event data group training is completed;

[0068] S4-6, after the current event group data training is completed, determine whether all event data group training is completed, if not, replace the next event data group and perform the operation from step S4-2; if it is determined that all event data group training is completed, the trained pre-trained language model is obtained.

[0069] In step S4-5, if the loss value is greater than the preset threshold, the current event group data is iterated through back propagation, and the operation from step S4-2 is performed until the result with the loss value less than the preset threshold is obtained; if the current event data group training is not completed at the time of judgment, it is also iterated until the current training stage is completed.

[0070] The foregoing step S5 finally tests the optimal element extraction model by using a test set, and an accuracy rate value is recorded as an F1 value. The accuracy rate is required to be 85%. When the F1 value of the accuracy rate test result reaches the required value, the complex event element extraction model is trained, the construction of the complex event element extraction model is realized, and the complex event element extraction model is applied to multiple fields. For details, refer to Figure 1 The schematic diagram is shown in FIG. 6.

[0071] In the embodiment, if the F1 value of the accuracy rate test result does not reach the required value in the accuracy rate test process, the optimal element extraction model is manually corrected, and the same training process as the pre-training language model is carried out for the incremental training.

[0072] Finally, when the trained element extraction model is applied to multiple fields, the results of the element extraction work in different fields are manually checked and confirmed, incorrect element results are corrected, and the corresponding results are fed back to the construction part of the element extraction model. According to the text corpus in the field, the incremental optimization training process is carried out for the element extraction model, so as to obtain the element extraction model optimized and updated for the field and having practical application value.

[0073] For details, refer to Figure 5 The complex event element extraction method shown in the embodiment can be applied to multiple fields for complex event element extraction work. The optimal element extraction model composed of the RoBERTa model and the CRF conditional random field model obtains event sentence element labels through the model, and obtains the final event element extraction result through label mapping, so as to realize the construction and application of the complex event element extraction model. When applied in different fields, the event element extraction result is manually checked and confirmed, incorrect element results are corrected and fed back to the model training and construction part, the element extraction model is incrementally optimized, the complex event element extraction model is updated while used, the cross-field migration application ability is realized more quickly as a whole, the required calculation training amount is relatively small, and the element extraction of the complex event sentence can be realized.

Claims

1. A multi-domain oriented complex event element extraction method, characterized in that: The element extraction model comprises a construction part and an application part; The construction part of the element extraction model forms an event element data set according to an event element label system, performs iterative training of the pre-trained language model on the element extraction task using the event element data set, and obtains the element extraction model after the training is completed; The application part of the element extraction model applies the trained element extraction model to multiple fields, performs event detection on multiple field text corpora, obtains event sentence data, performs element extraction on the event sentence data using the element extraction model, confirms the result of the element extraction, and performs incremental optimization training of the element extraction model; The element extraction model is a complex event element extraction model, and the construction process of the complex event element extraction model comprises the following steps: S1, establishing an event element label system; S2, obtaining a complex event sentence data set; S3, manually labeling the complex event sentence data set according to the event element label system to form a complex event element data set; S4, training the pre-trained language model using the complex event element data set to obtain an optimal element extraction model; S5, testing the extraction result accuracy of the optimal element extraction model, and obtaining the trained element extraction model when the accuracy meets the index requirement, and completing the construction of the complex event element extraction model; In step S4, the complex event element data set is divided into a training set, a validation set and a test set; the pre-trained language model is trained through the training set, and the optimal element extraction model is selected from the trained pre-trained language model through the validation set; in step S5, the optimal element extraction model is tested for accuracy through the test set; The accuracy value is recorded as F1 value, the index requirement value of the accuracy is 85%, and the trained complex event element extraction model is obtained when the F1 value of the accuracy test result reaches the index requirement value, the construction of the complex event element extraction model is realized, and the complex event element extraction model is applied to multiple fields.

2. The multi-domain oriented complex event element extraction method according to claim 1, characterized in that: In step S1, the establishment of the event element label system defines each element part existing in the complex event sentence by referring to the BIO label definition method of the sequence labeling task; The element part comprises a common element part and a plurality of sub-event element parts; the common element part comprises a common subject, a common object, a common time, a common place and a common trigger word, the sub-event element part comprises a sub-event subject, a sub-event object, a sub-event event, a sub-event place and a sub-event trigger word, and the sub-event element part is uniformly numbered using the same serial number for element composition for a plurality of sub-events.

3. The multi-domain oriented complex event element extraction method according to claim 2, characterized in that: In step S3 of the constructing process of the complex event element extraction model, manual labeling is performed, specifically: according to the event element label system, each element in each complex event sentence in the complex event sentence data set is manually labeled as a shared element part or a sub-event element part, thereby finally forming the complex event element data set.

4. The multi-domain oriented complex event element extraction method of claim 1, wherein: In step S4, the pre-trained language model is trained by using the training set, specifically including the following steps: S4-1, the complex event sentence data set in the training set before manual labeling is divided into a plurality of event data groups of a specific size; S4-2, an event data group is input into the RoBERTa model to obtain a semantic feature vector matrix of each event sentence in the event data group; S4-3, the semantic feature vector matrix is input into the linear classifier to obtain an element label probability corresponding to each element in each event sentence, thereby forming an element label probability matrix; S4-4, the element label probability matrix is input into the CRF model to obtain an element label corresponding to each element in each event sentence in the event data group; S4-5, the log likelihood ratio loss of the element label and the real label after manual labeling is calculated, and if the loss value is less than a preset threshold, it is determined whether the current event data group training is completed; S4-6, after the current event group data training is completed, it is determined whether all event data group training is completed, if not, the next event data group is replaced, and the operation from step S4-2 is performed; if it is determined that all event data group training is completed, the pre-trained language model after training is obtained.

5. The multi-domain oriented complex event element extraction method of claim 4, wherein: In step S4-5, if the loss value is greater than the preset threshold, the current event group data is iterated by back propagation, and the operation from step S4-2 is performed until the result with the loss value less than the preset threshold is obtained.

6. The multi-domain oriented complex event element extraction method of claim 1, wherein: In the accuracy test process, if the F1 value of the accuracy test result does not reach the index requirement value, the optimal element extraction model is manually corrected, and the same training process as the pre-trained language model is carried out for incremental training.

7. The multi-domain oriented complex event element extraction method according to claim 1, characterized in that: When the trained element extraction model is applied to multiple fields, the results of the element extraction work in different fields are manually checked and confirmed, the incorrect element results are corrected, and the corresponding results are fed back to the construction part of the element extraction model. According to the text corpus of the field, the element extraction model is subjected to incremental optimization training process, and an element extraction model optimized and updated for the field and having practical application value is obtained.

Images