Event element information extraction method based on GPlinker

Abstract

The invention relates to an event element information extraction method based on a GPlinker, and belongs to the technical field of artificial intelligence and information extraction. In order to solve the problems of unbalanced sample distribution, poor data labeling quality, data pollution and the like, data is cleaned by adopting a model hyper-parameter search method based on an event argument relationship and trigger word confidence; encoding the event text by using an open-source pre-training model, and performing event trigger word extraction and event classification by using a GPlinker model; a hierarchical learning rate strategy is adopted to carry out model training, and a model fusion method based on voting correction is adopted. According to the method, the event element information extraction capability of the model is enhanced, and the stability and accuracy of the model are improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence and information extraction technology, and particularly relates to a GPlinker-based event element information extraction method. Background Art

[0002] Event extraction aims to structure event information in unstructured text and is a key research area in natural language processing. Event detection, a key subtask of event extraction, aims to identify the location of event triggers in a given text and classify them into the correct event type. GPlinker, a joint entity-relationship extraction model based on GlobalPointer, achieves more accurate identification of events and their related elements by linking the relationships between trigger words and event arguments.

[0003] For general domain events, mature open-source datasets such as ACE2005 have been developed, which have promoted research in general domain event detection and event extraction. However, for some specific domain events, due to their unique nature, there is a lack of authoritative, high-quality, and publicly available datasets for event extraction, both domestically and internationally, and text datasets are particularly scarce. Furthermore, while deep neural network models have achieved promising results for event detection, they are limited by the quality requirements of training datasets. When faced with issues such as uneven sample distribution, poor data annotation quality, and adversarial data contamination, the models' effectiveness is difficult to guarantee, resulting in a lack of robustness.

[0004] To address the above problems, the present invention proposes a GPlinker-based event element information extraction method. The GPlinker model is introduced on the basis of traditional event detection, and combined with data cleaning strategy, hierarchical learning rate strategy and model fusion strategy, to promote the research on the robustness of special domain event detection models, support the implementation of domain event extraction related technologies, and promote the development of artificial intelligence applications in related fields. Summary of the Invention

[0005] (1) Technical issues to be resolved

[0006] The technical problem to be solved by the present invention is how to provide an event factor information extraction method based on GPlinker to solve problems such as uneven sample distribution, poor data annotation quality, and data pollution.

[0007] (2) Technical solution

[0008] In order to solve the above technical problems, the present invention proposes a method for extracting event element information based on GPlinker, which includes the following steps:

[0009] Step 1: Data cleaning: Clean the data based on event argument relationships, trigger word confidence, and model hyperparameter search methods.

[0010] Step 2: Task Abstraction: Convert the event detection task into a named entity recognition (NER) task, treating the event type as the entity type in the NER task and the trigger word as the entity to be extracted in the NER task. Adopt an "encode-decode" strategy: in the encoding phase, use an open-source pre-trained model to encode the event text. In the decoding phase, use the GPlinker model to extract event trigger words and classify events.

[0011] Step 3: Model training: Use a hierarchical learning rate strategy to train the model so that the pre-trained model can better adapt to the training of downstream tasks;

[0012] Step 4: Model integration: A model fusion method based on voting correction is used. After encoding with a pre-trained model, a grid search method is used to train multiple different GPlinker models with different hyperparameters for decoding. The best-performing GPlinker model is selected as the main model. The output of this model is used as one of the candidate results, called the main model output. The remaining models are then used to correct the main model output.

[0013] (3) Beneficial effects

[0014] The present invention proposes a method for extracting event element information based on GPlinker. The main advantages of the present invention are reflected in the following aspects:

[0015] (1) The present invention uses the GPlinker model to design a confidence-based data cleaning method, which improves the quality of the training set.

[0016] (2) The present invention uses the GPlinker model as a decoding model for event detection, which enhances the model's ability to extract event element information.

[0017] (3) The layered learning rate strategy improves the adaptability of the pre-trained model to downstream tasks.

[0018] (4) The model fusion method based on voting correction improves the stability and accuracy of the model by integrating the results of multiple GPlinker models in the decoding stage. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 It is the overall framework of the method of the present invention;

[0020] Figure 2 This is an example diagram of label contaminated data;

[0021] Figure 3It is a confidence-based data cleaning process;

[0022] Figure 4 This is the structure diagram of the event detection model based on GPlinker;

[0023] Figure 5 is the layered learning rate strategy graph. DETAILED DESCRIPTION

[0024] In order to make the purpose, content and advantages of the present invention more clear, the specific implementation methods of the present invention are further described in detail below with reference to the accompanying drawings and examples.

[0025] The present invention relates to a GPlinker-based event element information extraction method, which aims to improve the accuracy of event detection and data cleaning efficiency in special domain text corpora.

[0026] This invention discloses a method for extracting event element information based on GPlinker. By using machine learning, deep learning, and other technologies, the model maintains good performance even in situations with uneven sample distribution, poor data annotation quality, and data contamination. The main problems to be solved by this invention are the following two aspects:

[0027] The quality of data annotation is poor and there is serious data pollution. It is necessary to reduce the impact of polluted data or harmful samples on the model learning process.

[0028] The data distribution of each category in the dataset is extremely unbalanced, and the model's recognition ability for small sample categories needs to be enhanced.

[0029] The present invention provides a method for extracting event element information based on GPlinker, such as Figure 1 shown.

[0030] Step 1: Data Cleaning

[0031] Data is cleaned based on event argument relationships, trigger word confidence, and model hyperparameter search methods.

[0032] exist Figure 2 In the label contaminated data example, the labeled trigger word is "crash", and the event type is incorrectly labeled as "maneuver (Manoeuvre)", and the correct event type label should be "accident (Accident)".

[0033] S11. The present invention first performs preliminary data cleaning on the original data based on the event argument relationship;

[0034] S12. Then, the data is cleaned again based on the trigger word confidence.

[0035] S13. Use the model hyperparameter search method to further clean the data based on the model.

[0036] Since the original data set contains the majority of accurate samples, the model can learn relatively accurate feature representations from them. In the present invention, a model hyperparameter search method, Grid-Search, is used to further clean the data.

[0037] S131. Based on the cleaned data from S12, multiple different event detection models are trained simultaneously using different hyperparameters, namely, the GPlinker model.

[0038] S132, then use the trained event detection model to predict the data in the training set and the validation set; if the trigger word confidence score in the prediction result of a certain data is greater than a certain threshold, the original annotation result of the data is modified to the model prediction result, otherwise the original annotation of the data is retained. The specific process is as follows Figure 3 shown.

[0039] The trigger word confidence score calculation formula is as follows:

[0040]

[0041] Where N is the number of GPlinker models when cleaning data based on the model;<i,j> is the span of the trigger word position in the text, i and j represent the starting position and ending position of the trigger word respectively; Prediction for the mth model<i,j> The corresponding word is the probability score of the trigger word.

[0042] Step 2: Task Abstraction

[0043] In this paper, the event detection task is converted into a named entity recognition (NER) task, the event type is regarded as the entity type in the NER task, and the trigger word is regarded as the entity to be extracted in the NER task. Specifically, the present invention adopts an "encoding-decoding" strategy. In the encoding stage, an open source pre-trained model is used to encode the event text. In the decoding stage, the GPlinker model is used to extract event trigger words and classify events. Figure 4 shown.

[0044] Since "non-event" type samples do not contain trigger words, the label matrix may be filled with zeros when constructing sample entity labels. For datasets containing a large number of "non-event" texts, this situation will aggravate the data distribution imbalance and increase the difficulty of model training. Therefore, the present invention adds a special mark [N] at the end of each text as the trigger word for "non-event" event type samples, thereby reducing the loss imbalance between the "non-event" event type and other event types, and improving the effectiveness of the model when the distribution of the training dataset is extremely uneven.

[0045] (1) Coding stage

[0046] Pre-trained models obtain more universal representations of words from large-scale corpora. In the latest CLUE natural language understanding (NLU) task, various pre-trained models achieved optimal results and can provide better model initialization.

[0047] Therefore, the present invention uses a pre-trained model as an encoder to represent the input text and obtain the corresponding context representation for each input token. The formula is as follows:

[0048] h1,h2,…,h n =PLM(w1,w2,…,w n )

[0049] Among them, PLM(·) is the pre-training model used, such as BERT; w i is the i-th token of the input text; h i is the context representation vector of the i-th token obtained through the pre-training model; n is the length of the text token.

[0050] (2) Decoding stage

[0051] In the decoding process, the present invention uses the GPlinker model to decode, transforming the multi-label classification problem of event trigger words into a multi-round binary classification problem, and calculating the span in each round.<i,j> The corresponding word is the score of the event type trigger word and the score of the non-t event type. Then, the scores are compared and the highest score is taken to obtain the event trigger word span and event type. This decoding method automatically balances the weight of each item and can better address the sample imbalance problem in this task.

[0052] The present invention has obtained the representation of each token in the sentence h1,h2,…,h in the encoding stage. n For the i-th word, the present invention uses two linear transformations to obtain the span representation.

[0053] r start,i,t =W q,t hi +b q,t

[0054] r end,i,t =W k,t h i +b k,t

[0055] in, and Respectively represent the starting and ending positions of the i-th word as the trigger word of event type t; W q,t , b q,t , W k,t , b k,t These are the parameter matrices that the model needs to train.

[0056] Then, the span is calculated by dot product<i,j> The vector product of is taken as the score of the trigger word of event type t.

[0057]

[0058] in, is an orthogonal matrix that satisfies When calculating the score, ROPE relative position encoding is added to each token to record context information; i,t , k j,t are the trigger word start and end position representation vectors with relative position encoding added; s t (i,j) represents the span<i,j> The corresponding word is the score of the trigger word of event type t.

[0059] This encoding method can bring better generalization and accelerate the convergence of the target task. In the final reasoning step, the span that satisfies the following expression is regarded as the event trigger word span of type t.

[0060]

[0061] Step 3: Model training

[0062] Because the pre-trained model parameters have already reached a relatively high level, and the subsequent GPlinker model decoding structure is trained from scratch, a very low learning rate is often used during model training to ensure that the knowledge learned by the pre-trained model is not lost. However, in cases where the distribution of domain samples is extremely uneven, using a small learning rate can make it difficult to fully learn the downstream task.

[0063] Therefore, the present invention adopts a layered learning rate strategy to train the model, so that the pre-trained model can better adapt to the training of downstream tasks. When using samples to fine-tune downstream tasks, we set the learning rate of the first layer of the pre-trained model to be kept at 2*e-5 , the last layer learning rate is kept at 1*e -4 , the learning rate of each intermediate layer is kept increasing in sequence, and the learning rate of the downstream decoding stage is set to 1*e -3 , so that the pre-trained model can better adapt to the training of downstream tasks. Figure 5 shown.

[0064] Specifically, the learning rate formula for each layer of the pre-training model is as follows:

[0065]

[0066] lr(i)=2*e -5 ,i=0

[0067] Where LN is the number of layers of the pre-trained model; α is the learning rate growth factor, α>1; lr(i) is the learning rate of the pre-trained model Layeri+1.

[0068] Step 4: Model Integration

[0069] The present invention designs a model fusion method based on voting correction, and uses this method to improve the robustness of the model. First, after encoding with a pre-trained model, a grid search method is used to train multiple different GPlinker models using different hyperparameters for decoding. The GPlinker model with the best performance is selected as the main model, and the output of this model is used as one of the candidate results, referred to as the main model output. The main model output is then corrected using the remaining models. The present invention uses a hard voting strategy to aggregate the prediction results of the remaining models, and statistics the prediction results of each model by cumulative addition to achieve the effect of correcting the main model.

[0070] Unlike traditional voting methods, this method doesn't rely on a majority-follows strategy. Because the primary model is the most accurate among multiple models, this method prioritizes its results. Only when a prediction exceeds a threshold is it used as the final result, thereby correcting the primary model. The threshold in this method is set to half the total number of models.

[0071] An example of model integration is shown in Table 1. The present invention trained 7 models, selected the best model as the main model, and the remaining 6 models as auxiliary models for voting. The voting threshold was set to half of the total number of models, that is, 3.5 votes.

[0072] Table 1 Model ensemble examples

[0073]

[0074]

[0075] In Example 1, the primary model predicted a "Manoeuvre" event with the trigger word "landing," while four auxiliary models all considered it an "Experimental" event with the trigger word "first flight." This exceeded our set threshold, so we selected the result predicted by the majority of models as the final model prediction. This allows us to promptly correct any errors learned by individual models during training, thereby improving the overall robustness of the model. In Example 2, the predictions of the remaining models were more dispersed. While some results had a vote count greater than 1, none exceeded the threshold. In this case, we selected the output of the primary model as the final model prediction.

[0076] The main advantages of the present invention are embodied in the following aspects:

[0077] (1) The present invention uses the GPlinker model to design a confidence-based data cleaning method, which improves the quality of the training set.

[0078] (2) The present invention uses the GPlinker model as a decoding model for event detection, which enhances the model's ability to extract event element information.

[0079] (3) The layered learning rate strategy improves the adaptability of the pre-trained model to downstream tasks.

[0080] (4) The model fusion method based on voting correction improves the stability and accuracy of the model by integrating the results of multiple GPlinker models in the decoding stage.

[0081] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the technical principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for extracting event element information based on GPlinker, characterized in that: The method comprises the following steps: Step 1: Data cleaning: Clean the data based on event argument relationships, trigger word confidence, and model hyperparameter search methods. Step 2: Task Abstraction: Convert the event detection task into a named entity recognition (NER) task, treating the event type as the entity type in the NER task and the trigger word as the entity to be extracted in the NER task. Adopt an "encode-decode" strategy. In the encoding phase, use an open-source pre-trained model to encode the event text. In the decoding phase, use the GPlinker model to extract event trigger words and classify events. Step 3: Model training: Use a hierarchical learning rate strategy to train the model so that the pre-trained model can better adapt to the training of downstream tasks; Step 4: Model integration: A model fusion method based on voting correction is used. After encoding with a pre-trained model, a grid search method is used to train multiple different GPlinker models with different hyperparameters for decoding. The best-performing GPlinker model is selected as the main model. The output of this model is used as one of the candidate results, called the main model output. The remaining models are then used to correct the main model output.

2. The event element information extraction method based on GPlinker according to claim 1, characterized in that: The step one comprises: S11. First, perform preliminary data cleaning on the original data based on the event argument relationship; S12, then clean the data again based on the trigger word confidence; S13. Use the model hyperparameter search method to further clean the data based on the model.

3. The event element information extraction method based on GPlinker according to claim 2, characterized in that: S13 further cleans the data using a model hyperparameter search method, Grid-Search, which specifically includes: S131. Based on the cleaned data from S12, multiple different event detection models are trained simultaneously using different hyperparameters, namely, the GPlinker model. S132. The trained event detection model is then used to predict the data in the training set and the validation set. If the trigger word confidence score in the prediction result of a certain data is greater than a certain threshold, the original annotation result of the data is modified to the model prediction result, otherwise the original annotation of the data is retained.

4. The event element information extraction method based on GPlinker according to claim 3, characterized in that: The trigger word confidence score calculation formula is as follows: Where N is the number of GPlinker models when cleaning data based on the model;<i,j> is the span of the trigger word position in the text, i and j represent the starting position and ending position of the trigger word respectively; Prediction for the mth model<i,j> The corresponding word is the probability score of the trigger word.

5. The method for extracting event element information based on GPlinker according to any one of claims 1 to 4, characterized in that: In the step 2, during the encoding phase, Use the pre-trained model as the encoder to represent the input text and get the corresponding context representation for each input token. The formula is as follows: h1,h2,…,h n =PLM(w1,w2,…,w n ) Among them, PLM(·) is the pre-training model used; w i is the i-th token of the input text; h i is the context representation vector of the i-th token obtained through the pre-training model; n is the length of the text token.

6. The event element information extraction method based on GPlinker according to claim 5, characterized in that: In the second step, during the decoding phase, Use the GPlinker model for decoding, transforming the multi-label classification problem of event trigger words into a multi-round binary classification problem, calculating the span in each round.<i,j> The corresponding words are the scores of the t event type trigger words and the scores of non-t event types, and then compared, and the highest score is taken to obtain the event trigger word span and event type; In the encoding stage, the representation of each token in the sentence h1,h2,…,h n ; For the i-th word, two linear transformations are used to obtain the span representation; r start,i,t =W q,t h i +b q,t r end,i,t =W k,t h i +b k,t in, and Respectively represent the starting and ending positions of the i-th word as the trigger word of event type t; W q,t , b q,t , W k,t , b k,t These are the parameter matrices that the model needs to train; Then, the span is calculated by dot product<i,j> The vector product of is the score of the trigger word of event type t; in, is an orthogonal matrix that satisfies When calculating the score, ROPE relative position encoding is added to each token to record context information; i,t , k j,t are the trigger word start and end position representation vectors with relative position encoding added; s t (i,j) represents the span<i,j> The corresponding word is the score of the trigger word of event type t; In the final reasoning step, the span that satisfies the following expression is considered as an event trigger span of type t; 7. The method for extracting event element information based on GPlinker according to claim 6, characterized in that: In the step 2, a special mark [N] is added at the end of each text as a trigger word for the "non-event" event type sample.

8. The event element information extraction method based on GPlinker according to claim 6, characterized in that: In step 3, when using samples to fine-tune the downstream task, the learning rate of the first layer of the pre-trained model is set to 2*e -5 , the last layer learning rate is kept at 1*e -4 , the learning rate of each intermediate layer is kept increasing in sequence, and the learning rate of the downstream decoding stage is set to 1*e -3 , so that the pre-trained model can better adapt to the training of downstream tasks.

9. The event element information extraction method based on GPlinker according to claim 8, characterized in that: The learning rate formula for each layer of the pre-training model is as follows: lr(i)=2*e -5 ,i=0 Where LN is the number of layers of the pre-trained model; α is the learning rate growth factor, α>1; lr(i) is the learning rate of the pre-trained model Layeri+1.

10. The event element information extraction method based on GPlinker according to claim 6, characterized in that: In the step 4, a hard voting strategy is used to aggregate the prediction results of the remaining models, and the prediction results of each model are counted by cumulative addition to achieve the effect of correcting the main model; only when the number of votes for a certain prediction result is greater than a threshold, this result is used as the final result, thereby achieving the effect of correcting the main model.

Images