A method and apparatus for biased few-shot event detection based on multi-step prompting learning

Abstract

This invention provides a method and apparatus for de-biased few-shot event detection based on multi-step cue learning. The method includes the following steps: (1) Selecting an initial dataset and dividing it into a data matrix that conforms to the few-shot training settings; selecting instances from each preset event type and distinguishing them to form a training set, a validation set, and a test set; (2) Performing multi-step processing on the event texts of the training set, validation set, and test set; identifying the event text and obtaining the predicted trigger words in one subtask; and classifying the event texts based on the trigger words in another subtask; (3) Clustering the event texts after each classification to obtain the centroid of each cluster as the representation of the event label, outputting the probability of each event type, and the event type with the highest probability is the detected event label. This application can adapt to real few-shot settings and scenarios with insufficient data, solves the context bypass problem, and improves generalization performance.

Description

Technical Field

[0001] This invention belongs to the field of event detection technology, specifically relating to a method for detecting biased few-sample events based on multi-step cue learning. Background Technology

[0002] Event detection (ED) is a key step in event extraction (EE) and a crucial task in natural language processing (NLP). It aims to identify event trigger words and classify them into predefined event types. ED tasks help computers understand human language and are used in many applications such as automatic content extraction, question answering, and knowledge graphs. Typically, event detection (ED) is an important task consisting of two interconnected subtasks: trigger identification and event classification, and it is widely applied in many downstream tasks.

[0003] However, existing event detection models have low applicability in real-world scenarios with limited labeled data, and trigger bias in the dataset can lead to context bypass and insufficient generalization. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a biased few-sample event detection method based on multi-step cue learning, so as to solve the problems of low applicability of existing event detection models in real-world scenarios with limited labeled data, and the triggering bias in the dataset causing context bypass and insufficient generalization.

[0005] To achieve the above objectives, the technical solution adopted in this application is to provide a method for detecting biased few-shot events based on multi-step cue learning, the method comprising the following steps:

[0006] (1) Select the initial dataset, divide the initial dataset into a data matrix that conforms to the small sample training settings, select instances from each preset event type and distinguish them to form a training set, a validation set and a test set;

[0007] (2) Perform multi-step processing on the event texts of the training set, the validation set and the test set, that is, execute two sub-tasks simultaneously and train two consecutive sub-cues in one iteration, identify the event text and obtain the predicted trigger word in one of the sub-tasks, and classify the event text according to the trigger word in the other sub-task.

[0008] (3) Cluster the event texts after each classification to obtain the centroid of each cluster as the representation of the event label, output the probability of each event type, and the event type with the highest probability is the detected event label.

[0009] Optionally, in step (1), the step of selecting instances from each event type and distinguishing them to form a training set, a validation set, and a test set specifically includes: randomly sampling K instances under each event type in the initial dataset to form a K-sample training set, where the length of the K-sample training set is N*K, and N is the size of the event label space in the original dataset;

[0010] Repeat this process to generate a K-sample validation set that does not overlap with the training set from the remaining unsampled instances;

[0011] The remaining unsampled instances form the K-sample test set.

[0012] Optionally, step (2) further includes modifying the event text of the training set, the validation set, and the test set into a prompt function, the specific form of which is:

[0013]

[0014] in The prompt template is a manually constructed prompt template. Includes a mask location [MASK];

[0015] x is the event text, [CLS] is the category instruction, and [SEP] is the separator.

[0016] Optionally, in step (2), obtaining the predicted trigger word specifically includes: given an event text, by mapping the massive vocabulary of a large-scale pre-trained language model to the original text, the probability distribution of the trigger word is obtained, and the candidate word with the highest probability is the predicted trigger word.

[0017] Optionally, the probability distribution formula for the trigger word is as follows:

[0018]

[0019] Among them, H [MASK] Here, t is the hidden layer vector of [MASK], w is the input event text sequence, f1(x) is the prompt f1 provided by the prompt function, and w i The candidate word with the highest probability selected within the event text sequence w.

[0020] Optionally, in step (2), the event classification specifically includes classifying the initial event text and trigger words into predefined event types.

[0021] Optionally, step (2) further includes: setting additional knowledge-enhanced ontology text, wherein the knowledge-enhanced ontology text specifically includes: adding an ontology text after the event text after recognizing the event text and obtaining the predicted trigger word, thereby further explaining its meaning;

[0022] In event classification, another ontology text is placed between the event text and the predicted trigger word, thereby clarifying the event classification target of the event text and the predicted trigger word.

[0023] Optionally, in step (3), the clustering of the classified event texts to obtain the centroid of each cluster as the representation of the event label, and the output of the probability of each event type specifically include:

[0024] Given d-dimensional event embeddings e0 obtained from the event classifier, the distance between the event embeddings is measured using a Euclidean metric; the cluster centroids are further computed as label representations. All of these form an event vector space E = {e1, e2, ..., e} N}, that is, a prototype network, from which the predicted probabilities are obtained. The formula is as follows:

[0025]

[0026] in For Euclidean distance.

[0027] Optionally, in step (3), the output probability of each event type specifically includes:

[0028] Given a set of event labels Y = {y1, y2, ..., y...} N Based on the generated d-dimensional event embedding e0, the probability of a candidate event type e∈Y is expressed as P. e :

[0029]

[0030] Where, p i (i = 1, 2, ...,) represents the event label y. i The predicted probability, f2() is the prompt f2 provided by the prompt function.

[0031] This application also provides a biased few-shot event detection device based on multi-step prompting learning, including: an undersampling module, a multi-step prompting module, and a prototype network module;

[0032] The undersampling module divides the input initial dataset into a data matrix that conforms to the few-sample training settings, selects instances from each preset event type, and distinguishes them to form a training set, a validation set, and a test set;

[0033] The multi-step prompting module performs multi-step processing on the event text of the training set, the validation set, and the test set. That is, in one iteration, two sub-tasks are executed simultaneously and two consecutive sub-prompts are trained synchronously. In one of the sub-tasks, the event text is identified and the predicted trigger word is obtained. In the other sub-task, the event text is classified according to the trigger word.

[0034] The prototype network module clusters the classified event texts to obtain the centroid of each cluster as the representation of the event label, and outputs the probability of each event type. The event type with the highest probability is the detected event label.

[0035] The beneficial effects of this invention are as follows: by setting up an undersampling module, a multi-step prompting module, and a prototype module, a new training set is constructed through the undersampling module to adapt to real small sample settings and scenarios with insufficient data; the multi-step prompting module is equipped with a knowledge-enhanced ontology text, which is used to model the entire event text by supplementing the ontology text and prompting learning, and fully mining prior knowledge in the pre-trained language model, effectively overcoming the problems of context bypassing and difficulty in generalization caused by trigger bias; the prototype module makes up for the weakness of event classification due to lack of data, eliminates noise of high-frequency labels, and improves the recognition ability of low-frequency labels, thereby improving generalization performance. Attached Figure Description

[0036] Figure 1 A flowchart illustrating a biased few-shot event detection method based on multi-step cue learning provided by this invention. Figure 1 ;

[0037] Figure 2 A flowchart illustrating a biased few-shot event detection method based on multi-step cue learning provided by this invention. Figure 2 ;

[0038] Figure 3 A schematic diagram of the multi-step prompting module in a bias-removing few-sample event detection device based on multi-step prompting learning provided by the present invention;

[0039] Figure 4 This is a schematic diagram illustrating the model performance of ACE-2005 and FewEvent in a 4-sample setting on the full or sampled test sets.

[0040] Figure 5 This is a schematic diagram illustrating the impact of input length in a 32-sample setting. Detailed Implementation

[0041] To make the objectives, technical solutions, and advantages of the present invention clearer, the technical solutions of the present invention will be described in more detail below with reference to the accompanying drawings of the preferred embodiments. In the drawings, the same or similar reference numerals denote the same or similar components or components having the same or similar functions throughout. The described embodiments are some, but not all, embodiments of the present invention. The embodiments described below with reference to the accompanying drawings are exemplary and intended to explain the present invention, and should not be construed as limiting the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention. The embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0042] In the description of this invention, it should be noted that, unless otherwise explicitly specified and limited, the terms "installation," "connection," and "linking" should be interpreted broadly. For example, they can refer to a fixed connection, an indirect connection through an intermediate medium, or a connection within two components or an interaction between two components. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0043] In the description of this invention, it should be understood that the terms "upper", "lower", "front", "rear", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this invention.

[0044] The terms "first," "second," and "third" (if any) in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a particular order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented, for example, in orders other than those illustrated or described herein.

[0045] Furthermore, the terms “comprising” and “having”, and any variations thereof, are intended to cover non-exclusive inclusion, such as a process, method, system, product, or maintenance tool that includes a series of steps or units, not necessarily limited to those steps or units that are explicitly listed, but may include other steps or units that are not explicitly listed or that are inherent to such process, method, product, or maintenance tool.

[0046] As attached Figure 1-5 As shown, this invention provides a method for detecting biased few-shot events based on multi-step cue learning, which includes the following steps:

[0047] (1) Select the initial dataset and set the undersampling module. The initial dataset is divided into a data matrix that meets the small sample training settings through the undersampling module. Instances are selected from each event type and distinguished to form training set, validation set and test set.

[0048] (2) Set up a multi-step prompting module to process the event text input from the training set, validation set and test set in multiple steps. The multi-step prompting module executes two sub-tasks simultaneously in one iteration and trains two consecutive sub-prompts at the same time. In one sub-task, the event text is identified and the predicted trigger word is obtained. In the other sub-task, the event text is classified according to the trigger word.

[0049] (3) Introduce a prototype network module to cluster the event texts after each classification, so as to obtain the centroid of each cluster as the representation of the event label.

[0050] Specifically, based on a real-world few-shot event detection paradigm, this application proposes a multi-step prompt learning model (MsPrompt). The workflow of the multi-step prompt learning model (MsPrompt) is as follows: Figure 2 As shown. First, an undersampling module is used to split the initially labeled dataset into a data matrix. To meet the requirements of realistic small sample training settings, N is the number of event types, and K is the number of samples for each event type. It should be noted that the initial labeled dataset can be any existing published dataset, such as the ACE-2005 dataset and the FewEvent dataset. The following explanation uses the ACE-2005 dataset and the FewEvent dataset as examples, and this embodiment does not impose any limitations on them.

[0051] Then, an instance x is randomly sampled from X. i The information is then input into the next multi-step prompt module. Trigger words are then identified from the event text and input along with the original text into the event classifier, where x... i Embedded by d-dimensional events Finally, the event embedding e0 is mapped to the event vector space E = {e1, e2, ..., e} in the prototype network. N},in It is event type y i The d-dimensional embedding is used to output the probability p of each event type. i The most probable type is the predicted event label.

[0052] Specifically, in step (1), the frequency of occurrence of labels in the event label set Y is highly unbalanced and follows a long-tail distribution, which may introduce unpredictable biases to event detection. To avoid such biases in the dataset and improve the generalization ability of scarce event types with a small number of instances, this embodiment uses an undersampling module in ED to select the same number of instances from each event type to form a new training and validation set without label bias.

[0053] Specifically, given an ED dataset, this embodiment randomly samples K instances for each event type to form a K-sample training set, i.e.,

[0054]

[0055] Where N is the size of the event label space in the original dataset, and the length of the training set is |X|. train |=N×K. Then, repeat this operation on the remaining unsampled instances to generate those disjoint at X. train K-sample validation set X valid ,Right now Finally, the remaining unsampled instances form the test set X. test =XX valid -X train .

[0056] The newly constructed training, validation, and test sets can better meet the needs of low-resource scenarios in reality, adapt to the true few-shot learning paradigm, and more effectively evaluate the performance of the FSED model.

[0057] Specifically, in step (2), since the traditional ED paradigm x→(x,t)→y is a multi-step process, this embodiment extends the general one-step prompt to a multi-step prompt, forcing the model to execute two sub-tasks simultaneously in one iteration and train two consecutive sub-prompts concurrently. In the prompt model, we modify the original input text x into the prompt function f. prompt (x), which has the following form:

[0058]

[0059] The prompt template T contains a mask position [MASK], which will be mapped to the answer z. Here, x is the input event text, [CLS] is the classification instruction, and [SEP] is the separator.

[0060] This module contains two corresponding steps: trigger identification and event classification, as well as a supplementary knowledge-enhanced ontology.

[0061] Specifically, trigger recognition involves identifying trigger words.

[0062] Identifying trigger words can be viewed as a text annotation task. For example, in this embodiment, "Trigger word is [MASK]." is manually constructed as a prefix prompt template. Where [MASK] is the mask position, learned through hints at x i Matching trigger word t i . Template Concatenate each input text x (e.g., the event text "And I agree that we shouldn't send people over there.") to obtain the modified prompt f1 as:

[0063]

[0064] In detail, given an event text sequence w = (w1, w2, ..., w... L ), where L is the text length. The trigger word t can be represented by filling [MASK] using a masked language modeling process. Then, by mapping the massive vocabulary of the pre-trained language model to the original text sequence w, the probability distribution of the trigger word is obtained, denoted as P. t :

[0065]

[0066] Among them, H [MASK] This is the hidden layer vector of [MASK]. t is the trigger word, and f1(x) is the cue f1 provided by the cue function. In the event text sequence w, the candidate word with the highest probability w... i These are the identified trigger words. Then, annotated event text is generated, i.e., w = (w1, w2, ..., t, ..., w L ).

[0067] Accordingly, after obtaining the predicted trigger words, the text can be classified into predefined event types, a process known as event classification. The prefix prompt template used in the above embodiments of this application... It is "This is an event about [MASK]". Here, [MASK] represents the event to which the text belongs; the newly assembled prompt f2 is then input into the Masked Language Model (MLM) to generate the event embedding.

[0068]

[0069] Where x is an event text that triggers an infiltration trigger word. (For example, in...) Figure 3In the example, “And I agree that we shouldn't send people over there. [SEP] Trigger word is send.”, it's worth noting that during the validation or testing phase, the trigger word embedded in x is the predicted word obtained by the trigger recognizer, while during the training phase, is the true trigger label that ensures the prompt model can learn accurate information.

[0070] Then, given a set of event labels Y = {y1, y2, ..., y...} N Based on the generated event embedding e0, the probability of a candidate event type e∈Y is denoted as P. e :

[0071]

[0072] Where, p i (i = 1, 2, ..., N) represents the event label y. i The predicted probability, The target event is labeled. To train the event classifier, we use cross-entropy loss as the optimization objective:

[0073]

[0074] In this context, the real event label is represented as a one-hot vector. In summary, the total loss function of MsPrompt can be expressed as:

[0075] L=αL t +βL e ,

[0076] in, These are the adjustable parameters for the trigger word recognizer and the event classifier, respectively.

[0077] In the trigger word recognizer and event classifier, the above embodiments employ various prompt templates. As clues, target trigger words and events are detected through a masked language modeling process. However, "trigger word" and "event" remain difficult for pre-trained language models to understand. Therefore, this embodiment introduces a knowledge-enhanced ontology to expand the semantics of these keywords and their relevance to the current task, in order to better extract potential knowledge from the language model.

[0078] Specifically, for the trigger word recognizer, this embodiment adds an ontology text "Trigger word: a word that can trigger an event, usually a verb or noun in the sentence." after the event text, further explaining the meaning of "trigger word." Similarly, in the event classifier, another ontology text "Event: which type the sentence or trigger belongs to." is placed between the event text and the trigger word, which helps to clarify the target of event classification for the model.

[0079] In step (3), due to insufficient generalization caused by severe trigger bias, classifying sparse labels becomes more challenging in truly small-sample settings. Therefore, when generating the probability distribution of predicted event labels in the above embodiments, this application does not use the traditional method of mapping the embedding e0 from the vocabulary of a large-scale pre-trained language model to the label space through an expressor. Instead, it introduces a prototype network module that clusters all instances to obtain the centroid of each cluster as a representation of the event label. This module has strong generalization ability, can reduce the interference of edge instances in dense clusters, i.e., noise of high-frequency labels, and effectively capture the inter-class relationships and spatial information of all samples in sparse clusters, thereby improving the recognition ability of low-frequency labels.

[0080] Specifically, given the d-dimensional event embeddings e0 obtained from the event classifier, we measure the distance between the event embeddings using a Euclidean metric. The cluster centroids are further computed as label representations. All of these form an event vector space E = {e1, e2, ..., e} N This is a prototype network. Therefore, the predicted probability... It can be represented as:

[0081]

[0082] In the formula, Let represent the Euclidean distance. Then, we obtain the probability distribution in the equation and the event type with the highest predicted probability, which can be regarded as the centroid closest to e0 in the prototype network.

[0083] The following uses the ACE-2005 and FewEvent datasets as examples to evaluate the bias-reduced few-shot event detection method based on multi-step cue learning provided in the above embodiments.

[0084] The statistics for ACE-2005 and FewEvent are shown in Table 1. Since the average trigger word length in both datasets is close to 1, in this embodiment, the trigger recognizer is set to select one word for each input.

[0085] Table 1: Statistics for ACE-2005 and FewEvent.

[0086]

[0087] We evaluate our model and baseline under truly small sample settings with different sample sizes K∈{4, 8, 16, 32}, and then we obtain a K-sample training set X. train and validation set X valid The test set and other examples are shown in Table 2. Note that if the total number of texts for an event type is less than or equal to 2K and insufficient to split the training / validation / test sets in our truly small sample setting, that event type will be ignored during sampling. On a full sample dataset, we set the training / validation / test set ratio to 8:1:1.

[0088] Table 2: Training, validation, and test sets for k samples and the full sample.

[0089]

[0090] We sample from an initial dataset on a fixed random seed of 42 and use bert-base-uncased to obtain a representation of the text for each event. The equation is L = αL. t +βL e The parameters α and β are set to 1. The prototype network E is first randomly initialized, and the network parameters are updated synchronously with the optimization objective and cue model. The batch sizes for training and testing are set to 32 and 128, respectively. The number of iterations for ACE-2005 and FewEvent are 500 and 100, respectively, with learning rates of 1e-3 and 1e-2, and the decay weights for bert-base-uncased are 1e-6 and 1e-5, respectively. Furthermore, the training process is terminated according to an early stopping strategy when the model loss no longer decreases after 1000 iterations. In the above embodiment, MsPrompt is implemented under several random seeds to average the results, including accuracy, weighted precision, weighted recall, and weighted F1 score. For simplicity, we omit "weighted" in the tables below. Additionally, the metrics used in Tables 3 and 4 are weighted F1 scores.

[0091] First, a baseline is established based on different metric learning methods: to ensure comparability of results, we used the same encoder in the proposed model. [CLS] is used to obtain the embeddings of the input text, including the trigger word and the event in sequence. The event embeddings are then fed into different classifiers to map to event labels. The number of iterations for ACE-2005 and FewEvent are 500 and 50, respectively. Other parameters are consistent with the proposed model MsPrompt.

[0092] Neural-Bert: A fully connected layer is applied to map the embedded output from the hidden layer to the event label space, thus obtaining the embedded label probability distribution.

[0093] Relation-BERT follows the concept of relational networks for few-shot event detection. We concatenate event label features with the current text features obtained from the embedding module, forming a feature mapping connection. Finally, we obtain the relationship score between the text to be predicted and each event label through the relation module. The embedding module and relation module used in this paper are a prototype network and a single fully connected layer, respectively.

[0094] Proto-Bert: It uses a prototype network as an event classifier to calculate the Euclidean distance between the event embedding and each event type, and obtains the nearest event type as the predicted label.

[0095] The following are suggestions-based methods:

[0096] KiPT: Uses T5 as an encoder to obtain soft suggestion templates and introduces an external knowledge base to build knowledge-injected suggestions.

[0097] ZEOP is an advanced event detection method based on cue-based learning and ordered contrastive learning. It employs BERT-base-uncased as the encoder. It extends contrastive learning to construct four types of contrastive samples, effectively enhancing the learning and classification of unseen event labels, and uses cue-based prediction to obtain trigger words and event embeddings, detecting events through a prototype network. Since the model is trained on full data, the same data segmentation operation as our proposed method is used on smaller sample settings.

[0098] This example demonstrates the event detection performance of our model and the baseline on two public datasets, ACE-2005 and FewewEvent. Table 3 shows our evaluation of the model in a small sample setting with K∈{4, 8, 16, 32}.

[0099] Table 3: Overall Performance. The best-performing baseline and the best-performing result in each column are shown underlined and in bold, respectively.

[0100]

[0101]

[0102] Generally, more samples are beneficial for model training in low-resource scenarios. Therefore, results typically perform better in a large k-sample setting. In particular, comparing the results of FewEvent with ACE-2005, we find that almost all models outperform the former. The underlying reasons for this phenomenon are very similar: FewEvent contains more training samples to help the model correctly classify event types.

[0103] Table 3 shows that cue-based learning models generally outperform traditional metric-based learning baselines. Specifically, MsPrompt performs best on ACE-2005 with K=4, 8, 32 and K=4, 16, 32 on the FewEvents, while ZEOP performs well on ACE-2005 with K=16 and on the FewEvents with K=8. Notably, MsPrompt achieves significant improvements over the baseline in most cases. For example, we found that on ACE-2005, MsPrompt outperforms the best baseline on 4, 8, and 32 sample settings by 0.80%, 1.37%, and 0.28%, respectively, and also exceeds it by 3.04%, 1.94%, and 0.11% on the FewEvents with 4, 16, and 32 samples, respectively.

[0104] Table 4: Performance of MsPrompt and ZEOP in extremely low resource scenarios with K sample settings, where K∈{1,2,3,4}.

[0105]

[0106] Furthermore, MsPrompt's performance improvement in extremely low-resource scenarios is more significant than other methods. Compared to KiPT and ZEOP, MsPrompt achieves improvements of 5.80% and 3.04% respectively on the 4-sample setting of FewEvents. In ACE-2005, MsPrompt's improvements relative to KiPT and ZEOP are 0.80% and 1.78% respectively. Therefore, we conducted an extended experiment to evaluate the performance of MsPrompt and the best-performing baseline ZEOP in extremely low-resource scenarios, as shown in Table 4. We can observe that MsPrompt outperforms ZEOP in the few-sample settings of K=1, 2, 3, 4. This can be explained by the outstanding performance of cue learning in low-resource scenarios, namely, exploring the latent knowledge from the pre-trained language model to enhance model training. Compared to the other two cue-based methods, MsPrompt employs a full-process cue model with two consecutive sub-tasks in event detection, maximizing the advantages of cue learning in few-sample learning and fully utilizing the latent knowledge in the pre-trained language model.

[0107] Among the three metric learning models, Proto-Bert generally outperforms Neural-Bert and Relation-Bert on FewEvents, with its performance steadily improving as K increases from 4 to 32. When it comes to ACE-2005, Relation-Bert outperforms the others and is more robust across various K-sample settings. This is likely due to the label space and the size of the training sets in both datasets. ACE-2005 and FewEvents contain event types of 13-30 and 34-100, respectively (see Table 2). By classifying events through clustering in the label space, Proto-Bert is more effective than Relation-Bert in scenarios with more labels and is more sensitive to the size of the training set. We can observe that as the K value increases, Proto-Bert's results increase from 15.20% to 57.13% on ACE-2005 and from 43.09% to 72.95% on FewEvents. This indicates that more instances do indeed improve the clustering and classification performance of Proto-Bert. Therefore, we use the Proto-Bert prototype network as the classifier in our model.

[0108] Table 5: Subtask performance of MsPrompt. Unlike Event Detection (ED), Event Classification (EC) feeds the actual trigger labels into the event classifier, rather than the predicted trigger words.

[0109]

[0110] To discuss the impact of trigger words on event detection, we input predicted trigger words and true trigger labels into the event classifier, respectively, and obtained the classification performance of ED and EC, as shown in Table 5. EC significantly outperforms ED. Specifically, the improvements for K=4, 8, 16, and 32 on FewEvent are 3.89%, 4.25%, 3.64%, and 2.72%, respectively, and the improvement for the full sample is 8.49%. For ACE-2005, EC's improvements on K=4, 8, 16, 32, and the full sample are 7.04%, 4.26%, 7.00%, 4.08%, 4.08%, and 9.63%, respectively. Under the same data settings, on FewEvent, EC's improvement over ED is weaker than ACE-2005, which may be determined by the size of the event labels. FewEvent contains more event labels, making it less sensitive to improvements in event classification caused by correct trigger words.

[0111] Due to X test Still affected by trigger bias, the fairest evaluation should be based on an unbiased test set. Therefore, under TUS, we uniformly sample instances for each trigger word of an event type. Building on TUS, under COS, we select as many obfuscated trigger words as possible that are similar to those of other event types. This is compared to the biased full test set X. test Compared to the dataset after IUS (uniform sampling for each event type), the dataset after TUS and COS sampling methods eliminates trigger biases that could lead to performance exaggeration. When evaluating MsPrompt using sampling methods such as IUS, TUS, and COS, to ensure model performance, [further details are needed]. Figure 3 In our evaluation, we assessed the debiasing performance of IUS, TUS, COS, and the full test set, as well as the best-performing baseline ZEOP.

[0112] like Figure 4 As shown, we observed that on IUS, the uniform test set of events outperforms the initial full test set X with a long-tailed distribution of event types. test Therefore, it can be inferred that datasets with long-tailed label distributions limit the performance of small-sample event detection. Compared to ZEOP, our model MsPrompt performs well not only on imbalanced datasets but also on datasets with uniform events, reflecting excellent robustness.

[0113] For the unbiased test sets TUS and COS, the results on ACE-2005 and FewEvent showed varying degrees of decline compared to IUS and the full test set, with the decline being more severe on COS. This confirms that the trigger bias in the dataset causes the event detection model to rely too heavily on these trigger cues, while the actual performance of small-sample event detection is overestimated. Furthermore, MsPrompt consistently outperforms ZEOP on TUS and COS, indicating that MsPrompt still offers strong and excellent performance improvements in fair debiasing scenarios. This improvement is more evident on COS, with improvements of approximately 5% and 10% on ACE-2005 and FewEvent, respectively, suggesting that it is better at classifying MsPrompt's confusing trigger words than ZEOP. This is because MsPrompt truly focuses on context, rather than misclassifying confusing trigger words by relying on trigger bias.

[0114] Furthermore, compared to the results on the full test set, MsPrompt's performance on TUS fluctuates by no more than 5%, even showing a slight improvement over ACE-2005, demonstrating MsPrompt's outstanding performance in deferring biased few-shot event detection. This can be explained by the fact that MsPrompt utilizes supplementary ontology text and cue learning to model the entire event text, and fully leverages prior knowledge in the pre-trained language model, effectively overcoming the problems of context bypassing and poor generalization caused by trigger bias.

[0115] To discuss the impact of input length, we compared the performance of MsPrompt and the best-performing baseline ZEOP in a 32-sample setting. Since the average length of the event text in Table 1 is 27.32 on ACE-2005 and 32.78 on FewEvent, we selected five equally spaced length intervals: (10, 20], (20, 30], (30, 40], (40, 50], and (50, 60). For example, the test instance “(10, 20]” represents a sample with a text length between 10 and 20. Therefore, we obtained five segmented test sets from the corresponding 32-sample test sets on ACE-2005 and FewEvent, respectively. The results are as follows... Figure 4 As shown.

[0116] Figure 5 The overall trend is that the longer the input length, the lower the model performance on both MsPrompt and ZEOP datasets. This is because longer sentences are more complex and their semantics are more difficult to understand. Simultaneously, as text length increases, the input carries more noise and invalid information, gradually increasing the challenge for triggering the recognizer, which affects the overall performance of event detection. This decline is more pronounced on FewEvent, which contains more difficult-to-distinguish event labels.

[0117] Furthermore, in ACE-2005, compared to ZEOP, MsPrompt initially performed slightly worse within the length range (10, 20], but then surpassed it at larger length ranges (20, 30], (30, 40], (40, 50], and (50, 60], where its advantage became more pronounced. This trend also exists in FewEvent, where MsPrompt outperforms ZEOP starting from length intervals of (30, 40]. Therefore, we can observe that MsPrompt has an advantage over ZEOP with long inputs. Moreover, as sentence length and complexity increase, MsPrompt... MPT demonstrates a significant advantage in few-shot event detection. This is thanks to the excellent semantic modeling capabilities of Masked Language Modeling (MLM). Even in long event texts, the trigger recognizer can more accurately identify key information and trigger words. The event classifier can also effectively improve event detection performance by combining long event text, ontology text, and identified trigger information. Notably, supplementing the ontology text enables the model to understand the current event detection task more quickly, such as understanding the definitions of trigger words and events. This helps the model effectively avoid interference from invalid information in long event texts and better grasp the key information of the text.

[0118] To investigate the impact of input sequences on few-sample event detection, we evaluated all combinations of input sequences in the trigger recognizer and event classifier in Table 6.

[0119] Table 6: Impact of Input Sequences on 32 Samples. “M”, “O”, and “T” represent event text, ontology text, and trigger words, respectively. Results from MsPrompt are shown in bold. ▼ indicates the column with the largest performance degradation compared to MsPrompt.

[0120]

[0121]

[0122] For the trigger recognizer, when the order of the event text and the body text was changed to "O+M", the accuracy of trigger recognition decreased by 8.47%, which was more significant than the decrease of other order combinations in the event classifier. At the same time, the performance of event detection also decreased to varying degrees. Specifically, the F1-score was 3.17% lower than the default order combination in MsPrompt, namely "M+O".

[0123] For event classifiers, different combinations of the order of event text, ontology text, and trigger words have a negligible impact on the accuracy of trigger word recognition, fluctuating within a range of 1%. This minimal impact stems from the formula L = αL. t +βL eThe joint optimization of trigger loss and event loss was performed. However, changing the order caused a significant performance drop in event detection. Compared to the default order of the event classifier in MsPrompt, "M+O+T", the weighted f1 scores of other order combinations in Table 6 decreased by 12.06%, 4.25%, 52.69%, 41.11%, and 52.65%, respectively. Among them, the "O+T+M" combination achieved the largest decrease.

[0124] It's worth noting that when the event text is placed at the very end of the input, the performance of both the "O+T+M" and "T+O+M" combinations drops significantly by more than 50%. This directly reflects the dominant role of the event text in event detection performance, while the trigger words and ontology text play a supplementary role. This phenomenon suggests that in event detection, we need to pay more attention to the overall semantic information in event mentions, rather than using simple trigger word recognition and classification to encompass the event information included in the sentence.

[0125] To verify the contribution of different modules in MsPrompt to event detection performance, we conducted an ablation study using our model in a 32-sample setting from ACE-2005. In the ablation study, we removed three specific modules to explore their impact on MsPrompt: "-trigger recognizer," "-event classifier," and "-ontology text." "-trigger recognizer" and "-event classifier" represent ignoring the trigger recognizer and event classifier modules, respectively. Correspondingly, we used [CLS] to directly perform semantic modeling on the original event text and obtain the predicted trigger words or event types. "-ontology text" refers to removing the embedded ontology text from both the trigger recognizer and event classifier modules. The ablation results are shown in Table 7.

[0126] Table 7: Ablation studies with 32 samples. "-" indicates that the module has been removed from our suggestion message. Results from MsPrompt are underlined. ▼ indicates the column with the largest performance degradation compared to MsPrompt.

[0127]

[0128] As shown in Table 7, removing the trigger recognizer resulted in the most severe drop in trigger recognition performance across the entire model, with accuracy decreasing from 66.80% to 65.32%, a drop of 1.48%. This decline confirms the effectiveness of the trigger recognizer and its indispensability to the MsPrompt model. Removing the event classifier module led to a sharp decrease in event detection performance, specifically a 6.77% drop in the weighted F1 score. Clearly, the event classifier plays a significant role in improving the performance of few-shot event detection. When switching to "-ontology text," we observed the largest decrease in all event detection performance metrics, with weighted precision, recall, and F1 score decreasing by 6.87%, 8.56%, and 10.17%, respectively. This fully demonstrates the significant contribution of ontology text to few-shot event detection, enabling the prompting model to quickly learn the event detection task in low-resource scenarios and truly leveraging human prior knowledge to guide the training of the pre-trained language model.

[0129] To explore our model's ability to mitigate the context bypassing problem caused by trigger bias, we investigated cases where predicted trigger words matched true trigger labels. For example, in ACE-2005, events mentioning the trigger word "war" were almost all labeled with the event type "Conflict:Attack". However, in MsPrompt, which identified events with the trigger word "war", many other sparse event labels were predicted in addition to the dense event type "Conflict:Attack".

[0130] Although MsPrompt predicts "war" as the trigger word for the events mentioned in the above examples, the event type detection results are different. We can observe that the manually labeled event in the dataset is still "Conflict:Attack," while our model considers the text to belong to the event type "Personnel:Elect." Combining this with the semantic understanding of the original text, we can find that it mainly describes the latter event type, meaning that our model's detection is more accurate.

[0131] In summary, during the construction of event detection datasets, manual event labeling inevitably leads to labeling inertia, where trigger words are habitually categorized into the same event type. This exposes the context bypass problem, which is exacerbated by trigger bias and can result in incorrect predictions. In contrast, our model, MsPrompt, focuses not only on trigger words but also on the original event text in the event classifier. Therefore, it significantly avoids this labeling inertia and greatly mitigates the context bypass problem caused by trigger bias, demonstrating the debiasing effect of our model.

[0132] In summary, the beneficial effects of this invention are as follows: by setting up an undersampling module, a multi-step prompting module, and a prototype module, a new training set is constructed through the undersampling module to adapt to real small sample settings and scenarios with insufficient data; by equipping the multi-step prompting module with a knowledge-enhanced ontology text, the event semantics and potential prior knowledge in the pre-trained language model are fully utilized to solve the context bypass problem; and the prototype module compensates for the weakness of event classification due to data scarcity, thereby improving generalization performance.

[0133] This application also provides a partial few-shot event detection device based on multi-step prompting learning, including an undersampling module, a multi-step prompting module, and a prototype network module. The undersampling module distinguishes the input initial data to form a training set, a validation set, and a test set. The multi-step prompting module obtains predicted trigger words from the event text input to the training set, validation set, and test set in the undersampling module, and classifies the event text. The prototype network module clusters the classified event text output by the multi-step prompting module to obtain the centroid of each cluster as the representation of the event label.

[0134] Those skilled in the art should understand that the discussion of any of the above embodiments is merely exemplary and is not intended to imply that the scope of protection of this application is limited to these examples; under the concept of this application, the technical features of the above embodiments or different embodiments can also be combined, the steps can be implemented in any order, and there are many other variations of different aspects of one or more embodiments of this application as described above, which are not provided in detail for the sake of brevity.

[0135] One or more embodiments in this application are intended to cover all such substitutions, modifications, and variations that fall within the broad scope of this application. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of one or more embodiments in this application should be included within the protection scope of this application.

Claims

1. A method for detecting biased few-shot events based on multi-step cue learning, characterized in that, The method includes the following steps: (1) Select the initial dataset, divide the initial dataset into a data matrix that conforms to the small sample training settings, select instances from each preset event type and distinguish them to form a training set, a validation set and a test set; (2) Perform multi-step processing on the event texts of the training set, the validation set and the test set, that is, execute two sub-tasks simultaneously and train two consecutive sub-cues in one iteration, identify the event text and obtain the predicted trigger word in one of the sub-tasks, and classify the event text according to the trigger word in the other sub-task. Step (2) further includes modifying the event text of the training set, the validation set, and the test set into a prompt function, the specific form of which is: in The prompt template is a manually constructed prompt template. Includes a mask location [MASK]; [CLS] is the event text, [CLS] is the category instruction, and [SEP] is the separator. In step (2), obtaining the predicted trigger word specifically includes: given an event text, by mapping the massive vocabulary of a large-scale pre-trained language model to the original text, the probability distribution of the trigger word is obtained, and the candidate word with the highest probability is the predicted trigger word; The probability distribution formula for the trigger word is as follows: in, This is the hidden layer vector of [MASK], where t is the trigger word. The input is a sequence of event text. The prompt provided by the prompt function , For event text sequence The candidate word with the highest probability selected from within; In step (2), the method further includes setting additional knowledge-enhanced ontology text, which specifically includes adding an ontology text after the event text after recognizing the event text and obtaining the predicted trigger word, thereby further explaining its meaning; In event classification, another ontology text is placed between the event text and the predicted trigger word, thereby clarifying the event classification target of the event text and the predicted trigger word; (3) Cluster the event texts after each classification to obtain the centroid of each cluster as the representation of the event label, output the probability of each event type, and the event type with the highest probability is the detected event label; In step (3), the output probability for each event type specifically includes: Given a set of event labels Based on the generated d-dimensional event embedding Candidate event types The probability of is expressed as : in, For event tags The predicted probability, The prompt provided by the prompt function .

2. The method for detecting biased few-shot events based on multi-step cue learning as described in claim 1, characterized in that, In step (1), the step of selecting instances from each event type and distinguishing them to form a training set, a validation set and a test set specifically includes: randomly sampling K instances under each event type in the initial dataset to form a K-sample training set. The length of the K-sample training set is N*K, where N is the size of the event label space in the original dataset. Repeat this process to generate a K-sample validation set that does not overlap with the training set from the remaining unsampled instances; The remaining unsampled instances form the K-sample test set.

3. The method for detecting biased few-shot events based on multi-step cue learning as described in claim 1, characterized in that, In step (2), the event classification specifically includes classifying event text and trigger words into predefined event types.

4. The method for detecting biased few-shot events based on multi-step cue learning as described in claim 1, characterized in that, In step (3), the clustering of the classified event texts to obtain the centroid of each cluster as the representation of the event label, and the output of the probability of each event type specifically include: Given a d-dimensional event embedding obtained from an event classifier The distance between event embeddings is measured using Euclidean metric; the cluster centroids are further calculated as label representations. All of these form an event vector space. That is, a prototype network, from which the predicted probability is obtained. The formula is as follows: in For Euclidean distance.

5. A bias-reducing few-shot event detection device based on multi-step cue learning, characterized in that, include: Undersampling module, multi-step prompting module, and prototype network module; The undersampling module divides the input initial dataset into a data matrix that conforms to the few-sample training settings, selects instances from each preset event type, and distinguishes them to form a training set, a validation set, and a test set; The multi-step prompting module performs multi-step processing on the event text of the training set, the validation set, and the test set. That is, in one iteration, two sub-tasks are executed simultaneously and two consecutive sub-prompts are trained synchronously. In one of the sub-tasks, the event text is identified and the predicted trigger word is obtained. In the other sub-task, the event text is classified according to the trigger word. It also includes modifying the event text of the training set, the validation set, and the test set into prompt functions, the specific form of which is: in The prompt template is a manually constructed prompt template. Includes a mask location [MASK]; [CLS] is the event text, [CLS] is the category instruction, and [SEP] is the separator. The specific steps to obtain the predicted trigger word include: given an event text, mapping the massive vocabulary of a large-scale pre-trained language model to the original text to obtain the probability distribution of the trigger word, and the candidate word with the highest probability is the predicted trigger word. The probability distribution formula for the trigger word is as follows: in, This is the hidden layer vector of [MASK], where t is the trigger word. The input is a sequence of event text. The prompt provided by the prompt function , For event text sequence The candidate word with the highest probability selected from within; It also includes: setting additional knowledge-enhanced ontology text, specifically including: adding an ontology text after the event text after recognizing the event text and obtaining the predicted trigger word, thereby further elaborating on its meaning; In event classification, another ontology text is placed between the event text and the predicted trigger word, thereby clarifying the event classification target of the event text and the predicted trigger word; The prototype network module clusters the classified event texts to obtain the centroid of each cluster as the representation of the event label, and outputs the probability of each event type. The event type with the highest probability is the detected event label. The probability of each event type output specifically includes: Given a set of event labels Based on the generated d-dimensional event embedding Candidate event types The probability of is expressed as : in, For event tags The predicted probability, The prompt provided by the prompt function .

Images