A prompt fine-tuning based event detection method
By using a cue-based fine-tuning approach, and leveraging the BART encoder and generator to construct learnable soft cues, the problem of semantic information ignoring in existing methods is solved, resulting in more efficient event detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- NANJING UNIV OF SCI & TECH
- Filing Date
- 2023-12-04
- Publication Date
- 2026-07-31
AI Technical Summary
Existing event detection methods ignore the semantic information in event type labels, and the discrete manual prompts used are not suitable for the current task, resulting in poor detection performance.
We employ a cue-based fine-tuning approach. By constructing learnable soft cues and combining them with a BART encoder and generator, we train a multi-label classifier. We then utilize the semantic information of event type labels to optimize the loss weights and generate event trigger words.
It improves the accuracy and adaptability of event detection, surpassing traditional generative event detection schemes, and is better able to identify event trigger words and determine event types.
Smart Images
Figure CN117609699B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to an event detection method, and more particularly to an event detection method based on cue-based fine-tuning. Background Technology
[0002] As the primary carrier of information, the importance of text to human society is self-evident. With the advent of the big data era, relying solely on human reading comprehension speed to locate the information we need from a vast sea of text is extremely tedious and redundant. Therefore, accurately and quickly extracting valuable information from text (information extraction) is crucial. As a subtask of information extraction, the event detection task is defined as extracting the type of event and event trigger words (words that indicate the occurrence of an event) from unstructured text.
[0003] In the research and practice in this direction, most traditional event detection methods are based on sequence labeling schemes, that is, judging whether each word in the text is a trigger word for the event. Although this method is simple, it often ignores the semantic information contained in the event type label. At the same time, with the rise of the cue learning paradigm, the cue usually contains event type label information. Therefore, some researchers have constructed cue templates to complete the event detection task. However, these methods basically use discrete and fixed manual cues, which cannot guarantee that they are the most suitable for the current task. Summary of the Invention
[0004] Purpose of the invention: The technical problem to be solved by the present invention is to provide an event detection method based on prompting and fine-tuning, which addresses the shortcomings of the prior art.
[0005] To address the aforementioned technical problems, this invention discloses an event detection method based on cue-based fine-tuning, comprising the following steps:
[0006] Step S1: Text preprocessing, splitting the original dataset into training set, validation set and test set, and converting the input original text dataset into an input form that conforms to the BART model;
[0007] Step S2: Build a classification model and train a classifier. By obtaining the encoding end hidden layer representation of the original text, train a multi-label classifier to determine the event type to which the text belongs.
[0008] Step S3: Construct learnable cues by introducing a BART encoder to convert the original discrete cue information into learnable cues;
[0009] Step S4: Construct a generative model and train the generator. The input to the encoder of the generator is the embedded representation of the original text, and the input to the decoder is the learnable prompt obtained in step S3. The two parts together guide the generative model to generate the trigger word of the event.
[0010] Step S5: Joint training. During the training phase, the classification model and the generative model are trained simultaneously, and the above models are further optimized by dynamically adjusting the loss weights.
[0011] Step S6: Using the trained model described above, detect the input text to complete the event detection based on prompts and fine-tuning.
[0012] Furthermore, the text preprocessing described in step S1 specifically includes:
[0013] Step S11: Data Segmentation and Reading: First, the dataset is divided into three parts: training set, validation set and test set according to the ratio of 8:1:1. Then, the files in the input original text dataset are parsed to obtain the original text, event type label, event trigger word label, target text to be generated and event type prompt template for each sample.
[0014] Step S12: Data cleaning, removing redundant non-text content from the original text;
[0015] Step S13: Data labeling. Preset tags are added to the event trigger words of the sample at a preset ratio. Specifically, trigger word start tags are added before and after the event trigger words. <trigger>and trigger word end marker< / trigger> .
[0016] Furthermore, the construction of the classification model and training of the classifier described in step S2 specifically includes:
[0017] Step S21: Encode each original text using a word segmenter to obtain a text tag sequence and text attention tags;
[0018] Step S22: Use the text tag sequence and text attention tags together as input to the BART encoder to obtain the hidden layer representation of the sentence in the original text;
[0019] Step S23: Construct a multilayer perceptron, map the hidden layer representation vector of the acquired sentence to the label dimension, where the label dimension is the total number of event types for each sample in the original text dataset, and perform cross-entropy operation with the event type labels described in step S11 to obtain the classification loss L1 for subsequent training.
[0020] Furthermore, the original discrete prompt information mentioned in step S3 includes at least: a description of the event type and a description of the trigger word.
[0021] Furthermore, the construction of learnable cues described in step 3 specifically includes:
[0022] Step S31: Based on the event type cue templates described in step S11, each event type cue template is encoded into a cue template tag sequence and a cue template attention tag using a word segmenter;
[0023] Step S32: Feed the cue template tag sequence into another BART encoder to obtain the context representation of the cue template part, which is the cue that can be learned.
[0024] Furthermore, step 4, which involves constructing the generative model and training the generator, specifically includes:
[0025] Step S41: The generative model adopts the BART framework, wherein the encoding end is input with the text tag sequence and text attention tag obtained in step S21, and the decoding end is input with the learnable cue obtained in step S32 and the cue template attention tag obtained in step S31.
[0026] Step S42: Obtain the generation result of the generative model and calculate the cross-entropy with the target text to be generated as described in step S11 to obtain the generation loss L2 for subsequent training.
[0027] Furthermore, the generative model described in step S41 adopts the BART framework, specifically including:
[0028] The generation model consists of two parts: an encoder and a decoder. The encoder is used to encode and model the input sequence, and the decoder is used to generate the target sequence based on the output of the encoder.
[0029] Furthermore, the joint training described in step 5 specifically includes:
[0030] Step S51: Add the classification loss L1 and the generation loss L2 together with weights w1 and w2 respectively;
[0031] Step S52: After each training round on the training set, recalculate the classification loss L1' and generation loss L2' on the validation set. Adjust w1 and w2 according to the ratio of w1*L1' and w2*L2' on the current validation set so that the values of w1*L1' and w2*L2' are equal.
[0032] Step S53: Based on the backpropagation of the above loss, iterate continuously until the maximum number of iterations is reached to complete the joint training.
[0033] Furthermore, the event type label mentioned in step S11 is in the form of a unique hot code label.
[0034] Furthermore, the initial values of the weights w1 and w2 mentioned in step S51 are both 1.
[0035] Beneficial effects:
[0036] The method proposed in this invention, while incorporating semantic information from event type labels, combines a cue fine-tuning strategy from current cue learning paradigms to transform original discrete cueing into learnable soft cueing, ultimately yielding task-appropriate cues. Experimental results on relevant datasets demonstrate that this invention outperforms most generative event detection schemes, further proving its feasibility. Attached Figure Description
[0037] The present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments, and the advantages of the present invention in the above and / or other aspects will become clearer.
[0038] Figure 1 This is a flowchart of the overall process of model training.
[0039] Figure 2 This is a diagram showing the experimental results of the present invention. Detailed Implementation
[0040] This invention provides a method for event detection that can fully utilize event label semantics while training learnable soft cues, namely, a cue-based fine-tuning event detection method, which specifically includes the following steps:
[0041] Step S1: Text preprocessing, converting the raw text into an input format that conforms to the pre-trained language model.
[0042] Step S2: Classifier training. A multi-label classifier was trained by obtaining the encoding-end hidden layer representation of the original text to determine the event type to which the text belongs.
[0043] Step S3: Construction of learnable cues (soft cues) introduces an additional BART Encoder (BART: a bidirectional autoregressive transformer model, consisting of an encoder and a decoder. The encoder acquires the text representation, which serves as the input to the decoder; the decoder generates the target text based on the encoder's output and the label input. Reference: Lewis M, Liu Y, Goyal N, et al. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[J]. arXiv preprint arXiv:1910.13461,2019.), which transforms the original discrete cue information into learnable cues.
[0044] Step S4: Generator training. The input to the encoder is the embedded representation of the original text, and the input to the decoder is the obtained soft cue. Both parts jointly guide the model to generate event trigger words.
[0045] Step S5: Joint training. During the training phase, the classification model and the generative model are trained simultaneously, and further optimization is achieved by dynamically adjusting the loss weights.
[0046] The BART model mentioned in the article is a bidirectional and auto-regressive transformer model.
[0047] Step S1 includes:
[0048] Step S11: Data cleaning, removing redundant content that is not part of the main text.
[0049] Step S12: Read the data, parse the original JSON file in the dataset, and obtain relevant information such as the original text, event type, event trigger words, target to be generated, and event type prompt template for each sample.
[0050] Step S13: Data annotation. A special marker is added to the trigger words of the original text at a certain ratio (adding "" before and after the trigger word). <trigger> "and"< / trigger> This improves the model's ability to identify trigger words and determine event types.
[0051] Furthermore, in step S11, the data mainly comes from publicly available event extraction task datasets, such as ACE2005, ERE, wikiEvent, etc., which contain a large amount of raw corpus and event labels.
[0052] Furthermore, in step S12, considering that some samples contain multiple event types (each event type corresponds to a different trigger word), these samples are repeated n times (n is the number of event types contained in the sample) when processing such samples, and are regarded as n different samples.
[0053] Furthermore, in step S13, a special label is applied to a certain proportion of the trigger words in the original text of the training set, which will improve the model's ability to identify trigger words and determine event types. Through comparative experiments, this invention has found that the optimal labeling ratio is 20%. It should be noted that partial labeling is used during training, but during the inference and testing phase, the data in the test set is all unlabeled raw data.
[0054] Step S2 includes:
[0055] Step S21: Encode each text using a tokenizer to obtain the token sequence input_ids and its attention tag attention_mask;
[0056] The tag sequence input_ids: queries the position of each word in the vocabulary and returns its index value as its tag. Each tag corresponds to a unique identifier in the vocabulary.
[0057] Attention mask: A mask sequence of the same length as input_ids, used to indicate which locations are the true input and which are padding. Its purpose is to tell the model which locations to focus on during self-attention.
[0058] Step S22: Use the labeled sequence input_ids and attention label attention_mask as input to the BARTEncoder to obtain the hidden layer representation of the sentence.
[0059] Step S23: Construct a Multilayer Perceptron (MLP, a basic feedforward artificial neural network model consisting of multiple fully connected layers), map the obtained hidden layer representation vectors to the label dimension (the label dimension is the total number of event types in the dataset), and perform cross-entropy calculation with the one-hot labels to obtain the classification loss L1.
[0060] Furthermore, in step S23, the selection of event one-hot code labels and loss functions is explained in detail as follows: Considering that some texts do not contain event information, and some texts contain two or more event types, it is not appropriate to use the most common multi-class classification loss. Therefore, a multi-label classification loss function is adopted: Taking the ACE2005 dataset as an example, there are 33 event types in this dataset, and the label dimension is 33. If a text contains a certain event type, then the dimension of that event type is set to 1, and the rest are set to 0. Event type labels are designed in this form of one-hot encoding.
[0061] Step S3 includes:
[0062] Step S31: Based on the original event type cue templates, the tokenizer encodes each event type cue template into a token sequence template_input_ids and its attention tag decoder_attention_mask (which is essentially the same as input_ids and attention_mask in step S21).
[0063] Step S32: Feed the prompt template tag sequence template_input_ids into another BART Encoder to obtain the context representation of the prompt template part. This part is also trainable, that is, a learnable soft prompt, which will be used as the input decoder_embedding to the subsequent decoding end (extending the original tag sequence representation to a higher-dimensional feature space to obtain a denser word representation, so this is a higher-dimensional vector) to guide the generative model to generate the desired target text.
[0064] Furthermore, in step S31, the prompt template is manually initialized, specifically including: a description of the event type, a description of the trigger words, and the target text to be generated. Through ablation experiments, this invention has found that these three parts in the initial template are all crucial; the absence of any one of them will affect the final performance.
[0065] Furthermore, in step S32, the present invention found through comparative experiments that using BART Encoder to construct soft cues is better than using Long Short-Term Memory Neural Network LSTM (LSTM (Long Short-Term Memory), a commonly used Recurrent Neural Network (RNN) architecture specifically designed to process and model long-term dependencies in time series data), because BART Encoder can obtain richer contextual representation information from the cue template.
[0066] Step S4 includes:
[0067] Step S41: Throughout the BART framework, the encoder receives the input_ids of the original labeled sequence and its attention label_mask, while the decoder receives the learnable soft cue decoder_embedding and the cue template attention label decoder_attention_mask.
[0068] Step S42: Obtain the logits value of the generative model and calculate the generation loss L2 with the target text to be generated.
[0069] Furthermore, the target text in step S42 is explained as follows: The final target to be generated is "Eventtrigger is..." <trigger>."in," <trigger>The location is replaced with the trigger word from the label. If a sample has no event / no trigger word, then the generated target text here is "Event trigger is..." <trigger>If a sample contains multiple trigger words, use "and" to connect them, i.e., "Event trigger is...". <trigger1>and <trigger2>.".
[0070] Step S5 includes:
[0071] Step S51: After obtaining the classification loss L1 and the generation loss L2, sum them by weight, with initial weights w1 and w2 both being 1.0.
[0072] Step S52: In each training round, calculate the classification losses L1' and L2' on the validation set. Adjust w1 and w2 based on the ratio of w1*L1' and w2*L2', making the ratio as close to 1 as possible.
[0073] Step S53: Backpropagate the error and iterate continuously until the maximum number of iterations is reached.
[0074] Furthermore, in step S53, in each iteration, the performance of the model is evaluated on the validation set, the evaluation index of event detection is calculated, and the model with the highest evaluation index is saved.
[0075] Example:
[0076] This embodiment describes in detail the process of feeding a text into the model of this invention for training under an experimental environment consisting of an Ubuntu 18.04 operating system, a Python 3 programming environment, an Intel Core i7-9700 CPU, 32GB of memory, and an RTX 2070 GPU.
[0077] like Figure 1 As shown, an event detection method based on cue fine-tuning includes the following steps:
[0078] Step S1: Text preprocessing, converting the original text into an input format that conforms to the pre-trained language model; including:
[0079] Step S11: Data cleaning, removing redundant content that is not part of the main text.
[0080] Step S12: Read the data and parse the original JSON file in the dataset to obtain the original text, event type, event trigger words, target to be generated, event type prompt template, and other related information for each sample. The sentence in the figure contains two event types, "Personal-End-Position" and "Personal-Start-Position". Assuming that these two event types are class 0 and class 1 respectively, the event type label can be represented as [1,1,0,0,0,...0] (dimension 33).
[0081] The example in the image shows the extraction of the trigger word "leaving" under the event type "Personal-End-Position". Therefore, the corresponding prompt template (event type description + trigger word description + generation goal) is "The event is related to a person stops working for an organization or a hiring manager". <sep>leaving indicates Personal_End_Position event happened. <sep>The event trigger is leaving. <sep>The colon ("") serves as the sentence separator. Correspondingly, the target generated text is "Eventtrigger is leaving."
[0082] Step S13: Data annotation. A special marker is added to the trigger words of the original text at a certain ratio (adding "..." before and after the trigger word). <trigger> "and"< / trigger> This improves the model's ability to identify trigger words and determine event types. For example, the annotated data in the image reads, "Davies is [Trigger]leaving[ / Trigger]to[Trigger]become[ / Trigger]chairman of the London School...."
[0083] Step S2: Classifier training. A multi-label classifier was trained by obtaining the encoding-end hidden layer representation of the original text to determine the event type to which the text belongs; including:
[0084] Step S21: Encode each text using a tokenizer to obtain the token sequence input_ids and its attention tag attention_mask.
[0085] Step S22: Use the labeled sequence input_ids and attention label attention_mask as input to the BARTEncoder to obtain the hidden layer representation of the sentence.
[0086] Step S23: Construct a Multilayer Perceptron (MLP), map the acquired hidden layer representation vectors to the label dimension (the label dimension is the total number of event types in the dataset, which is 33 here), and perform cross-entropy calculation with the one-hot labels to obtain the classification loss L1.
[0087] Step S3: Construction of learnable cues (soft cues), introducing an additional BART Encoder to transform the original discrete cue information into learnable cues; including:
[0088] Step S31: Based on the original event type cue templates, the tokenizer encodes each event type cue template into a token sequence template_input_ids and its attention tag decoder_attention_mask.
[0089] Step S32: Feed the prompt template tag sequence template_input_ids into another BART Encoder to obtain the context representation of the prompt template part. This part is also trainable, that is, a learnable soft prompt, which will be used as the input decoder_embedding to the subsequent decoding end to guide the generation model to generate the required target text.
[0090] Step S4: Generator training. The input to the encoder is the embedded representation of the original text, and the input to the decoder is the obtained soft cue. Both parts jointly guide the model to generate event trigger words; including:
[0091] Step S41: Throughout the BART framework, the encoder receives the input_ids of the original labeled sequence and its attention label_mask, while the decoder receives the learnable soft cue decoder_embedding and the cue template attention label decoder_attention_mask.
[0092] Step S42: Obtain the logits value of the generative model and calculate the generation loss L2 with the target text to be generated.
[0093] Step S5: Joint training, during the training phase, the classification model and the generative model are trained simultaneously, and further optimized by dynamically adjusting the loss weights; including:
[0094] Step S51: After obtaining the classification loss L1 and the generation loss L2, sum them by weight, with initial weights w1 and w2 both being 1.0.
[0095] Step S52: In each training round, calculate the classification losses L1' and L2' on the validation set. Adjust w1 and w2 based on the ratio of w1*L1' and w2*L2', making the ratio as close to 1 as possible.
[0096] Step S53: Backpropagate the error and iterate continuously until the maximum number of iterations is reached.
[0097] In summary, this invention has certain significance for event detection tasks: it can fully utilize the semantic information of event type labels, and the designed learnable soft cues can more accurately guide the model to understand the downstream task of event detection. Experiments have also demonstrated the feasibility of this invention, and its performance surpasses most generative event detection schemes, specifically as follows: Figure 2 As shown:
[0098] In the figure, JCG-SPEED is the scheme proposed in this invention. As can be seen, its results (Trigger-I for trigger word recognition and Trigger-C for trigger word classification) have surpassed most generative methods, which fully demonstrates the effectiveness of this scheme.
[0099] In its specific implementation, this application provides a computer storage medium and a corresponding data processing unit. The computer storage medium is capable of storing a computer program, which, when executed by the data processing unit, can run the invention's content regarding a prompt-based fine-tuning event detection method, as well as some or all of the steps in various embodiments. The storage medium can be a magnetic disk, optical disk, read-only memory (ROM), or random access memory (RAM), etc.
[0100] Those skilled in the art will clearly understand that the technical solutions in the embodiments of the present invention can be implemented using computer programs and their corresponding general-purpose hardware platforms. Based on this understanding, the technical solutions in the embodiments of the present invention, or the parts that contribute to the prior art, can be embodied in the form of computer programs, i.e., software products. These computer program software products can be stored in a storage medium and include several instructions to cause a device containing a data processing unit (which may be a personal computer, server, microcontroller, MCU, or network device, etc.) to execute the methods described in various embodiments or certain parts of the embodiments of the present invention.
[0101] This invention provides an idea and method for event detection based on prompting and fine-tuning. Many methods and approaches exist for implementing this technical solution; the above description is merely a preferred embodiment of the invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of this invention, and these improvements and modifications should also be considered within the scope of protection of this invention. All components not explicitly stated in this embodiment can be implemented using existing technologies.< / sep> < / sep> < / sep> < / trigger> < / trigger> < / trigger>
Claims
1. An event detection method based on cue-based fine-tuning, characterized in that, Includes the following steps: Step S1: Text preprocessing, splitting the original dataset into training set, validation set and test set, and converting the input original text dataset into an input form that conforms to the BART model; Then, the files in the input raw text dataset are parsed to obtain the raw text, event type label, event trigger word label, target text to be generated, and event type prompt template for each sample; Step S2: Build a classification model and train a classifier. By obtaining the encoding end hidden layer representation of the original text, train a multi-label classifier to determine the event type to which the text belongs. Step S3: Construct learnable cues by introducing a BART encoder to convert the original discrete cue information into learnable cues; Step S4: Construct a generative model and train the generator. The input to the encoder of the generator is the embedded representation of the original text, and the input to the decoder is the learnable prompt obtained in step S3. The two parts together guide the generative model to generate the trigger word of the event. Step S5: Joint training. During the training phase, the classification model and the generative model are trained simultaneously, and the above models are further optimized by dynamically adjusting the loss weights. Step S6: Using the trained model described above, detect the input text to complete the event detection based on prompting fine-tuning; Specifically, step S2, which involves constructing a classification model and training a classifier, includes: Step S21: Encode each original text using a word segmenter to obtain a text tag sequence and text attention tags; Step S22: Use the text tag sequence and text attention tags together as input to the BART encoder to obtain the hidden layer representation of the sentence in the original text; Step S23: Construct a multilayer perceptron, map the hidden layer representation vector of the acquired sentence to the label dimension, where the label dimension is the total number of event types for each sample in the original text dataset, and perform cross-entropy operation with the event type labels to obtain the classification loss L1 for subsequent training; The original discrete prompt information mentioned in step S3 includes at least: a description of the event type and a description of the trigger word; The construction of learnable cues described in step 3 specifically includes: Step S31: Based on the event type cue templates, each event type cue template is encoded into a cue template tag sequence and a cue template attention tag using a word segmenter; Step S32: Feed the cue template tag sequence into another BART encoder to obtain the contextual representation of the cue template part, which is the cue that can be learned.
2. The event detection method based on cue fine-tuning according to claim 1, characterized in that, The text preprocessing described in step S1 specifically includes: Step S11: Data Segmentation and Reading: First, the dataset is divided into three parts: training set, validation set and test set according to the ratio of 8:1:
1. Then, the files in the input original text dataset are parsed to obtain the original text, event type label, event trigger word label, target text to be generated and event type prompt template for each sample. Step S12: Data cleaning, removing redundant non-text content from the original text; Step S13: Data labeling. Preset tags are added to the event trigger words of the sample at a preset ratio. Specifically, trigger word start tags are added before and after the event trigger words. <trigger> and trigger word end marker< / trigger> .
3. The event detection method based on cue fine-tuning according to claim 2, characterized in that, Step 4, which involves building the generative model and training the generator, specifically includes: Step S41: The generative model adopts the BART framework, wherein the encoding end is input with the text tag sequence and text attention tag obtained in step S21, and the decoding end is input with the learnable cue obtained in step S32 and the cue template attention tag obtained in step S31. Step S42: Obtain the generation result of the generation model and calculate the cross-entropy with the target text to be generated to obtain the generation loss L2 for subsequent training.
4. The event detection method based on cue fine-tuning according to claim 3, characterized in that, The generative model described in step S41 adopts the BART framework, specifically including: The generation model consists of two parts: an encoder and a decoder. The encoder is used to encode and model the input sequence, and the decoder is used to generate the target sequence based on the output of the encoder.
5. The event detection method based on cue fine-tuning according to claim 4, characterized in that, The joint training mentioned in step 5 specifically includes: Step S51: Add the classification loss L1 and the generation loss L2 together with weights w1 and w2 respectively; Step S52: After each training round on the training set, recalculate the classification loss L1' and generation loss L2' on the validation set. Adjust w1 and w2 according to the ratio of w1*L1' and w2*L2' on the current validation set so that the values of w1*L1' and w2*L2' are equal. Step S53: Based on the backpropagation of the above loss, iterate continuously until the maximum number of iterations is reached to complete the joint training.
6. The event detection method based on cue fine-tuning according to claim 5, characterized in that, The event type label is in the form of a unique hot code label.
7. The event detection method based on cue fine-tuning according to claim 6, characterized in that, The initial values of the weights w1 and w2 mentioned in step S51 are both 1.