Open source news data-oriented small sample chapter-level event extraction method and system

Abstract

The invention discloses a small sample chapter-level event extraction method and system oriented to open source news data, and belongs to the technical field of natural language processing. The method comprises the following steps: generating an event extraction task set containing x event extraction tasks, and constructing an event extraction prompt template; performing meta-learning training on the large language model based on the event extraction task set to obtain a pre-trained language model; performing multi-task joint fine tuning on the pre-trained language model by using the event extraction prompt template to obtain a fine-tuned large voice model; and obtaining an event extraction result of the target text based on the fine-tuned large voice model. According to the method, efficient, robust and structured text-level event information extraction can be realized under the condition of scarcity of labeled samples.

Description

Technical Field

[0001] This invention belongs to the field of natural language processing technology, and in particular relates to a method and system for extracting small-sample, chapter-level events from open-source news data. Background Technology

[0002] In current natural language understanding and news data mining tasks, event extraction, as a key step in identifying structured event elements (including event types, trigger words, arguments, and their roles) from unstructured text, has been widely applied in various fields such as public opinion monitoring, news data analysis, and knowledge graph construction. Especially in dynamic text environments such as open-source news data, the rapid and accurate identification and extraction of event information is of great significance for supporting intelligent analysis and decision-making.

[0003] Existing event extraction methods mainly focus on several technical solutions, including supervised learning-based event extraction methods, transfer learning-based event extraction methods, multi-task joint learning and prompt-driven event modeling methods, and few-shot event extraction exploration methods based on meta-learning.

[0004] Supervised learning-based event extraction methods typically employ neural network models (such as BiLSTM-CRF and Transformer) trained on large amounts of manually labeled training data. The models learn contextual semantic features to identify trigger words and their corresponding argument roles. However, these methods are highly dependent on the amount of training data. In practical applications, especially when dealing with low-resource languages, emerging event types, or scenarios with few samples, their performance degrades significantly, and they are difficult to transfer to unseen tasks or domains.

[0005] Event extraction methods based on transfer learning introduce transfer learning strategies, transferring the model trained in the source domain to the target domain. They typically use pre-trained language models (such as BERT and RoBERTa) as feature extractors, fine-tuning them on a small amount of target domain data to improve the model's generalization ability and alleviate the problem of insufficient labeled data. However, their limitations lie in the fact that transfer learning still relies on supervision signals from the target domain and lacks modeling of differences between tasks, making it prone to performance instability during task switching.

[0006] Multi-task joint learning and prompt-driven event modeling methods extract sub-tasks from multiple events (such as trigger word recognition and argument recognition) for joint modeling, and use prompt templates to guide the pre-trained language model to focus on key information. However, although this method improves the model's ability to understand task semantics, it still suffers from poor robustness and insufficient generalization ability when dealing with complex scenarios such as role overlap, discourse reasoning, and argument cross-sentences, especially with a significant decline in performance under small sample conditions.

[0007] Meta-learning-based few-shot event extraction exploration methods view meta-learning as an important way to improve the model's ability to learn quickly under few-shot conditions. Its core idea is to train a "learner" capable of rapidly adapting to new tasks across multiple tasks. Some studies have applied it to event extraction tasks with some success. However, most existing methods neglect the stability issues during model parameter changes, easily leading to interference between optimization objectives across tasks, difficulty in model training convergence, and even overfitting. Summary of the Invention

[0008] This invention is designed for open-source news data environments and provides a method and system for extracting small-sample, chapter-level events from open-source news data. It comprehensively introduces a regularized meta-learning optimization strategy and a prompt-driven joint multi-task extraction mechanism, which can achieve efficient, robust, and structured chapter-level event information extraction under the condition of scarce labeled samples.

[0009] To achieve the above objectives, the technical solution of the present invention includes the following:

[0010] A method for extracting small-sample, chapter-level events from open-source news data, the method comprising:

[0011] Generate a task T containing x events for extraction. i The event extraction task set T = {T1.....T x}, and construct an event extraction prompt template;

[0012] Meta-learning training is performed on a large language model based on the event extraction task set T to obtain a pre-trained language model.

[0013] By using event extraction prompt templates, the pre-trained language model is jointly fine-tuned across multiple tasks to obtain a fine-tuned large speech model.

[0014] Event extraction results of the target text are obtained based on the fine-tuned large speech model. The event extraction results include: event type, trigger word and argument.

[0015] Furthermore, based on the event extraction task set T, a meta-learning training is performed on the large language model to obtain a pre-trained language model, including:

[0016] Extract task T for each event i N event types are randomly selected, and K samples are randomly selected for each event type to form the event extraction task T. i training set and test set

[0017] In the training set The above event extraction task Ti Train separately to obtain the event extraction task T. i The corresponding optimal parameter θ′ i ;

[0018] In the test set Above, based on the optimal parameter θ′ i Find the optimal parameter θ″ corresponding to the event extraction task set T;

[0019] Based on the optimal parameter θ″, the process is re-executed on the training set. The above event extraction task T i The language model is trained individually until the termination condition of the first iteration is met, resulting in a pre-trained language model.

[0020] Furthermore, in the training set The above event extraction task T i Train separately to obtain the event extraction task T. i The corresponding optimal parameter θ′ i ,include:

[0021] From the training set Selecting samples from the pool to calculate the event extraction task T i loss function Next, generate the parameter θ. i gradient Where λ represents the coefficient of the L2 regularization term;

[0022] Based on the gradient For task parameter θ i Update;

[0023] Based on the parameter update results, the process from the training set is re-executed. Selecting samples from the pool to calculate the event extraction task T i loss function The event extraction task T is obtained after the second iteration termination condition is met. i The corresponding optimal parameter θ′ i .

[0024] Furthermore, in the test set Above, based on the optimal parameter θ′ i Find the optimal parameters θ″ corresponding to the event extraction task set T, including:

[0025] Design the objective function Where θ′ represents the task T for extracting all events. i The corresponding optimal parameter θ′ i The set of parameters that make up the composition;

[0026] Calculate the gradient of the overall objective function with respect to the parameter θ′.

[0027] Based on the gradient The parameters of the overall objective are updated to obtain the optimal parameters θ″ corresponding to the extraction task set T.

[0028] Furthermore, the pre-trained language model is jointly fine-tuned across multiple tasks using event extraction prompt templates to obtain a fine-tuned large speech model.

[0029] The training text and event extraction prompt templates are integrated into the model input sequence I;

[0030] Encode the model input sequence I to obtain several hidden state vectors h i The hidden state vector sequence H is formed;

[0031] Extract the event features F contained in the hidden state vector sequence H;

[0032] Calculate the probability distribution v of trigger words in the training text. t The probability distribution v of the sum of arguments a And combine the event feature F to generate an enhanced trigger word feature vector F. mt and the enhanced argument eigenvector F ma ;

[0033] Based on the enhanced trigger word feature vector F respectively mt and the enhanced argument eigenvector F ma Perform multi-class prediction of event type and argument roles to obtain event type prediction output. And the role prediction output

[0034] Predict output based on event type And the role prediction output Construct a loss function to fine-tune the parameters of the pre-trained language model.

[0035] Furthermore, the event features F contained in the hidden state vector sequence H are extracted, including:

[0036] Design a feature extraction shared layer, which consists of L stacked Transformer layers; wherein the information processing process of the l-th Transformer layer includes:

[0037] The input vector is mapped to the query vector Q, the key vector K, and the value vector V; where the input vector of the first Transformer layer is the hidden state vector h. iThe input vectors of other Transformer layers are the output vectors of the previous Transformer layer;

[0038] Calculate the attention weight A based on the query vector Q and the key vector K;

[0039] Calculate the weighted sum of the attention weights and the value vector to obtain the output H′;

[0040] The output H′ is subjected to layer normalization and residual connection to obtain the result H″;

[0041] The result H″ is transformed using a feedforward network to obtain the transformed result FFN(H″);

[0042] Based on the result H″ and the transformation result FFN(H″), the output vector of the l-th Transformer layer is obtained.

[0043] The output vector of the Lth Transformer layer As an event feature F.

[0044] Furthermore, predict output based on event type. And the role prediction output Constructing a loss function to fine-tune the parameters of the pre-trained language model includes:

[0045] Constructing a trigger word recognition loss function in, Key trigger words corresponding to real event types The label for the event location in the event extraction prompt template;

[0046] Constructing an argument role recognition loss function in, Representing the real argument role

[0047] Construct an event type classification loss function in, Indicates the type of real event;

[0048] Constructing an argument role classification loss function in, Indicates a real-world argument type tag;

[0049] Generate joint loss function L total The joint loss function L total For trigger word recognition loss function L trigger Argument Role Recognition Loss Function L argument Event type classification loss function L cls-eventSum of arguments for role classification loss function L cls-role The weighted sum;

[0050] The gradient descent algorithm is used to analyze the joint loss function L. total Optimization is performed to fine-tune the parameters of the pre-trained language model.

[0051] A small-sample, chapter-level event extraction system for open-source news data, the system comprising:

[0052] The task generation and template building module is used to generate a task T containing x events for extraction. i The event extraction task set T = {T1.....T x}, and construct an event extraction prompt template;

[0053] The meta-learning module is used to perform meta-learning training on a large language model based on the event extraction task set T to obtain a pre-trained language model.

[0054] The prompt learning module is used to perform multi-task joint fine-tuning of the pre-trained language model using event-extracted prompt templates to obtain a fine-tuned large speech model.

[0055] The event extraction module is used to obtain event extraction results of the target text based on the fine-tuned large speech model. The event extraction results include: event type, trigger word, and argument.

[0056] An electronic device includes: a processor and a memory storing computer program instructions; the processor, when executing the computer program instructions, implements the small-sample, chapter-level event extraction method for open-source news data as described above.

[0057] A computer-readable storage medium storing computer program instructions that, when executed by a processor, implement the small-sample, chapter-level event extraction method for open-source news data as described above.

[0058] Compared with the prior art, the present invention has at least the following beneficial effects.

[0059] (1) Significantly improves the model's adaptability and generalization ability in scenarios with few samples. This invention introduces a regularization mechanism into the inner training process of meta-learning, effectively suppressing the problem of large fluctuations in model parameters and difficulty in stable learning under conditions of very little labeled data. The outer shared meta-parameters learn knowledge representations with broad transferability through a cross-task average loss optimization strategy, enabling the model to adapt quickly when faced with unseen event types or new argument role categories, significantly improving the robustness and practicality of event extraction tasks under data-scarce conditions.

[0060] (2) Improve the stability of the training process and the convergence consistency of the model across tasks. The constraint of the regularization term on parameter changes effectively prevents the training instability problem caused by inner layer overfitting or gradient divergence in traditional meta-learning methods. Experiments show that the meta-learning framework with regularization exhibits a smoother loss descent curve and a more consistent optimal parameter convergence path during multi-task training, which is conducive to the continuous enhancement of the model's generalization ability.

[0061] (3) Enhance the model's overall perception of event structure and its ability to focus on key elements. The prompt template designed in this invention embeds structural information such as event category and argument role type into the input sequence, giving full play to the sensitivity of the pre-trained language model to prompt information and guiding the model to focus on the core semantic region of the event. This design improves the accuracy of the model in identifying trigger words and argument positions and reduces the probability of misjudgment caused by interference from redundant information in the text.

[0062] (4) Achieving unified extraction and collaborative modeling of event elements. This invention breaks away from the limitation of separate training of the two sub-tasks, "trigger word recognition" and "argument recognition," in traditional event extraction tasks. It proposes a joint multi-task learning strategy, which achieves feature linkage learning among event elements through a unified parameter sharing mechanism. Experimental comparisons show that this mechanism outperforms single-task modeling schemes in both trigger word and argument recognition tasks, and has a significant advantage, especially in complex contexts with overlapping roles or semantic ambiguity.

[0063] (5) Supports document-level extraction and complex event semantic structure modeling. This invention is applicable to handling complex semantic structures commonly found in real news articles, such as cross-sentence arguments, role reuse, and multiple event co-occurrence. The output contains structured information on multiple elements, including event type, trigger words, and argument roles, providing high-quality basic data support for downstream tasks such as news data reasoning and knowledge graph construction. Attached Figure Description

[0064] Figure 1 This is a flowchart illustrating the overall framework of the present invention.

[0065] Figure 2 The inner flowchart is for the meta-learning optimization algorithm that incorporates regularization.

[0066] Figure 3 The outer flowchart is for combining regularized meta-learning optimization algorithms.

[0067] Figure 4 This is a flowchart of a joint multi-task event extraction algorithm based on prompting learning. Detailed Implementation

[0068] To enable those skilled in the art to better understand the technical solutions in the embodiments of the present invention, and to make the objectives, features, and advantages of the present invention more apparent and understandable, the core technologies of the present invention will be further described in detail below with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.

[0069] This invention provides a method for small-sample, article-level event extraction from open-source news data. It accurately extracts events from open-source news data using a limited number of labeled event samples. The invention mainly consists of two parts: a meta-learning optimization algorithm incorporating regularization and a joint multi-task event extraction algorithm based on cue learning. The meta-learning optimization algorithm incorporating regularization trains the event extraction module using a regularized meta-learning framework, effectively improving the event extraction model's adaptability to unseen event argument role types. The joint multi-task event extraction algorithm based on cue learning designs a unified event extraction training framework using the joint multi-task learning concept and combines cue templates with a pre-trained language model, fully utilizing event information and effectively alleviating the problem of inaccurate argument role classification when facing role overlap.

[0070] The method flow of the present invention is as follows: Figure 1 As shown, it includes the following steps S1 to S4.

[0071] Step S1: Generate a task T containing x events for extraction. i The event extraction task set T = {T1.....T x}, and construct an event extraction prompt template.

[0072] This step is mainly used to determine the initial set of event extraction tasks to be processed, prepare a small sample labeled data containing trigger words and arguments, and construct targeted prompt templates based on the semantic features of various events. The templates are combined with the original input to guide the model to focus on the event structure elements, which serve as the input basis for subsequent multi-task training.

[0073] Regarding task set generation, this invention constructs a set of x event extraction tasks {T1.....T x For any event extraction task T i For each event type, N event types are randomly selected, and K samples are randomly selected from each event type to form a training set. and test set Both training and test sets contain detailed event information, including key information such as event trigger words and argument roles. Additionally, for each event extraction task T... i Construct a parameter set θ respectively i This is used to store the model parameters for the task.

[0074] In terms of constructing prompt templates, this invention designs and embeds specific prompt templates for multi-task joint extraction based on the semantic structure and type characteristics of the events to be extracted. The prompt templates introduce structured prompt information into the original input text, thereby unifying the traditional event type identification and argument role labeling tasks into mask language modeling tasks, thus guiding the large language model to focus on key semantic fragments related to the event.

[0075] The core idea of ​​cue learning is to provide the model with explicit guidance or cues, enabling it to focus more on relevant information when processing specific tasks. For event extraction tasks, a cue template similar to a masked language model has been specifically designed:

[0076] event <event>argument role <role>Context

[0077] This includes the prompting statement "event". <event>argument role <role> ， <event>and <role>This is used to indicate the part that the model needs to fill in, followed by a closing marker. Subsequent parts also begin with a closing marker, with contextual information about the event added in the middle to provide the model with the background knowledge needed to predict event elements. This design can effectively and intuitively guide the pre-trained language model to use contextual information to predict the content of the masked part, while also associating events with argument roles, enabling the model to classify arguments more accurately.

[0078] Step S2: Perform meta-learning training on the large language model based on the event extraction task set T to obtain a pre-trained language model.

[0079] This invention first initializes shared meta-parameters and enters the outer optimization loop. Based on the meta-learning concept, the shared meta-parameters of the model are initialized for the initial representation of all tasks. When entering the outer loop, the task set is traversed, and the inner optimization process is called one by one. The meta-parameters are continuously updated through task-level learning feedback information to enhance the model's ability to learn common patterns among different tasks. The following embodiment is a meta-learning framework based on the MAML (Model-Agnostic Meta-Learning) concept. It can also achieve rapid adaptation through other equivalent meta-learning strategies, such as those based on Reptile, ProtoNet, and FOMAML (First-order MAML) algorithms. Only the inner and outer layer parameter update methods need to be adjusted, without changing the overall training process or extraction target.

[0080] Subsequently, task-level adaptive training is performed with the introduction of regularization. For each event extraction task, the model fine-tunes its parameters based on the inherited meta-parameters and minimizes its loss function for the current task. Simultaneously, a regularization term is introduced into the inner training layer to constrain changes in task parameters, thereby improving the stability and robustness of the model training and enhancing its generalization ability.

[0081] Finally, the meta-parameters are updated based on the multi-task average loss. After completing the inner-layer training for all tasks, the model performance for each task is statistically analyzed, and the current shared meta-parameters are optimized through backpropagation using the average loss as the objective function. This step ensures that the model can extract general extraction capabilities across multiple different tasks, thereby improving its ability to quickly adapt to unseen event categories or argument role types.

[0082] Specifically, this step consists of two layers: the inner layer (step S21) iterates and trains each event sampling task individually to obtain the optimal parameters, and the outer layer (step S22) finds the average optimal parameters for all tasks and adds a regularization term to the loss calculation in the inner and outer loops to ensure the robustness of the model.

[0083] Step 21: Inner layer of the algorithm.

[0084] Extract task T for each event i The model is trained separately, and the parameters for each task are updated individually. Finally, the optimal parameters for each task are obtained, allowing the model to quickly adapt to the goals of new tasks using these optimal parameters. The process is as follows: Figure 2 Step S21 further includes:

[0085] S21-1: For a certain task T i (in ), from its corresponding training set Samples are selected from the data to calculate the loss function. Regarding the parameter θ for this task i gradient:

[0086]

[0087] S21-2: Regarding the task parameter θ i Update, let θ′ i Represents the updated task parameter θ i ,Right now:

[0088]

[0089] Where α represents the learning rate of the inner loop gradient descent, and λ is the coefficient of the L2 regularization term. It should be noted that this invention can also employ KL divergence, gradient penalty, Fisher information matrix, or other methods to constrain parameter variations, depending on task requirements.

[0090] S21-3: Perform multiple iterations of the above process within the inner loop to obtain the optimal parameter θ. i In this stage, Sgd(θ,D) is used. t r) represents the formula as follows:

[0091]

[0092] Where θ′ represents the updated optimal parameter set.

[0093] Step 22: Outer layer of the algorithm.

[0094] The outer layer of the algorithm focuses on how to optimize and adjust the model's meta-parameters using the entire event extraction task set, primarily by minimizing the average loss across multiple tasks. The process is as follows: Figure 3 As shown, step S22 further includes:

[0095] S22-1: First, confirm that the overall objective is to calculate the minimum average loss over multiple event extraction tasks. Then, design the overall objective function as L. meta (Sgd(θ,D tr ),D te The calculation formula is as follows:

[0096]

[0097] Among them, L meta The objective function, i.e., minimizing the average loss, is obtained by averaging the losses of all event extraction tasks, with an L2 regularization term (λ / 2)||θ|| added. 2 And Sgd(θ, D) tr ) is θ′, which is the optimal set of parameters obtained from training a single task; Let represent the loss on the i-th task, and x represent the total number of event extraction tasks. Specifically, a test set D is introduced into the overall objective function. te This indicates that the overall objective function depends not only on the optimal parameter set θ′, but also explicitly on the test set D. te By utilizing test set data to optimize the model's generalization ability, it can handle and generalize to new, unseen data or tasks more quickly and effectively. Furthermore, the introduction of regularization terms allows for parameter adjustments during training, thereby encouraging the model to find more robust solutions.

[0098] S22-2: To learn the performance of the optimization model on the event extraction task, calculate the gradient of the overall objective function with respect to the parameter θ′, also considering the test set D. te Impact on the performance of each event extraction task:

[0099]

[0100] S22-3: The gradient descent algorithm is also used to update the parameters of the overall objective:

[0101]

[0102] Where β is the learning rate. The direction of the loss function growth is the direction in which the parameter update is in the opposite direction of the gradient, with the aim of reducing the value of the loss function.

[0103] S22-4: Finally, the above process is iterated and updated multiple times in the outer loop to obtain the optimal parameter θ″. During training, the task data covers event information of different types, domains, and complexities, and the introduction of L2 regularization helps the model find more robust solutions, effectively preventing overfitting. The combination of meta-learning ideas and regularization also enables the model to generalize more quickly to various event extraction tasks.

[0104] Step S3: Use event extraction prompt templates to perform multi-task joint fine-tuning of the pre-trained language model to obtain the fine-tuned large speech model.

[0105] Existing few-shot event extraction methods mostly focus on the event detection and argument extraction subtasks, failing to effectively extract event elements jointly. To address this issue, this invention proposes a joint multi-task learning method that integrates event detection and argument extraction tasks. The model can simultaneously learn and optimize both key tasks, improving overall performance and avoiding error accumulation.

[0106] Specifically, this invention utilizes a pre-constructed event prompt template to combine event extraction with prompt learning and a pre-trained language model. Task input is guided to the pre-trained language model, and a joint multi-task learning strategy is employed to simultaneously train event trigger word recognition and event argument recognition. Through shared model parameters and a joint learning mechanism, collaborative extraction of event elements and feature representation fusion are achieved, enabling the model to learn the mapping pattern between the prompt template and event semantics. The joint multi-task event extraction method flow is as follows: Figure 4 .

[0107] S31: First, the input sequence needs to be constructed. Input text X, input prompt template T, and then integrate the two into the model input sequence I, i.e.

[0108] I = T(X)

[0109] The function T applies the prompt template to the input text X.

[0110] S32: Encode the input sequence using a pre-trained language model. Given an input sequence I, the pre-trained language model R outputs a hidden state vector h for each token i. i ,Right now

[0111] h i =R(I) i

[0112] These hidden states together form a hidden state vector sequence H.

[0113] S33: Design a feature extraction shared layer to extract the event features contained in the encoded hidden state vector sequence H. The feature extraction shared layer consists of multiple Transformer layers, each with a hidden state vector h. i Mapped to query (Q), key (K), and value (V) vectors:

[0114] Q = HW Q ,

[0115] K = HW K ,

[0116] V = HW V

[0117] Among them W Q W K W V It is a learnable weight matrix.

[0118] Then, the attention weights are calculated, and the normalized attention weights A are obtained by applying the softmax function:

[0119]

[0120] Where, d k Represents the dimension of the key vector.

[0121] The output is obtained by calculating a weighted sum using the attention weights and value vectors:

[0122] H′=AV

[0123] Subsequently, layer normalization (LayerNorm) and residual connections need to be performed on the output H′:

[0124] H″ = LayerNorm(H + H′)

[0125] Then, a feedforward network (FFN) is used to transform the above result H″:

[0126] FFN(H″)=max(0,H″W1+b1)W2+b2

[0127] Where W1, W2, b1, b2 are the weights and biases of FFN.

[0128] Finally, we obtain the final output of the Transformer layer:

[0129] H output =LayerNorm(H″+FFN(H″))

[0130] After stacking multiple Transformer layers, the final shared feature representation F can be obtained from the last layer:

[0131]

[0132] Where L represents the index of the last level.

[0133] S34: In order to effectively train the model to extract events, a joint multi-task learning method is adopted. By designing a specific task layer, a task similar to a masked language model is constructed with the help of prompt templates, and combined with the probability distribution of masked word filling prediction. Then, based on the knowledge learned from the shared feature vector and the masked probability distribution, the multi-task target is predicted.

[0134] For the two attributes trigger and argument, two independent probability distributions can be obtained:

[0135] v t =softmax(W t H m +b t )

[0136] v a =softmax(W a H r +b a )

[0137] Where W and b are the weights and biases for each prediction task, and H... m H r It is the hidden state output by the pre-trained language model that corresponds to the mask position.

[0138] After detecting trigger words and arguments, the obtained predicted probability distribution is combined with previously obtained shared features to obtain an enhanced feature vector containing hidden mask location prediction information.

[0139] F mt =Concat(F,v t )

[0140] F ma =Concat(F,v a )

[0141] The Concat function represents the concatenation operation, F mt F ma These represent the enhanced feature vectors, which combine features extracted from the original text with contextual information provided by the MLM probability distribution.

[0142] Based on the different characteristics of event type and argument roles, the softmax function is used to perform multi-class prediction of event type and argument roles, i.e.

[0143]

[0144] Among them, W event W role b event b role These represent the weights and biases after incorporating the MLM probabilities, respectively. This provides the predicted output for each task.

[0145] S35: To optimize the model's performance on the event extraction task, a supervised loss function is defined for trigger word recognition and argument extraction. Since this invention transforms the task into a masked language modeling form using prompt templates, the model... <event>and

[0146] <role>The predicted mask positions should align with the true words. Therefore, the cross-entropy loss function is used to calculate the prediction errors for trigger words and argument roles, respectively.

[0147] Trigger-related loss L trigger .make This indicates the key trigger word corresponding to the actual event type, and this word is used as a prompt template. <event>Location label. Trigger word recognition loss is defined as:

[0148]

[0149] Where idx(·) represents the index of the real word in the vocabulary.

[0150] Argument-related loss L argument .make Representing real argument roles, used as prompt templates <role>Location label. The argument role recognition loss is defined as:

[0151]

[0152] To further improve classification accuracy, an auxiliary classification loss is introduced, and the classification is directly supervised by event type and argument role:

[0153]

[0154] To optimize the model, a joint loss function is constructed, defined as the loss function L for all tasks. total Weighted sum:

[0155]

[0156] λ t L is the weight coefficient for the t-th task, used to adjust the contribution of different tasks to the total loss. t It is the loss function for the t-th task, and the task loss includes L. trigger L argument L cls-event L cls-role 4 items.

[0157] The gradient descent algorithm is used to analyze the joint loss function L. total Optimize,

[0158]

[0159] Where θ represents the model parameters and η is the learning rate. It is L total The gradient with respect to θ.

[0160] Step S4: Obtain the event extraction results of the target text based on the fine-tuned large speech model.

[0161] Once the model has completed training and is accepting new text input, it outputs structured event information, including event type, trigger words, and argument roles, based on the model's extraction results. The output can be directly used in downstream applications such as event understanding, knowledge graph construction, or news data reasoning.

[0162] The present invention will be illustrated by an example below.

[0163] Step S1: Initialize task and prompt template construction.

[0164] In the open-source news corpus, several typical event extraction tasks were first selected, such as corporate mergers and acquisitions, product launches, natural disasters, and public health events. Each task was equipped with a small number of labeled samples, which were labeled with event trigger words and their corresponding argument roles (time, location, participants, etc.).

[0165] For example, the original news clip:

[0166] Xiaoming arrived at location A in April 2023 and began discussing an academic issue with Xiaohong.

[0167] Structured event annotation results:

[0168]

[0169]

[0170] Based on the semantic features of various events, a prompt template is constructed for each type of event, for example:

[0171] Does the news report describe an event of the type "[diplomatic visit]"? If so, please identify the trigger words and related elements. (Time, Location, Participants, Moderator, Topic)

[0172] The template and the original input text are concatenated together and used as input to guide the pre-trained language model to focus on event structure information, providing semantic guidance for subsequent model training.

[0173] Step S2: Employing a meta-learning optimization framework incorporating regularization, we first initialize the shared model meta-parameters θ to uniformly represent the initial state of all tasks. Then, we initiate the outer optimization loop to prepare for subsequent meta-training on multiple event extraction tasks.

[0174] In the outer loop, the system will perform inner adaptive training for each task in sequence (see step S21), and update the shared meta-parameters after all tasks are completed (see step S22).

[0175] Step S21: Task-level adaptive training and regularization.

[0176] In each task, the model is fine-tuned based on the current meta-parameters to optimize the task-specific loss function and improve the ability to identify the event type.

[0177] To avoid parameter overfitting or oscillation, a regularization term is introduced as a constraint to ensure smooth parameter changes during training within the task, thereby improving the model's stability and generalization ability in scenarios with few samples.

[0178] The inner optimization process can be repeated several times until the task loss converges or a set number of steps are reached.

[0179] Step S22: Update meta-parameters based on multi-task average loss.

[0180] After completing the inner layer training for each task, the task loss values ​​are collected, and the meta-parameters θ are updated based on the average loss using the backpropagation algorithm.

[0181] This step constitutes a complete outer optimization cycle, ensuring that the model refines its extraction capabilities across multiple tasks, thereby enhancing its adaptability to unseen events or arguments. This outer process can be continuously iterated based on model performance, forming multiple loops.

[0182] Step S3: Cue-driven joint multi-task event extraction training.

[0183] The trained prompt template and task text are combined and input into the pre-trained language model. A joint multi-task learning mechanism is adopted to train the event trigger word recognition task and the event argument recognition task at the same time, thereby improving the collaborative extraction capability between event elements.

[0184] During training, each subtask shares model parameters and performs joint optimization, effectively enhancing the modeling ability of semantic relationships between event elements. Furthermore, this invention can employ one or more mainstream pre-trained language models such as BERT, RoBERTa, T5, BART, and GLM, combined with constructed event prompt templates and a multi-task learning structure, to complete the identification and structured extraction of event elements from open-source news data text. The selected model possesses good transfer learning and context modeling capabilities, adapting to the needs of event extraction under small sample conditions. It should be noted that this invention can also replace the aforementioned mainstream pre-trained language models with structure-aware models (such as ERNIE and KEPLER) or long-text Transformers with stronger document-level modeling capabilities (such as Longformer and BigBird), achieving structural information capture and task transfer in larger-scale document contexts.

[0185] Step S4: Output the structured event extraction results.

[0186] After the model is trained, inputting new open-source news text allows the model to automatically identify the event type, trigger words, and their arguments (time, location, actors, etc.), and output them in a structured format. For example:

[0187] Original news text:

[0188] Typhoon Lan made landfall in area B in August 2023, causing traffic disruptions and widespread power outages.

[0189] Extraction results:

[0190] [Event Type]: Natural Disaster (Typhoon).

[0191] [Trigger words]: login, cause.

[0192] [Analysis]: Time: July 2024; Location: Area B; Disaster Type: Typhoon Lan; Impact: Traffic disruption, widespread power outages.

[0193] In addition to the aforementioned structured event element set, the output of this invention can also be extended to event triples, timeline format, or graph node representation structure, depending on the application scenario.

[0194] In summary, this invention is applicable to processing open-source news data containing a small number of labeled samples, and is especially suitable for jointly extracting event types, event trigger words, and event arguments from article-level texts such as news and reports. It has strong domain adaptability and low resource adaptability.

[0195] Existing event extraction models often struggle to learn effectively when faced with a lack of labeled data, particularly exhibiting significant performance degradation for unfamiliar event types or argument role categories. This invention addresses this issue by introducing a meta-learning optimization algorithm incorporating regularization. Utilizing an inner and outer loop structure to model multiple tasks, the inner layer improves individual performance through task-level parameter updates, while the outer layer extracts common knowledge across tasks through meta-parameter optimization. This significantly enhances the model's ability to quickly adapt to new tasks, thus solving the problem of improving the adaptability and generalization performance of event extraction models for unseen tasks under limited sample conditions.

[0196] Traditional meta-learning methods suffer from large model fluctuations and overfitting during inner-layer task training, affecting overall training effectiveness and generalization performance. This invention introduces regularization constraints during the inner-layer task optimization phase. By imposing reasonable restrictions on the parameter update process, the model maintains consistency in parameter update paths across multiple tasks, effectively improving the model's stability, consistency, and overall convergence during training.

[0197] Most current methods model event trigger word identification and argument identification as two independent tasks, lacking an integrated modeling strategy for the overall semantics of the event, which affects the consistency of extraction results. The joint multi-task event extraction algorithm based on cue learning proposed in this invention constructs specific cue templates to guide the pre-trained language model to uniformly perceive event elements, and utilizes a multi-task joint learning mechanism to simultaneously extract trigger words and arguments, improving the model's overall perception of the event's semantic structure and achieving collaborative extraction of event trigger words and argument roles.

[0198] In open-source news articles, event arguments may be distributed across sentences or have semantic overlap across multiple role categories, posing a significant challenge to traditional methods. This invention addresses this by integrating event semantic features into prompt templates and combining this with the contextual modeling capabilities of a pre-trained language model to guide the model in constructing a structured representation, thereby improving the model's accuracy in identifying cross-sentence arguments and role overlap.

[0199] The embodiments described above are merely illustrative of implementation methods of the present invention, and while the descriptions are detailed, they should not be construed as limiting the scope of the present invention. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these all fall within the scope of protection of the present invention. Therefore, the scope of protection of this patent should be determined by the appended claims.< / role> < / event> < / role> < / event> < / role> < / event> < / role> < / event> < / role> < / event>

Claims

1. A method for extracting small-sample, chapter-level events from open-source news data, characterized in that, The method includes: Generate a task T containing x events for extraction. i The event extraction task set T = {T1.....T x }, and construct an event extraction prompt template; Meta-learning training is performed on a large language model based on the event extraction task set T to obtain a pre-trained language model. By using event extraction prompt templates, the pre-trained language model is jointly fine-tuned across multiple tasks to obtain a fine-tuned large speech model. Event extraction results of the target text are obtained based on the fine-tuned large speech model. The event extraction results include: event type, trigger word and argument.

2. The method according to claim 1, characterized in that, Meta-learning training is performed on a large language model based on an event extraction task set T to obtain a pre-trained language model, including: Extract task T for each event i N event types are randomly selected, and K samples are randomly selected for each event type to form the event extraction task T. i training set and test set In the training set The above event extraction task T i Train separately to obtain the event extraction task T. i The corresponding optimal parameter θ′ i ; In the test set Above, based on the optimal parameter θ′ i Find the optimal parameter θ″ corresponding to the event extraction task set T; Based on the optimal parameter θ″, the process is re-executed on the training set. The above event extraction task T i The language model is trained individually until the termination condition of the first iteration is met, resulting in a pre-trained language model.

3. The method according to claim 2, characterized in that, In the training set The above event extraction task T i Train separately to obtain the event extraction task T. i The corresponding optimal parameter θ′ i ,include: From the training set Selecting samples from the pool to calculate the event extraction task T i loss function Next, generate the parameter θ. i gradient Where λ represents the coefficient of the L2 regularization term; Based on the gradient For task parameter θ i Update; Based on the parameter update results, the process from the training set is re-executed. Selecting samples from the pool to calculate the event extraction task T i loss function The event extraction task T is obtained after the second iteration termination condition is met. i The corresponding optimal parameter θ′ i .

4. The method according to claim 2, characterized in that, In the test set Above, based on the optimal parameter θ′ i Find the optimal parameters θ″ corresponding to the event extraction task set T, including: Design the objective function Where, θ ′ This indicates that task T is extracted from all events. i The corresponding optimal parameter θ′ i The set of parameters that make up the composition; Calculate the overall objective function with respect to parameter θ ′ gradient Based on the gradient The parameters of the overall objective are updated to obtain the optimal parameters θ″ corresponding to the extraction task set T.

5. The method according to claim 1, characterized in that, By using event extraction prompt templates, the pre-trained language model is jointly fine-tuned across multiple tasks to obtain a fine-tuned large speech model. The training text and event extraction prompt templates are integrated into the model input sequence I; Encode the model input sequence I to obtain several hidden state vectors h i The hidden state vector sequence H is formed; Extract the event features F contained in the hidden state vector sequence H; Calculate the probability distribution v of trigger words in the training text. t The probability distribution v of the sum of arguments a And combine the event feature F to generate an enhanced trigger word feature vector F. mt and the enhanced argument eigenvector F ma ; Based on the enhanced trigger word feature vector F respectively mt and the enhanced argument eigenvector F ma Perform multi-class prediction of event type and argument roles to obtain event type prediction output. And the role prediction output Predict output based on event type And the role prediction output Construct a loss function to fine-tune the parameters of the pre-trained language model.

6. The method according to claim 5, characterized in that, Extract the event features F contained in the hidden state vector sequence H, including: Design a feature extraction shared layer, which consists of L stacked Transformer layers; wherein the information processing process of the l-th Transformer layer includes: The input vector is mapped to the query vector Q, the key vector K, and the value vector V; where the input vector of the first Transformer layer is the hidden state vector h. i The input vectors of other Transformer layers are the output vectors of the previous Transformer layer; Calculate the attention weight A based on the query vector Q and the key vector K; Calculate the weighted sum of the attention weights and the value vector to obtain the output H. ′ ; For output H ′ Perform layer normalization and residual connection to obtain the result H″; The result H″ is transformed using a feedforward network to obtain the transformed result FFN(H″); Based on the result H″ and the transformation result FFN(H″), the output vector of the l-th Transformer layer is obtained. The output vector of the Lth Transformer layer As an event feature F.

7. The method according to claim 5, characterized in that, Predict output based on event type And the role prediction output Constructing a loss function to fine-tune the parameters of the pre-trained language model includes: Constructing a trigger word recognition loss function in, Key trigger words corresponding to real event types The label for the event location in the event extraction prompt template; Constructing an argument role recognition loss function in, Representing the real argument role Construct an event type classification loss function in, Indicates the type of real event; Constructing an argument role classification loss function in, Indicates a real-world argument type tag; Generate joint loss function L total The joint loss function L total For trigger word recognition loss function L trigger Argument Role Recognition Loss Function L argument Event type classification loss function L cls-event Sum of arguments for role classification loss function L cls-role The weighted sum; The gradient descent algorithm is used to analyze the joint loss function L. total Optimization is performed to fine-tune the parameters of the pre-trained language model.

8. A small-sample, chapter-level event extraction system for open-source news data, characterized in that, The system includes: The task generation and template building module is used to generate a task T containing x events for extraction. i The event extraction task set T = {T1.....T x }, and construct an event extraction prompt template; The meta-learning module is used to perform meta-learning training on a large language model based on the event extraction task set T to obtain a pre-trained language model. The prompt learning module is used to perform multi-task joint fine-tuning of the pre-trained language model using event-extracted prompt templates to obtain a fine-tuned large speech model. The event extraction module is used to obtain event extraction results of the target text based on the fine-tuned large speech model. The event extraction results include: event type, trigger word, and argument.

9. An electronic device, characterized in that, The electronic device includes: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, it implements the small sample chapter-level event extraction method for open source news data as described in any one of claims 1-7.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer program instructions that, when executed by a processor, implement the small-sample, chapter-level event extraction method for open-source news data as described in any one of claims 1-7.

Images