Document-level event argument extraction method fusing role capsule and prompt gating

Abstract

The invention relates to a document-level event argument extraction method fusing role capsules and prompt gating, which comprises the following steps of: extracting data from document-level event parameters, and constructing a data set; preprocessing the data set to obtain input of an event parameter extraction model; constructing a document-level event parameter extraction model, and performing model training and verification; the document-level event parameter extraction model comprises a coding layer, a trigger word modeling layer, a gating prompt fusion layer, a role perception capsule network layer and a boundary prediction and matching layer; and inputting the test set of the data set into the model for testing, and optimizing the model through an ablation experiment. According to the method, a capsule network structure based on role categories is introduced, semantic representation capable of sensing the structural relationship between roles is generated around event trigger words, a prompt gating mechanism is designed to perform fusion modeling on structured prompts and context semantics, a loss function is optimized by using a Hungary matching algorithm and a span, and the probability of event triggering is lowered. The expression ability of the prompt information is improved, so that the matching precision is improved.

Description

Technical Field

[0001] This invention belongs to the field of information extraction in natural language processing, specifically relating to a document-level event argument extraction method that integrates role capsules and cue gating. Background Technology

[0002] With the explosive growth of online text data, mining structured event information from massive amounts of unstructured text has become a crucial foundation for intelligent applications such as public opinion analysis, knowledge graph construction, and social computing. Event extraction aims to identify specific event types, trigger words, and their participating roles from natural language. Event parameter extraction, a key subtask of this task, aims to identify the entity boundaries and type information corresponding to various semantic roles in each event. Current research on event parameter extraction mainly relies on attention mechanisms to focus on the semantics of trigger words or prompts, but this approach has significant shortcomings: it fails to fully capture the potential structural relationships between different argument roles; when texts involve multiple events and complex inter-sentence dependencies, role representation is easily interfered with by irrelevant contextual information; and existing neural networks often ignore highly overlapping multiple relational features in sentences.

[0003] To address the aforementioned issues, a capsule network and attention routing algorithm were combined to accurately extract multi-label relationships. However, this approach treats prompts as static input or simply concatenates them into the context, lacking a dynamic interaction mechanism. This results in poor model performance when recognizing event parameters with ambiguous semantic boundaries or unclear contextual dependencies. A hierarchical annotation framework based on gated multi-layer attention was proposed, using a gating mechanism to dynamically fuse sentence-level and document-level information, improving the model's trigger word recognition performance. However, the effective integration of role modeling and prompt interaction remains a challenge. Summary of the Invention

[0004] The technical problem of this invention is to solve the problem that existing document-level event parameter extraction techniques are difficult to accurately model the semantic relationship between event trigger words and arguments; and the ability to fuse structured prompt information is insufficient, which limits the performance of event parameter recognition and classification.

[0005] The purpose of this invention is to solve the above problems by generating semantic representations that can perceive the structural relationships between roles, with event trigger words as the core, thereby strengthening the intrinsic connection between different argument roles; and dynamically adjusting the adaptation weight of prompt information in specific contexts through a gating mechanism, so that prompts can play a more accurate guiding role in semantic boundary recognition and semantic alignment.

[0006] The technical solution of this invention is a document-level event argument extraction method that integrates role capsules and cue gating, comprising the following steps: S1: Extract data from document-level event parameters and construct a dataset; S2: Preprocess dataset D1 to obtain the input for the event parameter extraction model; S3: Construct a document-level event parameter extraction model and perform model training and validation; the document-level event parameter extraction model includes an input encoding layer, a trigger word modeling layer, a gating cue fusion layer, a role-aware capsule network layer, and a boundary prediction and matching layer; S4: Input dataset D1 into the model for testing, and optimize the model through ablation experiments.

[0007] Furthermore, in step S2, the dataset D1 is preprocessed, including the following sub-steps: 1) Annotate the text containing event trigger words, and use special markers to record the order in which the annotated trigger words appear. Then, segment the annotated text to obtain a segmentation sequence. 2) Connect all event notification texts into a final notification text, add the corresponding event type at the beginning of each event notification text, and wrap it with a unique identifier to help distinguish them; 3) Connect the word segmentation sequence with the prompt text to form the model input.

[0008] Preferably, in step S3, the input encoding layer constructs an encoding and decoding architecture based on Roberta, inputs the concatenated word segmentation sequence and prompt text into the encoder, and then generates an event-related context representation and a prompt representation combined with the context through the decoder.

[0009] Furthermore, step S3 triggers the word modeling layer, which includes the following sub-steps: 1) Obtain the contextual representation of the trigger word. After input, the contextual representation and start and end positions are obtained through encoding. 2) The trigger word context vector is obtained through average pooling, and then the vectors extracted by the attention vector weighting matrix are weighted and summed. 3) The weighted summation result is concatenated with the trigger word representation to form a fusion feature.

[0010] Preferably, in step S3, the gating cue fusion layer includes: extracting the corresponding event triggers from the attention matrix A. Context attention vector ∈ And the argument slots in the corresponding prompt template Context attention vector ∈ Furthermore, by performing an averaging operation on relevant sub-word tags through attention heads, and then fusing contextual cues and hints, the context-enhanced representation is calculated.

[0011] Furthermore, the gating fusion layer also includes: designing a gating mechanism to target the decoder for the first... The first event Individual argument slot Output representation With context aggregation representation The two components are combined to dynamically adjust their relative roles in the fusion process; the calculation formula is as follows: ; ; ; In the formula, , ∈ These are all learnable weight matrix parameters. yes function, This indicates element-wise multiplication.

[0012] Preferably, in step S3, the role-aware capsule network layer captures the deep structural association between candidate cues and trigger words through a dynamic routing mechanism, and relies on role priors to guide the construction of role representations, including the following sub-steps: 1) Obtain the trigger word representation of the i-th event. The candidate fusion hints for the j-th argument slot are then combined and averaging to obtain a joint representation of the candidate pairs. ; 2) Through a set of learnable linear transformations, each Projection is a representation of the predicted role candidates; 3) Through a dynamic routing algorithm, the feature representations of candidate arguments are adaptively assigned to the corresponding role capsules in multiple iterations, ultimately generating capsule representations that fuse role semantics. .

[0013] Furthermore, the dynamic routing algorithm includes the following sub-steps: 1) Initialize routing coefficients Softmax is used to calculate the coupling weights between candidates and capsules. ; 2) Each candidate argument Prediction vector Coupling weights of the candidate assigned to the capsule Multiply by all candidate arguments and sum over them to obtain the first... The aggregated input vector of the capsules ; 3) Use compression Function to obtain capsule output ,Will And role candidate prediction Perform a dot product to measure their similarity, and update the routing coefficients based on the similarity between the capsule output and the candidate predictions. ; 4) Based on the current role category, select the capsule output corresponding to its index. This serves as the semantic vector for the role of the argument slot.

[0014] Preferably, dynamic routing iteration, which involves repeated execution to achieve dynamic allocation, is calculated as follows: ; ; In the formula, It is the first The aggregated input vector of each capsule, It is the capsule output corresponding to the index. It represents the prediction of role candidates. Represents the j-th candidate argument The routing coefficient between the m-th capsule and the m-th capsule. These are the updated routing coefficients.

[0015] Furthermore, in step S3, the boundary prediction and matching layer includes fusing the role semantic vector and slot representation to obtain the final representation of each slot, and converting it into a span selector to determine the start and end positions of arguments in the text; where the same event may have multiple arguments with the same role, a binary matching loss is used in combination with the Hungarian algorithm to find the optimal allocation optimization, thereby improving the recognition accuracy of multi-class and multi-quantity targets.

[0016] Compared with the prior art, the beneficial effects of the present invention include: 1) This invention provides a document-level event argument extraction method that integrates role capsules and cue gating. It utilizes a role-aware capsule network module to dynamically generate semantic representations of corresponding roles by combining trigger words and candidate argument information. This strengthens the semantic association modeling between trigger words and arguments and improves the accuracy of semantic association modeling between trigger words and arguments.

[0017] 2) This invention provides a document-level event argument extraction method that integrates role capsules and cue gating. By leveraging the cue gating mechanism, it integrates structured cueing with contextual semantics to enhance the effective expression of cueing information.

[0018] 3) This invention provides a document-level event argument extraction method that integrates role capsules and cue gating. In model training, boundary matching and Hungarian algorithm optimization are integrated to further improve the matching accuracy of argument parameters.

[0019] 4) This invention provides a document-level event argument extraction method that integrates role capsules and cue gating. By solving the aforementioned problems of insufficient semantic association modeling and weak cue information fusion capabilities, it optimizes the overall effect of parameter recognition and classification. Attached Figure Description

[0020] The present invention will be further described below with reference to the accompanying drawings and embodiments.

[0021] Figure 1 This is a schematic diagram of the document-level event argument extraction method that integrates role capsules and cue gating according to an embodiment of the present invention; Figure 2 This is a schematic diagram of the document-level event parameter extraction model structure according to an embodiment of the present invention; Figure 3 This is a schematic diagram of the role-aware capsule network layer structure according to an embodiment of the present invention. Detailed Implementation

[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0023] like Figure 1 As shown, a document-level event argument extraction method that integrates role capsules and cue gating includes the following steps: S1: Extract data from document-level event parameters and construct a dataset.

[0024] The dataset was extracted using two publicly available English document-level event parameters, RAMS and WikiEvents, and divided into training, validation, and test sets according to the original dataset.

[0025] S2: Preprocess the dataset to obtain the input for the event parameter extraction model; S3: As Figure 2 As shown, a document-level event parameter extraction model is constructed, and the model is trained and validated. The document-level event parameter extraction model includes an input encoding layer, a trigger word modeling layer, a gating cue fusion layer, a role-aware capsule network layer, and a boundary prediction and matching layer. Step S3, the encoding layer, involves processing the original text, including the following sub-steps: 1) First, annotate the text containing event trigger words, and then use special tags to mark them. and The trigger words are marked and recorded in order of occurrence. Then, the marked text is segmented to obtain a word segmentation sequence. The expression is: ; In the formula, It is the first token of the text. It is the last token in the text. It refers to the first One trigger word; 2) Set the event notification text. Connect to obtain the final prompt text. Prompt text The expression is: ; In the formula, Indicates the first The prompt for each event, Indicates event type The first token.

[0026] 3) Add the corresponding event type at the beginning of each event notification text, and each event type is marked with a unique paired identifier ( and Enclosed by, and then connected to the word segmentation sequence and prompt text Get the input text The input is encoded in the encoder, and the calculation formula is: ; In the formula, This indicates a dependency-enhanced Transformer encoder. This indicates the input text. ; Represents a multi-head attention matrix. This represents a context-encoded representation; In step S3, the encoding layer further includes an encoding / decoding architecture built based on Roberta. The concatenated word segmentation sequence and prompt text are input into the encoder, and then the decoder generates event-related contextual representations and contextualized prompt representations. The attention bias value depends on the dependency relationship. The parameters, along with contextual information, will be called in combination after training. The expressions for generating event-related contextual representations and context-based hint representations are as follows: ; ; In the formula, the input represents , , , , Represents token and Learnable attentional bias between them Hyperparameters used to adjust the effect of this bias; Represents token and Dependency relationship between them ∈ and It is related to dependency The corresponding trainable parameters, Represents the hidden dimension of each attention head; The output of the encoder represents Input decoder generation ,in This indicates a fusion of event-oriented context and context-oriented tooltip representation, encompassing both event-oriented and context-oriented tooltip representations; ; In the formula, This represents the encoder output. This indicates a combination of event-oriented context and context-oriented prompt representation.

[0027] The trigger word modeling layer includes the following sub-steps: 1) Obtain the trigger word The context representation is obtained by encoding the input to obtain the context and start and end positions, respectively. and ; 2) Obtain the basic representation vector of the trigger word by performing average pooling on the representation region corresponding to the trigger word. The calculation formula is: ; In the formula, This represents the basic representation vector of the trigger word; 3) From the multi-head attention matrix Extract the attention vector corresponding to the trigger word. Then, a weighted sum is performed based on the context representation to obtain the context aggregation representation of the trigger word. The calculation formula is: ; In the formula, Event trigger words Attention weights for the j-th token in the input sequence. This represents the context encoding of the j-th token; 4) Represent the trigger word as a basic symbol Its corresponding context aggregation representation The features are concatenated to obtain the fused trigger word feature representation. . In step S3, the gated cue fusion layer dynamically integrates the cue vector with the context representation to model the argument representation at a fine-grained level: by averaging all attention heads and related subwords, the corresponding event triggers are extracted from the attention matrix A. Context attention vector And the argument slots in the corresponding prompt template Context attention vector Then, by performing an averaging operation on relevant sub-word tags through attention heads, and then fusing contextual cues and hints, it is used to calculate context-enhanced representations and argument extraction. The calculation formula is as follows: ; ; In the formula, This represents the attention weight of the k-th argument slot in the i-th event. This indicates context-enhanced representation. Represents the context attention vector. Represents the slot of the argument The corresponding context attention vector, express.

[0028] The gating fusion layer also includes: designing a gating mechanism to target the decoder for the first... The first event Individual argument slot Output representation With context aggregation representation Combining adaptive gating vectors Its value is in ,when When 0 is connected, the output depends more on the decoder for the first digit. The first event Individual argument slot Output representation ,when As the value approaches 1, the output becomes more dependent on the decoder for the first... The first event Individual argument slot Output representation With context aggregation representation The fused representation obtained by the common transformation is used to dynamically adjust the relative importance of the two in the fusion process. The calculation formula is as follows: ; ; ; In the formula, , ∈ These are all learnable weight matrix parameters. yes function, This indicates element-wise multiplication.

[0029] like Figure 3 As shown, the role-aware capsule network layer in step S3 captures the deep structural association between candidate cues and trigger words through a dynamic routing mechanism, and relies on role priors to guide the construction of role representations, including the following sub-steps: 1) Obtain the trigger word representation of the i-th event. The candidate fusion hints for the j-th argument slot are then combined and averaging to obtain a joint representation of the candidate pairs. The calculation formula is: ; ; In the formula, It is the representation of the j-th candidate argument in the k-th slot of the i-th event. The joint representation of candidate pairs is obtained through splicing and averaging operations; 2) Through a set of learnable linear transformations, each capsule has an independent linear layer, mapping each capsule to a role type, and representing each candidate argument. The candidate character prediction representations are generated by projecting a set of independent linear mappings onto the individual character capsule spaces. The expression is: ; In the formula, Indicates the first The learnable linear transformation matrix of each capsule ; Represents the bias vector; 3) Through a dynamic routing algorithm, the feature representations of candidate arguments are adaptively assigned to the corresponding role capsules in multiple iterations, ultimately generating capsule representations that fuse role semantics. .

[0030] The dynamic routing algorithm includes the following sub-steps: 1) Initialize routing coefficients The Softmax function is used to calculate the coupling weights between the candidate and the capsule. The calculation formula is: ; In the formula, Represents the j-th candidate argument With the Routing coefficients between capsules.

[0031] 2) Each candidate argument Prediction vector Coupling weights of the candidate assigned to the capsule Multiply by all candidate arguments and sum over them to obtain the first... The aggregated input vector of the capsules The calculation formula is: ; In the formula, This represents the aggregated input vector. Indicates the coupling weight of the capsule. Indicates candidate arguments Prediction vector.

[0032] 3) Use compression Function to obtain capsule output ,Will And role candidate prediction Perform a dot product to measure their similarity, and update the routing coefficients based on the similarity between the capsule output and the candidate predictions. .

[0033] 4) Based on the current role category, select the capsule output corresponding to its index. This serves as the semantic vector for the role of the argument slot.

[0034] Dynamic routing iteration, which involves repeated execution to achieve dynamic allocation, is calculated as follows: ; ; In the formula, It is the first The aggregated input vector of each capsule, It is the capsule output corresponding to the index. It represents the prediction of role candidates. Represents the j-th candidate argument The routing coefficient between the m-th capsule and the m-th capsule. These are the updated routing coefficients.

[0035] In step S3, the boundary prediction and matching layer includes fusing the semantic vector of the role and the slot representation to obtain the final representation of each slot, and converting it into a span selector to determine the start and end positions of the argument in the text. The same event may have multiple arguments of the same role. A binary matching loss is used in combination with the Hungarian algorithm to find the optimal allocation and optimize, thereby improving the recognition accuracy of multi-class and multi-quantity targets.

[0036] Specifically, the boundary prediction and matching layer obtains the character semantic vector output by the character perception capsule network layer. And the representation of each slot in each event obtained by the gating cue fusion layer. Then, these two are merged to obtain the final representation of each slot in each event. .

[0037] Character semantic vectors and the representation of each slot Convert to a set of span selectors The calculation formula is: = ; ; ; In the formula, , These are learnable parameters. This indicates element-wise addition. This indicates element-wise multiplication.

[0038] Will and Used to determine slot positions The start and end positions in the original text are calculated using the following formula: ; ; ; ; In the formula, This represents the argument span predicted by the model. and Indicates the first The first event The probability distribution vectors of the starting and ending positions of each argument slot. Indicates the first The first event The candidate argument span score corresponding to each argument slot This indicates the predicted argument span of the argument slot.

[0039] A binary matching loss is used to improve the recognition accuracy for multiple categories and quantities of targets. The calculation formula is as follows: ; In the formula, Represents the loss function. Indicates the number of events. Indicates the first One event, This represents the optimal allocation calculated using the Hungarian algorithm.

[0040] S4: Input dataset D1 into the model for testing, and optimize the model through ablation experiments.

[0041] On a public dataset, this invention was compared with existing event parameter extraction models. Strict parameter identification F1 (Arg-I) and strict parameter classification F1 (Arg-C) were used as evaluation metrics. The experimental results are shown in Tables 1 and 2. Table 1

[0042] Table 2

[0043] Ablation experiments were conducted on the document-level event parameter extraction model, specifically: (1) w / o ECN: indicates removing the role perception capsule module.

[0044] (2) w / o Gpfm: indicates that the gating prompt fusion module is removed.

[0045] (3) w / o ECN+ Gpfm: This means removing the character perception capsule module and the gating prompt fusion module at the same time.

[0046] The experimental results are shown in Tables 3 and 4: Table 3

[0047] Table 4

[0048] As shown in Tables 1 to 4, this invention outperforms other methods in both parameter identification F1 (Arg-I) and parameter classification F1 (Arg-C) parameters. This invention utilizes capsule networks to dynamically capture the role matching relationship between trigger words and candidate arguments, while employing a gating cue mechanism to achieve context fusion, thus more effectively presenting the semantic structure between arguments. Furthermore, it does not require additional structural labels or pre-built graphs, exhibiting stronger adaptability and generalizability.

[0049] The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A document-level event argument extraction method integrating role capsules and cue gating, characterized in that, Includes the following steps: S1: Extract data from document-level event parameters and construct a dataset; S2: Preprocess the dataset to obtain the input for the event parameter extraction model; S3: Construct a document-level event parameter extraction model, and train and validate the model; the document-level event parameter extraction model includes an encoding layer, a trigger word modeling layer, a gating cue fusion layer, a role-aware capsule network layer, and a boundary prediction and matching layer; S4: Input the test set of the dataset into the model for testing, and optimize the model through ablation experiments.

2. The document-level event argument extraction method integrating role capsules and cue gating as described in claim 1, characterized in that, In step S3, the encoding layer includes processing the original text, comprising the following sub-steps: 1) Annotate the text containing event trigger words, and use special markers to record the order in which the annotated trigger words appear. Then, segment the annotated text to obtain a segmentation sequence. 2) Connect all event notification texts into a final notification text, add the corresponding event type at the beginning of each event notification text, and wrap it with a unique identifier to help distinguish them; 3) Connect the word segmentation sequence with the prompt text to form the model input.

3. The document-level event argument extraction method integrating role capsules and cue gating as described in claim 2, characterized in that, In step S3, the input encoding layer further includes an encoding and decoding architecture based on Roberta, which inputs the concatenated word segmentation sequence and prompt text into the encoder, and then generates an event-related context representation and a prompt representation combined with the context through the decoder.

4. The document-level event argument extraction method integrating role capsules and cue gating as described in claim 1, characterized in that, The trigger word modeling layer in step S3 includes the following sub-steps: 1) Obtain the contextual representation of the trigger word. After input, the contextual representation and start and end positions are obtained through encoding. 2) Obtain the basic representation vector of the trigger word by performing average pooling on the representation region corresponding to the trigger word; 3) Extract the attention vector corresponding to the trigger word from the multi-head attention matrix, and perform a weighted summation with the context representation to obtain the context aggregation representation of the trigger word; 4) Concatenate the basic representation of the trigger word with its corresponding context aggregation representation to obtain the fused trigger word feature representation.

5. The document-level event argument extraction method integrating role capsules and cue gating as described in claim 1, characterized in that, In step S3, the gated cue fusion layer dynamically integrates the cue vector and the context representation to model the argument representation in a fine-grained manner: by averaging all attention heads and related sub-words, the context attention vector of the corresponding event trigger and the context attention vector of the argument slot in the corresponding cue template are extracted from the attention matrix. Then, the attention heads are used to perform an averaging operation on the related sub-word tags, thereby fusing the context clues and cue information to calculate the context-enhanced representation and argument extraction.

6. The document-level event argument extraction method integrating role capsules and cue gating, characterized in that, The gated cue fusion layer further includes: designing a gating mechanism to combine the decoder's output representation for argument slots in an event with the context aggregation representation, used to dynamically adjust the weighting of the two in the fusion process, calculated as follows: ； ； ； In the formula, , ∈ These are all learnable weight matrix parameters. yes function, This indicates element-wise multiplication.

7. The document-level event argument extraction method integrating role capsules and cue gating as described in claim 1, characterized in that, In step S3, the role-aware capsule network layer captures the deep structural association between candidate cues and trigger words through a dynamic routing mechanism, and relies on role priors to guide the construction of role representations, including the following sub-steps: 1) Obtain the trigger word representation of the event and the corresponding argument slot multiple candidate fusion hint representations, and obtain the joint representation of the candidate pair by concatenation and averaging operations; 2) Through a set of learnable linear transformations, each capsule is mapped to a role type, and each candidate argument representation is projected onto the space of each role capsule through a set of independent linear mappings to generate candidate role prediction representations; 3) Through a dynamic routing algorithm, the feature representations of candidate arguments are adaptively assigned to the corresponding role capsules in multiple iterations, ultimately generating a capsule representation that integrates role semantics.

8. The document-level event argument extraction method integrating role capsules and cue gating as described in claim 7, characterized in that, The dynamic routing algorithm includes the following sub-steps: 1) Initialize the routing coefficients and use the Softmax function to calculate the coupling weight between the candidate and the capsule; 2) Multiply the prediction vector of each candidate argument by the coupling weight assigned to the capsule, and sum over all candidate arguments to obtain the aggregated input vector of the capsule; 3) Use compression The function obtains the capsule output, performs a dot product with the role candidate prediction representation, measures their similarity, and updates the routing coefficients based on the similarity between the capsule output and the candidate prediction. 4) Based on the current role category, select the capsule output corresponding to its index as the role semantic vector for that argument slot.

9. The document-level event argument extraction method integrating role capsules and cue gating as described in claim 8, characterized in that, The dynamic routing iteration includes repeated execution to achieve dynamic allocation, and the calculation formula is: ； ； In the formula, It is the first The aggregated input vector of each capsule, It is the capsule output corresponding to the index. It represents the prediction of role candidates. Represents the j-th candidate argument The routing coefficient between the m-th capsule and the m-th capsule. These are the updated routing coefficients.

10. The document-level event argument extraction method integrating role capsules and cue gating as described in claim 1, characterized in that, In step S3, the boundary prediction and matching layer includes fusing the role semantic vector and slot representation to obtain the final representation of each slot, and converting it into a span selector to determine the start and end positions of arguments in the text; where the same event may have multiple arguments with the same role, a binary matching loss is used in combination with the Hungarian algorithm to find the optimal allocation optimization, thereby improving the recognition accuracy of multi-class and multi-quantity targets.

Images