Model, event argument extraction method and system based on multi-level attention mechanism

Abstract

The present invention discloses a model, event argument extraction method, and system based on a multi-level attention mechanism. First, the input text containing the event type and the event description is preprocessed, and the text in the dataset is encoded using a pre-trained language model to obtain the model's initial text representation. Second, the event type is input into the multi-level attention mechanism model to obtain event type-argument role-level attention features and argument role-argument role-level attention features. Then, the text representation is input into a bi-affine layer and fused with the event type-argument role-level attention features and argument role-argument role-level attention features to obtain the final fused classification features. Finally, the fused classification features are used as the input to the final classification layer, and the head and tail indexes of the event arguments of each role type are predicted using a 0 / 1 annotation format. Iterative training is performed to obtain the optimal model. This effectively improves the effect of extracting event arguments from documents.

Description

Technical Field

[0001] The present invention belongs to the research field of event argument extraction in information extraction in natural language processing, and specifically relates to a model based on a multi-level attention mechanism, an event argument extraction method and a system. Background Art

[0002] The internet has fully entered the era of big data, and everyone is overwhelmed by massive amounts of data. Data comes in a variety of forms, with text, as a key form of information, widely used across all industries. Faced with this vast array of text data, quickly obtaining the information people need is crucial. Information extraction tasks were born out of this need.

[0003] Event extraction is a core task in the field of information extraction. It aims to extract events from natural text and represent them in a structured form, such as a table. A complete event consists of an event trigger word that defines the event type and multiple event-related arguments. The trigger word and arguments are often entities. Based on the extraction phase, the event extraction task can be divided into two subtasks: event trigger word extraction and event argument extraction. Event argument extraction extracts all arguments related to the event, given the known event trigger word and event type.

[0004] Existing event argument extraction methods often make simple use of known information when extracting arguments. When introducing event category information, they simply concatenate its category vector onto the text representation, ignoring the hierarchical relationship between event types and event arguments defined in the event template. Furthermore, a single sentence often contains multiple arguments, each with varying degrees of domain and semantic correlation. This dependency information is often not incorporated into the modeling process. Therefore, how to rationally and efficiently utilize known event type information and the dependencies between arguments to assist in event argument extraction is of great research value. Summary of the Invention

[0005] The technical problem to be solved by the present invention is to provide a model, event argument extraction method and system based on a multi-level attention mechanism, which solves the problem of low classification accuracy of event argument extraction algorithms in the prior art.

[0006] The present invention adopts the following technical solutions to solve the above technical problems:

[0007] A method for constructing a model of a multi-level attention mechanism comprises the following steps: first, constructing an event type-argument role hierarchical relationship and an argument role-argument role hierarchical relationship, and representing them respectively using two-dimensional matrices; then, inputting the pre-acquired text representation and the event type-event argument hierarchical relationship matrix into an event type-event argument attention module, and calculating the event type-argument role hierarchical attention features; inputting the pre-acquired text representation and the event argument-event argument hierarchical relationship matrix into an event argument-event argument attention module, and calculating the argument role-argument role hierarchical attention features; finally, taking the event type-argument role hierarchical attention features and the argument role-argument role hierarchical attention features as the output of the model.

[0008] The specific process of calculating the event type-argument role level attention features is as follows:

[0009] Based on the official event template, the relationship between event types and argument roles is analyzed and represented using a two-dimensional matrix. When defining an event, the event template gives the arguments contained in a specific type of event. The two-dimensional relationship matrix is ​​constructed with the event type as the horizontal axis and the argument role as the vertical axis. If an event contains a certain argument, its value in the two-dimensional matrix is ​​set to 1, otherwise it is set to 0.

[0010] The specific process of calculating the argument role-argument role level attention features is as follows:

[0011] Analyze the dependency between argument roles and represent them using a two-dimensional matrix. Abstract the value attributes contained in the argument role into upper-level concepts to express the attributes of a certain dimension of the argument role. Construct a two-dimensional relationship matrix with the argument role type as the horizontal coordinate and the upper-level concept type as the vertical coordinate. If an argument has a certain attribute, its value in the two-dimensional matrix is ​​set to 1, otherwise it is set to 0.

[0012] The text representation is obtained by encoding the text in the original dataset using a pre-trained language model.

[0013] A method for extracting event arguments based on a multi-level attention mechanism includes the following steps:

[0014] Step 1: Preprocess the input text containing the event type and the description of the event, and use the pre-trained language model to encode the text in the dataset to obtain the model's initial text representation;

[0015] Step 2: Input the event type in step 1 into the model of the multi-level attention mechanism to obtain the event type-argument role level attention features and the argument role-argument role level attention features;

[0016] Step 3: Input the text representation obtained in step 1 into the dual affine layer and fuse it with the event type-argument role level attention features and argument role-argument role level attention features to obtain the final fused classification features;

[0017] Step 4: Use the fused classification features as the input of the final classification layer, use the 0 / 1 annotation format to predict the first and last indexes of the event arguments of each role type, iterate the training, and obtain the optimal model.

[0018] The specific process of step 1 is as follows:

[0019] The training set and test set are divided into long documents, and the long documents are separated into a set of sentences with a fixed length of 200 words. One sentence corresponds to a sample in the dataset. The pre-trained language model BERT is used for word embedding representation to obtain the initial text representation h.

[0020] For each sample, the event type-argument role hierarchical attention feature two-dimensional relationship matrix obtained by using its known event type e is searched to obtain the association vector between the event type and the argument role. Then, the semantic feature e corresponding to the argument role of the event type is obtained by looking up the table in the randomly initialized event type-argument role parameter matrix. uc , assuming that the event may contain k arguments, the text representation obtained in step 1 is combined with e uc Fusion and softmax function are used to calculate the attention score s of event type for argument role e ;

[0021] For each sample, the two-dimensional matrix of argument role-argument role level attention features is used to look up the randomly initialized argument role-argument role parameter matrix to obtain the semantic feature r of the association information between all arguments. uc ; Combine the text representation obtained in step 1 with r uc Fusion and use the softmax function to calculate the attention score s between argument roles based on the upper-level concept r and the argument-argument level characteristics of the sample e r ;

[0022] e r Concatenate it with the text representation h obtained in step 1 and calculate a probability matrix for each token in the text that provides the attention score between argument roles And for each candidate argument, use the max function to filter out the other argument with the greatest correlation, and obtain the argument-argument feature matrix h for the final classification r .

[0023] The specific process of step 3 is as follows:

[0024] The text representation obtained in step 1 is embedded into the input double affine layer, and a feedforward neural network is used to map it to a vector p that calculates the probability of each argument role. It is then fused with the event type-event argument level attention features and the event argument-event argument level attention features of the multi-level attention mechanism model to obtain the final fused classification features.

[0025] The specific process of step 4 is as follows:

[0026] The multi-level attention mechanism fuses the feature representation as the input of the final classification layer. The vector p obtained in step 3 is classified using multiple binary classifiers. The first and last indexes of the event arguments of each role type are predicted using a 0 / 1 annotation format. The optimal model is obtained through iterative training.

[0027] The event argument extraction system based on the multi-level attention mechanism includes a pre-trained language model, a span extraction module, a multi-level attention mechanism model, a feature fusion module, and an argument extraction module;

[0028] A pre-trained language model is used to receive external input including an event type and a text describing the event for pre-training to obtain event text representation;

[0029] The span extraction module is used to process the received text representation and obtain the initial classification features;

[0030] A multi-level attention mechanism model is used to receive event types and obtain two levels of features;

[0031] The feature fusion module is used to fuse the two-level features and the initial classification features to obtain the final fused classification features;

[0032] The argument extraction module is used to perform binary classification on the fused classification features to obtain the head and tail positions of the argument entities and extract the event argument parameters.

[0033] Compared with the prior art, the present invention has the following beneficial effects:

[0034] 1. This solution first explores the impact of introducing event types in templates on event roles, and uses the attention mechanism to establish event type-argument type contextual hierarchical features. Secondly, it explores the correlation between upper-level concepts between arguments, and uses the hierarchical attention mechanism to construct argument type-argument type contextual hierarchical features. Finally, by integrating multi-level attention mechanisms, the effect of extracting event arguments from documents is improved.

[0035] 2. The relationship between event type and event arguments, as well as the relationship between event arguments, are modeled separately through the attention mechanism. The obtained features will be fused with the text representation and used for the final event argument classification task, thereby achieving more accurate event argument extraction results.

[0036] 3. This solution proposes a model concept of a multi-level attention mechanism, taking into account the relevant information and concepts of the event argument extraction task. As an independent part, this model can be used in related fields and can effectively improve the processing performance of event argument extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0038] Figure 1 This is a flow chart of the event argument extraction method based on a multi-level attention mechanism of the present invention.

[0039] Figure 2 This is an abstract diagram of the event-argument hierarchy used in the present invention.

[0040] Figure 3 This is an abstract diagram of the argument-argument hierarchy used in the present invention.

[0041] Figure 4 This is a schematic diagram of the overall structure of the sentence-level event argument extraction task performed by the present invention. DETAILED DESCRIPTION

[0042] It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0043] It is understandable to those skilled in the art that some well-known structures and descriptions thereof may be omitted in the drawings.

[0044] In order to better illustrate this embodiment, the technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention.

[0045] It is worth noting that the concept of attention weight mentioned in this document can also be called attention score or attention feature in this field. Therefore, these three concepts appearing in this document represent the same meaning and are common concept replacements in this field. They will not cause unclear expressions or problems with variables and explanations.

[0046] A method for constructing a model of a multi-level attention mechanism comprises the following steps: first, constructing an event type-argument role hierarchical relationship and an argument role-argument role hierarchical relationship, and representing them respectively using two-dimensional matrices; then, inputting the pre-acquired text representation and the event type-event argument hierarchical relationship matrix into an event type-event argument attention module, and calculating the event type-argument role hierarchical attention features; inputting the pre-acquired text representation and the event argument-event argument hierarchical relationship matrix into an event argument-event argument attention module, and calculating the argument role-argument role hierarchical attention features; finally, taking the event type-argument role hierarchical attention features and the argument role-argument role hierarchical attention features as the output of the model.

[0047] The specific process of calculating the event type-argument role level attention features is as follows:

[0048] Based on the official event template, the relationship between event types and argument roles is analyzed and represented using a two-dimensional matrix. When defining an event, the event template gives the arguments contained in a specific type of event. The two-dimensional relationship matrix is ​​constructed with the event type as the horizontal axis and the argument role as the vertical axis. If an event contains a certain argument, its value in the two-dimensional matrix is ​​set to 1, otherwise it is set to 0.

[0049] The specific process of calculating the argument role-argument role level attention features is as follows:

[0050] Analyze the dependency between argument roles and represent them using a two-dimensional matrix. Abstract the value attributes contained in the argument role into a higher-level concept to express the attributes of a certain dimension of the argument role. Construct a two-dimensional relationship matrix with the argument role type as the horizontal coordinate and the higher-level concept type as the vertical coordinate. If an argument has a certain attribute, its value in the two-dimensional matrix is ​​set to 1, otherwise it is set to 0.

[0051] The text representation is obtained by encoding the text in the original dataset using a pre-trained language model.

[0052] The event argument extraction method based on the above multi-level attention mechanism includes the following steps:

[0053] Step 1: Preprocess the input text containing the event type and the description of the event, and use the pre-trained language model to encode the text in the dataset to obtain the model's initial text representation;

[0054] Step 2: Input the event type in step 1 into the model of the multi-level attention mechanism to obtain the event type-argument role level attention features and the argument role-argument role level attention features;

[0055] Step 3: Input the text representation obtained in step 1 into the dual affine layer and fuse it with the event type-argument role level attention features and argument role-argument role level attention features to obtain the final fused classification features;

[0056] Step 4: Use the fused classification features as the input of the final classification layer, use the 0 / 1 annotation format to predict the first and last indexes of the event arguments of each role type, iterate the training, and obtain the optimal model.

[0057] Specific embodiments, such as Figure 1 As shown,

[0058] The event argument extraction method based on the multi-level attention mechanism includes the following steps:

[0059] S1: Preprocess the input text containing the event type and the description of the event, and use the pre-trained language model to encode the text in the dataset to obtain the model's initial text representation;

[0060] S2: Use the existing official event template to construct the event type-argument role hierarchy and represent it using a two-dimensional matrix;

[0061] S3: Based on human experience and knowledge, the superior attributes of the arguments are abstracted, and the argument role-argument role hierarchical relationship is constructed through the superior attributes, and represented using a two-dimensional matrix;

[0062] S4: Input the event type and text representation in step S1 and the event type-argument role hierarchical relationship matrix obtained in step S2 into the event type-event argument attention module to calculate the event type-argument role hierarchical attention feature;

[0063] S5: Input the text representation obtained in step S1 and the argument role-argument role hierarchical relationship matrix obtained in step S3 into the event argument-event argument attention module, and calculate the argument role-argument role hierarchical attention feature;

[0064] S6: Embed the text representation obtained in step S1 into the input bi-affine layer and fuse it with the event type-event argument level attention feature obtained in step S4 and the argument role-argument role level attention feature obtained in step S5 to obtain the final classification feature;

[0065] S7: Use the fused classification feature representation obtained in step 6 as the input of the final classification layer, use the 0 / 1 annotation format to predict the first and last position indexes of the event argument of each role type, and iterate the training to obtain the optimal model.

[0066] The specific process of step S1 is as follows:

[0067] The dataset used for training the model is divided into a training set and a test set. The documents in the dataset are divided into sentence sets with a maximum length of 200 words. Arguments are extracted based on sentences. A sentence corresponds to a sample in the dataset. The pre-trained language model BERT is used for encoding, and each word is mapped to a fixed dimension d. h vector to obtain a universal semantic embedding text representation h:

[0068]

[0069] where h i is the word embedding representation corresponding to each word, tri indicates the position of the event trigger word, N indicates the length of the text sequence; the size of the text representation h is N*d h .

[0070] The specific process of step S2 is as follows:

[0071] The official event template is used to construct a two-dimensional relationship matrix between event type and argument role. The official event template presets the subordinate argument roles for each type of event, as shown in the diagram below. Figure 2 As shown, the actual argument role set of an event must be a subset of the template's preset argument role set. Based on the above theory, the hierarchical relationship between event types and argument roles is represented by a two-dimensional matrix; the horizontal axis of this two-dimensional relationship matrix represents the 33 event types and the vertical axis represents the 35 argument roles. If an argument role belongs to a certain event type, its corresponding position in the two-dimensional matrix is ​​set to 1, otherwise it is set to 0.

[0072] The specific process of step S3 is as follows:

[0073] Step S300: Construct an argument role-argument role two-dimensional relationship matrix based on cutting-edge research in the field.

[0074] Step S301: Argument roles are often not independent. Different argument roles are interrelated in some dimension, and this connection ultimately helps promote the joint extraction of arguments. Based on this theory, the upper-level concepts are abstracted into 8 categories (Person, Behavior, Entity, Good, Place, Org, Time, NA) according to expert design, as shown in the following diagram. Figure 3 shown.

[0075] Step S302: Design an argument role-argument role two-dimensional relationship matrix; the horizontal coordinate of the two-dimensional relationship matrix is ​​35 argument roles, and the vertical coordinate is 8 upper-level concepts. If an argument role contains an upper-level concept attribute, its corresponding position in the two-dimensional matrix is ​​set to 1, otherwise it is set to 0.

[0076] The specific process of step S4 is as follows:

[0077] Step S400: For each sample, use its known event type e to search in the event type-argument role level attention feature two-dimensional relationship matrix to obtain the association vector between the event type and the argument role, and calculate the event type-argument role level attention feature.

[0078] Step S401: Randomly initialize a size of num e (number of event types)*d h The two-dimensional query vector E (of the same dimension as the text representation h in step S1) e .

[0079] Step S402: In the query vector E according to the known event type e Query and expand on the first dimension to get a size of N*num r *d h The semantic feature vector e corresponding to the argument role uc , where num r is the number of argument types.

[0080] Step S403: Expand the text representation h from step S1 in the second dimension to obtain the same vector as e uc Vectors of the same size Then the two are concatenated and passed through the fully connected layer to obtain the feature vector h e , whose size is N*num r .

[0081] h e =tanh(W ae [h;e uc ])

[0082] Step S404: Use the softmax function to calculate the attention weight s of the event text towards the argument role e .

[0083]

[0084] Where i indicates the current argument type, k r Indicates the number of arguments of the current event type in the template.

[0085] The specific process of step S5 is as follows:

[0086] Step S500: Calculate argument role-argument role level attention features.

[0087] Step S501: For each sample, use the two-dimensional matrix of argument role-argument role level attention features and randomly initialize a matrix of size numc (number of upper-level concepts)*d h The two-dimensional query vector E (of the same dimension as the text representation h in step S1) r .

[0088] Step S502: The relationship between all arguments and upper-level concepts is recorded in the query vector E. c Query and get the size of num in the expanded dimension c *N*d h The semantic feature vector r of the association information between all arguments uc .

[0089] Step S503: Expand the text representation h from step S1 in the second dimension to obtain the same vector r uc Vectors of the same size Then the two are concatenated and passed through the fully connected layer to obtain the feature vector Its size is num c *N.

[0090] here

[0091] Step S504: Use the softmax function to calculate the attention weight of the argument role associated with the upper-level concept Size is num c *N.

[0092]

[0093] Where i represents the current position index and n represents the length of the current text sequence.

[0094] Step S505: For each argument, calculate the weighted average attention score s of all associated upper-level concepts r After dimension expansion, the size of the vector is N*num r (number of argument roles).

[0095]

[0096] Where i represents the current position index, k c Indicates the number of upper-level concept attributes contained in the current argument role, recorded as c1, c2, ..., c k .

[0097] Step S506: Expand the text representation h from step S1 in the second dimension to obtain a vector The vector s obtained in the previous step r Expand in the second dimension to get the vector Calculate the Hadamard product of the two to get a size of N*numr *d h The argument-argument level eigenvector e of r .

[0098]

[0099] Step S507: Dimensionally expand the text representation h from step S1 to obtain the vector The vector e obtained in the above steps r The vector obtained after dimension expansion Splicing and passing through the fully connected layer to obtain the feature vector This vector provides a probability matrix of attention scores between all argument roles for each token Its size is N*num r *num r .

[0100]

[0101] Step S508: For each candidate argument, use the max function to filter out the other argument with the greatest correlation, and obtain the character attention score matrix h with the highest correlation with each argument. r , size is N*num r .

[0102]

[0103] The specific process of step S6 is as follows:

[0104] Step S600: Combining Figure 4 The multi-feature fusion is described in detail to finally obtain the classification features.

[0105] Step S601: Input the text representation h from step S1 into two bi-affine layers respectively, and use a feedforward neural network to map it to a vector p′ that calculates the probability of each argument role, and obtain a probability matrix p′ corresponding to the first and last indexes. s / e , p′ s Indicates the start index, p′ e Indicates the end index, both sizes are N*num r *2.

[0106] p′=W1(tanh(W2·h+b2))+b1

[0107] Step S602: The event type-argument role feature vector s obtained in step S4 is e And the argument role-argument role feature vector h obtained in step 5 rFuse with the above vector p to obtain the final fused classification feature probability.

[0108] p=h r *(λ·s e +p′)

[0109] The specific process of step S7 is as follows:

[0110] The multi-level attention mechanism fuses the feature representation as the input of the final classification layer, and uses multiple binary classifiers to assign 0 / 1 labels to the first and last position indexes of each argument role.

[0111] y s / e =argmax(p s / e )

[0112] At this point, event argument extraction based on a multi-level attention mechanism has been completed. This approach fully exploits and utilizes the prior information inherent in the event itself. During the encoding process, the attention mechanism fully integrates the guidance of the event type on the argument role and the interrelationships between argument roles, enhancing semantic features and improving the accuracy and performance of event element extraction.

[0113] The event argument extraction system based on the multi-level attention mechanism includes a pre-trained language model, a span extraction module, a multi-level attention mechanism model, a feature fusion module, and an argument extraction module;

[0114] A pre-trained language model is used to receive external input including an event type and a text describing the event for pre-training to obtain event text representation;

[0115] The span extraction module is used to process the received text representation and obtain the initial classification features;

[0116] A multi-level attention mechanism model is used to receive event types and obtain two levels of features;

[0117] The feature fusion module is used to fuse the two-level features and the initial classification features to obtain the final fused classification features;

[0118] The argument extraction module is used to perform binary classification on the fused classification features to obtain the head and tail positions of the argument entities and extract the event argument parameters.

[0119] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in the present invention, and such modifications or substitutions are intended to be within the scope of protection of the present invention. Therefore, the scope of protection of the present invention shall be subject to the scope of protection of the claims.

Claims

1. A method for constructing a multi-level attention mechanism model, characterized by: First, we construct the event type-argument role hierarchical relationship and the argument role-argument role hierarchical relationship, and represent them using two-dimensional matrices respectively. Then, the pre-obtained text representation and event type-argument role hierarchical relationship matrix are input into the event type-event argument attention module to calculate the event type-argument role hierarchical attention features; The pre-acquired text representation and event argument-event argument hierarchical relationship matrix are input into the event argument-event argument attention module, and the argument role-argument role hierarchical attention features are calculated; finally, the event type-argument role hierarchical attention features and the argument role-argument role hierarchical attention features are used as the output of the model.

2. The method for constructing a multi-level attention mechanism model according to claim 1, wherein: The specific process of calculating the event type-argument role level attention features is as follows: Based on the official event template, the relationship between event types and argument roles is analyzed and represented using a two-dimensional matrix. When defining an event, the event template gives the arguments contained in a specific type of event. The two-dimensional relationship matrix is ​​constructed with the event type as the horizontal axis and the argument role as the vertical axis. If an event contains a certain argument, its value in the two-dimensional matrix is ​​set to 1, otherwise it is set to 0.

3. The method for constructing a multi-level attention mechanism model according to claim 1, wherein: The specific process of calculating the argument role-argument role level attention features is as follows: Analyze the dependency between argument roles and represent them using a two-dimensional matrix. Abstract the value attributes contained in the argument role into a higher-level concept to express the attributes of a certain dimension of the argument role. Construct a two-dimensional relationship matrix with the argument role type as the horizontal coordinate and the higher-level concept type as the vertical coordinate. If an argument has a certain attribute, its value in the two-dimensional matrix is ​​set to 1, otherwise it is set to 0.

4. The method for constructing a multi-level attention mechanism model according to claim 1, wherein: The text representation is obtained by encoding the text in the original dataset using a pre-trained language model.

5. A method for extracting event arguments based on a multi-level attention mechanism, characterized by: The steps include: Step 1: Preprocess the input text containing the event type and the description of the event, and use the pre-trained language model to encode the text in the dataset to obtain the model's initial text representation; Step 2: Input the event type in step 1 into the model of the multi-level attention mechanism described in any one of claims 1 to 4 to obtain event type-argument role level attention features and argument role-argument role level attention features; Step 3: Input the text representation obtained in step 1 into the dual affine layer and fuse it with the event type-argument role level attention features and argument role-argument role level attention features to obtain the final fused classification features; Step 4: Use the fused classification features as the input of the final classification layer, use the 0 / 1 annotation format to predict the first and last indexes of the event arguments of each role type, iterate the training, and obtain the optimal model.

6. The event argument extraction method based on a multi-level attention mechanism according to claim 5, characterized in that: The specific process of step 1 is as follows: The training set and test set are divided into long documents, and the long documents are separated into a set of sentences with a fixed length of 200 words. One sentence corresponds to a sample in the dataset. The pre-trained language model BERT is used for word embedding representation to obtain the initial text representation h.

7. The event argument extraction method based on a multi-level attention mechanism according to claim 6, characterized in that: For each sample, the event type-argument role hierarchical attention feature two-dimensional relationship matrix obtained by using its known event type e is searched to obtain the association vector between the event type and the argument role. Then, the semantic feature e corresponding to the argument role of the event type is obtained by looking up the table in the randomly initialized event type-argument role parameter matrix. uc , assuming that the event may contain k arguments, the text representation obtained in step 1 is combined with e uc Fusion and softmax function are used to calculate the attention score s of event type for argument role e ; For each sample, the two-dimensional matrix of argument role-argument role level attention features is used to look up the randomly initialized argument role-argument role parameter matrix to obtain the semantic feature r of the association information between all arguments. uc ; Combine the text representation obtained in step 1 with r uc Fusion and use the softmax function to calculate the attention score s between argument roles based on the upper-level concept r and the argument-argument level characteristics of the sample e r ; e r Concatenate it with the text representation h obtained in step 1 and calculate a probability matrix for each token in the text that provides the attention score between argument roles. And for each candidate argument, use the max function to filter out the argument with the greatest correlation, and obtain the argument-argument feature matrix h for the final classification. r .

8. The event argument extraction method based on a multi-level attention mechanism according to claim 5, characterized in that: The specific process of step 3 is as follows: The text representation obtained in step 1 is embedded into the input double affine layer, and a feedforward neural network is used to map it to a vector p that calculates the probability of each argument role. It is then fused with the event type-argument role level attention features and argument role-argument role level attention features of the multi-level attention mechanism model to obtain the final fused classification features.

9. The event argument extraction method based on a multi-level attention mechanism according to claim 5, characterized in that: The specific process of step 4 is as follows: The multi-level attention mechanism fuses the feature representation as the input of the final classification layer. The vector p obtained in step 3 is classified using multiple binary classifiers. The first and last indexes of the event arguments of each role type are predicted using a 0 / 1 annotation format. The optimal model is obtained through iterative training.

10. An event argument extraction system based on a multi-level attention mechanism, characterized by: It includes pre-trained language model, span extraction module, multi-level attention mechanism model, feature fusion module, and argument extraction module; among them, A pre-trained language model is used to receive external input including an event type and a text describing the event for pre-training to obtain event text representation; The span extraction module is used to process the received text representation and obtain the initial classification features; A multi-level attention mechanism model is used to receive event types and obtain two-level features; The feature fusion module is used to fuse the two-level features and the initial classification features to obtain the final fused classification features; The argument extraction module is used to perform binary classification on the fused classification features to obtain the head and tail positions of the argument entities and extract the event argument parameters.

Images