Document level event parameter extraction method and system based on context and role perception and storage medium

Abstract

The invention provides a document level event parameter extraction method and system based on context and role awareness and a storage medium, and the method comprises the steps: encoding an input document by using a pre-training model to output a context representation and an attention weight matrix, marking a trigger word to generate a trigger word representation, and generating an initial text fragment representation; obtaining the attention of the fragment of the text and the attention of the trigger word, and combining the attention of the fragment and the attention of the trigger word to obtain a trigger word-fragment local context representation; based on the attention of the role to the fragment and the attention of the fragment to the role, generating text fragment representation perceived by the role by using a bidirectional attention mechanism; and splicing the initial text fragment representation, the trigger word representation, the trigger word-fragment local context representation, the role-perceived text fragment representation, the distance representation between the text fragment and the trigger word and the element-by-element product representation of the text fragment and the trigger word, and then classifying to determine the role category of the event parameter. According to the method, the event parameters can be accurately extracted from the document.

Description

Technical Field

[0001] The present invention relates to the technical field of natural language processing, and in particular to a document-level event parameter extraction method, system and storage medium based on context and role perception. Background Art

[0002] Event parameter extraction technology is an important branch of information extraction tasks in natural language processing. Its main purpose is to automatically identify and extract important parameter information related to a specific event from unstructured text, such as the participating roles, the time and place of the event, etc., so as to systematically parse the core elements of the event.

[0003] With the development of big data and artificial intelligence, text-based event parameter extraction technology is widely used in public opinion monitoring, network security situation awareness, threat intelligence analysis and other fields. In these scenarios, it is crucial to quickly and accurately identify and understand event information. Event parameter extraction technology provides rich contextual information for various security analyses and provides strong support for identifying potential risks and threats.

[0004] However, traditional event parameter extraction methods are mostly limited to the sentence level. These methods are usually unable to effectively model complex contextual relationships and dependencies between multiple sentences. In particular, it is difficult to deeply understand the distribution of roles and their relationships in the text when identifying event parameter roles (such as victims, attackers, locations, etc.). In addition, traditional methods rely on rules or shallow machine learning models, which usually require manual construction of features or predefined labels, and are difficult to generalize to unseen syntactic structures and diverse expressions. These limitations make it difficult for traditional methods to cope with event data with higher diversity and complexity in actual scenarios.

[0005] In recent years, with the rise of deep learning technology, event parameter extraction methods based on neural networks have made significant progress, especially the introduction of pre-trained language models such as BERT (Bidirectional Encoder Representations from Transformers), which enables event parameter extraction models to better capture contextual information. However, the BERT model still faces some shortcomings when processing long texts and role classification, especially in document-level event parameter extraction. How to more effectively capture event context and role information is still a difficult problem to be solved. Summary of the invention

[0006] In view of this, an embodiment of the present invention provides a document-level event parameter extraction method and system based on context and role awareness to eliminate or improve one or more defects existing in the prior art.

[0007] One aspect of the present invention provides a document-level event parameter extraction method based on context and role awareness, the method comprising the following steps:

[0008] Encoding an input document using a pre-trained natural language model to generate and output a context representation and an attention weight matrix, marking trigger words in the input document to generate trigger word representations based on the tags of the trigger words, and generating an initial text fragment representation by aggregating the context representations;

[0009] Obtain the attention of the text segment and the attention of the trigger word, combine the attention of the trigger word and the attention of the text segment by element-by-element product, and weight the context representation based on the combined attention as a weight to obtain a local context representation of the trigger word-segment;

[0010] Based on the attention of the character to the fragment and vice versa, a bidirectional attention mechanism is used to fuse the character and fragment information to generate a character-aware text fragment representation.

[0011] The initial text segment representation, the trigger word representation, the local context representation of the trigger word-segment, the role-aware text segment representation, the distance representation between the text segment and the trigger word, and the element-by-element product representation of the text segment and the trigger word are concatenated and classified after concatenation to determine the role category of the event parameter.

[0012] In some embodiments of the present invention, the pre-trained natural language model is a BERT model, a RoBERTa model or an ALBERT model; marking the trigger words in the input document to generate the trigger word representation based on the markers of the trigger words includes: identifying the trigger word position in the input document by inserting special markers before and after the trigger words, and generating the trigger word representation based on the markers of the trigger words.

[0013] In some embodiments of the present invention, obtaining the attention of the text segment and the attention of the trigger word includes: obtaining the attention of the text segment by using an average pooling method for the attention weight vector corresponding to the text segment in the attention weight matrix; and taking the attention of the mark before the trigger word as the attention of the trigger word.

[0014] In some embodiments of the present invention, the context representation is weighted based on the combined attention as a weight to obtain a local context representation of the trigger word-fragment, including: weighting the context representation based on the combined attention as a weight to obtain a local context representation of the initial trigger word-fragment; using an effective channel attention network ECA-Net to weight the feature dimensions of the local context representation of the initial trigger word-fragment to obtain a weighted local context representation of the trigger word-fragment.

[0015] In some embodiments of the present invention, the bidirectional attention mechanism is used to fuse the role and fragment information based on the role's attention to the fragment and the fragment's attention to the role to generate a role-aware text fragment representation, including: using the role representation as a query vector, using the text fragment representation as a key vector and a value vector, generating a role-to-fragment attention weight matrix through a scaled dot product attention mechanism, and generating a context-aware role representation based on the generated role-to-fragment attention weight matrix and the text fragment representation; using the text fragment representation as a query vector, using the context-aware role representation as a key vector and a value vector, generating a fragment-to-role attention weight matrix through a scaled dot product attention mechanism, and generating a role-aware text fragment representation based on the generated fragment-to-role attention weight matrix and the context-aware role representation.

[0016] In some embodiments of the present invention, the initial text fragment representation, trigger word representation, local context representation of the trigger word-fragment, role-aware text fragment representation, distance representation between the text fragment and the trigger word, and element-by-element product representation of the text fragment and the trigger word are concatenated and classified, including: concatenating the initial text fragment representation, trigger word representation, local context representation of the trigger word-fragment, role-aware text fragment representation, distance representation between the text fragment and the trigger word, and element-by-element product representation of the text fragment representation and the trigger word representation; inputting the concatenated representation into a fully connected layer, and performing classification based on a focal loss function.

[0017] In some embodiments of the present invention, the method further includes: introducing a binary regularization term between the attention weight matrix of the character to the fragment and the attention weight matrix of the fragment to the character; the loss function used for classification after splicing is obtained based on the focal loss function and the weighted binary regularization term.

[0018] Another aspect of the present invention provides a document-level event parameter extraction system based on context and role awareness, the system comprising a processor, a memory and a computer program / instructions stored in the memory, the processor being used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method described above.

[0019] Another aspect of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which implements the steps of the method described above when the computer program / instruction is executed by a processor.

[0020] Another aspect of the present invention further provides a computer program product, comprising a computer program / instruction, which implements the steps of the above-mentioned method when executed by a processor.

[0021] The document-level event parameter extraction method and device based on context and role perception of the present invention can extract event parameters from documents more accurately and is suitable for fields such as automated threat intelligence analysis and network security situation awareness.

[0022] Additional advantages, purposes, and features of the present invention will be described in part in the following description, and will become apparent to those skilled in the art after studying the following, or may be learned from the practice of the present invention. The purposes and other advantages of the present invention may be achieved and obtained by the structures specifically indicated in the specification and the accompanying drawings.

[0023] Those skilled in the art will appreciate that the objectives and advantages that can be achieved with the present invention are not limited to the above specific description, and the above and other objectives that can be achieved by the present invention will be more clearly understood from the following detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The drawings described herein are used to provide a further understanding of the present invention, constitute a part of this application, and do not constitute a limitation of the present invention.

[0025] Figure 1 The figure is a flowchart of a document-level event parameter extraction method based on context and role perception in one embodiment of the present invention.

[0026] Figure 2 Schematic diagram of the architecture of a document-level event parameter extraction system based on context and role awareness in one embodiment of the present invention. DETAILED DESCRIPTION

[0027] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the embodiments and the accompanying drawings. Here, the illustrative embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0028] It should also be noted that, in order to avoid obscuring the present invention due to unnecessary details, only structures and / or processing steps closely related to the solutions according to the present invention are shown in the accompanying drawings, while other details that are not closely related to the present invention are omitted.

[0029] It should be emphasized that the term “include / comprises” when used herein refers to the presence of features, elements, steps or components, but does not exclude the presence or addition of one or more other features, elements, steps or components.

[0030] In order to overcome the shortcomings of the existing event parameter extraction methods, the present invention proposes a document-level event parameter extraction method based on context and role perception and a corresponding system, which can more accurately extract parameter role information in events by integrating the encoding module, the context perception module, the role perception module and the classification module. The method and system of the present invention are particularly suitable for text situations with complex contextual relationships involving cross-sentences and complex multi-arguments.

[0031] Figure 1 FIG. 1 is a flow chart of a method for extracting document-level event parameters based on context and role perception in one embodiment of the present invention. Figure 2 The following is the architecture diagram of the corresponding system. Figure 2 As shown, the system architecture of the present invention mainly includes the following four modules: an encoding module (encoder), a context-aware module, a role-aware module and a classification module. Among them, the encoding module is used to encode the input document using a pre-trained natural language model (such as a BERT model, a RoBERTa model or an ALBERT model), obtain the context representation and the attention weight matrix, and at the same time, generate the trigger word representation by marking the trigger word to enhance the model's perception of the event. The context-aware module is used to weight the context information between the trigger word and the fragment based on the attention mechanism, capture the context information through local context pooling, and further use the channel attention mechanism to enhance important features to obtain the local context representation of the trigger word-fragment. The role-aware module is used to adopt a dual attention mechanism so that the role and fragment information can interact with each other, enhance the model's understanding of the semantic role, and thus improve the extraction accuracy. The classification module integrates multiple features into the classification layer, and uses the focal loss function to solve the problem of category imbalance. Figure 2 In the output (span i ,role p ) and (span j ,role q ) represents the role categories of different clips. Furthermore, an L2 regularization term can be added to the classification module to prevent overfitting.

[0032] like Figure 1 As shown, a document-level event parameter extraction method based on context and role awareness according to an embodiment of the present invention comprises the following steps:

[0033] Step S110, using a pre-trained natural language model to encode the input document to generate and output a context representation and an attention weight matrix, marking trigger words in the input document to generate trigger word representations based on the markings of the trigger words, and generating an initial text fragment representation by aggregating the context representations.

[0034] This step can be implemented by an encoding module.

[0035] As an example, the pre-trained natural language model can be a pre-trained BERT model. The BERT model is used as a pre-trained model of the encoding module to construct the context representation and attention weight matrix of the document X to obtain rich semantic information. The output of the BERT model is as follows:

[0036] H,A=BERT(X) (1)

[0037] Among them, X is the input document, H is the contextual representation of the document, and A is the attention weight matrix.

[0038] The encoding module of the present invention also marks the trigger words in the input document, for example, inserting a special marker of the trigger word before and / or after the event trigger word, so that the BERT model can focus on the location of the trigger word to generate an accurate trigger word representation based on the marker of the trigger word. The use of this special marker helps the method to more clearly perceive the context in which the event occurred. As an example, the representation of the special marker before the trigger word can be extracted as the trigger word representation T.

[0039] The encoding module of the present invention also aggregates the context representation of the document by using an average pooling method to generate a text segment representation (which can be referred to as a segment representation) S. In order to distinguish it from the role-aware text segment representation described later, the text segment representation S is called the initial text segment representation, which can be expressed as:

[0040] S = AveragePooling(H) (2)

[0041] Among them, AveragePooling represents the average pooling operation.

[0042] Step S120, obtain the attention of the text segment and the attention of the trigger word, combine the attention of the trigger word and the attention of the text segment by element-by-element multiplication, and weight the context representation based on the combined attention as a weight to obtain a local context representation of the trigger word-segment.

[0043] This step can be implemented by a context awareness module.

[0044] As an example, the context-aware module can use average pooling to obtain the attention of text fragments. The calculation is as follows:

[0045]

[0046] Among them, start k , end kRespectively represent the starting index and ending index of the kth text segment, α i is the i-th row of the attention matrix A in formula (1), that is, the attention weight vector of the i-th word (token) to other words in the document.

[0047] For the attention weight of the trigger word, we can take the attention of the special mark before the trigger word and use it as the attention of the trigger word.

[0048] Next, the context-aware module uses an element-wise product operation to focus the attention of the trigger word and attention to text fragments Combined, and the combined attention weight is used to weight the context representation H to generate the local context embedding representation U of the trigger word-fragment (t,s) , that is, the context-aware module obtains key context information with higher weight by performing attention-based pooling on the local context of trigger words and text fragments. The calculation of the context-aware module is as follows:

[0049]

[0050] in, represents the combined attention weight matrix, represents the element-by-element product, H represents the context embedding representation calculated by formula (1), and H T represents the transpose of H.

[0051] The local context representation U of the trigger word-segment (t,s) The context-aware module may include multiple features corresponding to multiple trigger words and multiple fragments, and can be viewed as a feature map containing multiple channels (multiple features) from the feature dimension. Furthermore, optionally, in order to further improve the performance of the model, the context-aware module may also use the Efficient Channel Attention Network (ECA-Net) model to weight the feature dimensions to automatically learn important features and suppress irrelevant features, further enhancing the model's attention to key features. The feature dimensions are weighted to obtain the weighted trigger word-fragment local context representation U′ (t,s) The calculation of is as follows:

[0052] U′ (t,s) =ECA(U (t,s) ) (6)

[0053] In order to represent the local context of the weighted trigger word-segment U′ (t,s) To distinguish, the local context representation U of the trigger word-fragment (t,s)It can be called the local context representation of the initial trigger word-fragment.

[0054] The ECA-Net model in the context-aware module is used to adaptively adjust the weight of each channel through the ECA-Net channel attention mechanism, and perform channel weighting on the local context representation of the trigger word-fragment pair to selectively highlight important features and suppress unimportant or irrelevant features, so that the neural network can focus more on information related to the specific context. For a detailed description of the ECA-Net model, please refer to Wang Q, Wu B, Zhu P, et al. ECA-Net: Efficient channel attention for deep convolutional neural networks [C] / / Proceedings of the IEEE / CVF conference on computer vision and pattern recognition. 2020: 11534-11542, which will not be repeated here.

[0055] Step S130, based on the character's attention to the segment and the segment's attention to the character, a bidirectional attention mechanism is used to fuse the character and segment information to generate a character-aware text segment representation.

[0056] This step can be implemented by the role perception module.

[0057] More specifically, step S130 may include step S131 and step S132.

[0058] In step S131, the role-aware module uses the role representation as the query vector (query) and the initial text fragment representation as the key vector (key) and the value vector (value), generates an attention weight matrix of the role to the fragment through a scaled dot product attention mechanism, and generates a context-aware role representation based on the generated attention weight matrix of the role to the fragment and the initial fragment representation.

[0059] That is, based on the role-to-span attention, the role-aware module uses the role representation R as the query and the initial text segment representation S as the key and value, and fuses the role representation R with the initial text segment representation S through the scaled dot product attention mechanism to generate the role-to-span attention weight matrix, and generates the context-aware role representation R′ based on the generated role-to-span attention weight matrix and the initial text segment representation S. Based on the scaled dot product attention mechanism, the calculation is as follows:

[0060]

[0061] R′=W R2S S (8)

[0062] in, is the attention weight matrix of the role to the fragment, m is the number of argument roles, n is the number of fragments in the document, R is the role representation, S is the initial text fragment representation, d k is the embedding dimension of S. The role representation R is obtained as follows:

[0063] The argument role refers to the role played by the argument in the event referred to by the verb, such as agent, patient, object, experiencer, beneficiary, etc. Suppose the argument role set pre-set based on the data set is B = {b1, b2, ..., b i ,...,b m}, where m is the number of argument roles. Each argument role b i The representation is generated by an embedding layer Embedding(·), and the weights of the embedding layer are learnable parameters Where d is the embedding dimension. The specific calculation formula of the role representation R is as follows:

[0064] R=Embedding(b)=W r [Index(b)] (9)

[0065] in, is the embedding vector of role b, Index(b) represents the index of role b in the embedding table, W r is the embedding matrix, initialized to random values ​​and updated through training.

[0066] Step S132: Further, the role-aware module obtains a role-aware text segment representation based on the context-aware role representation and the segment's attention weight matrix to the role.

[0067] More specifically, based on the span-to-role attention, the role-aware module uses the initial text span representation S as the query vector query, the context-aware role representation R′ as the key vector key and the value vector value, generates a span-to-role attention weight matrix through the scaled dot product attention mechanism, and generates a role-aware text span representation S′ based on the generated span-to-role attention weight matrix and the context-aware role representation. That is, the role-aware module uses the span-to-role attention to integrate the context-aware role representation into the span representation to obtain the role-aware text span representation S′. Based on the scaled dot product attention mechanism, the calculations performed are as follows:

[0068]

[0069] S′=W S2R R′ (11)

[0070] in, is the attention weight matrix of the segment to the role, m is the number of argument roles, n is the number of segments in the document, R' is the context-aware role representation, calculated by formula (8), d k is the embedding dimension of R′, t Represents the transpose of R′.

[0071] In the role perception module, the bidirectional attention mechanism is used to enable the fusion of role and fragment information, realize the information interaction between the role and the text fragment, make the information transmission more complete, realize the construction of role-aware text fragment representation, and enhance the model's understanding of argument roles, thereby optimizing the recognition and classification of event parameter roles and ensuring the correct classification of event parameters.

[0072] Step S140, concatenates the initial text segment representation, the trigger word representation, the local context representation of the trigger word-segment, the role-aware text segment representation, the distance representation between the text segment and the trigger word, and the element-by-element product representation of the text segment and the trigger word, and classifies them after concatenation to determine the role category of the event parameter.

[0073] This step can be implemented by the classification module. The classification module concatenates the initial text segment representation S and the local context representation U′ of the trigger word-segment. (t,s) , trigger word representation T, role-aware text segment representation S′, distance representation between text segment and trigger word |ST|, and element-wise product representation of segment and trigger word Afterwards, the concatenated representations may be input into a fully connected layer, and a focal loss function may be used to classify the imbalanced data. In the embodiment of the present invention, the concatenation order of different representations during concatenation is not fixed and immutable, but may be flexibly set.

[0074] Through the fully connected layer, the concatenated features Classify and calculate the probability P(y ij ), as follows:

[0075]

[0076] Among them, Softmax is the activation function and FFNN represents the feed-forward fully connected layer.

[0077] Based on these multi-dimensional features, classification is performed through a fully connected layer, and the focal loss function is used to solve the category imbalance problem, which can effectively improve the method's sensitivity to rare categories and increase the robustness of parameter classification.

[0078] In the present invention, focal loss can be used as a loss function to pre-train the entire model. The trained model can solve the problem of class imbalance and improve the method's ability to distinguish minority classes. The formula of the loss function is as follows:

[0079]

[0080] Among them, α and β are adjustable weights, P(y ij ) is the probability that text segment i is predicted to be argument role category j, calculated by formula (12).

[0081] In addition, in some embodiments of the present invention, an additional regularization term, such as a binary regularization term, may be introduced between the attention weight matrices of the character-to-segment and the segment-to-character to force the model to learn features with more generalization capabilities by limiting the complexity of the model. As an example, the binary regularization term L2 is expressed as follows:

[0082]

[0083] Here, ||·||2 represents the L2 norm.

[0084] The loss of model training includes the following two items:

[0085]

[0086] Among them, λ is the weight of the regularization term.

[0087] Compared with the existing document-level event parameter extraction method, the document-level event parameter extraction method based on context and role perception proposed in the present invention introduces an attention-based context pooling method and combines it with a channel attention mechanism. Compared with the traditional maximum pooling method, context pooling and channel attention technology can focus on important features, so that the model can give higher weights to important context information and enhance the model's contextual semantic understanding ability. In addition, the method proposed in the present invention introduces a dual attention mechanism in the role perception module, which can more accurately capture the semantic role of the role and improve the accuracy of the model's parameter role classification.

[0088] For the above method, the present invention is verified on two public datasets, RAMS (see Ebner S, Xia P, Culkin R, et al. Multi-Sentence Argument Linking [C] / / Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 8057-8077.) and WikiEvents (see Li S, Ji H, Han J. Document-Level Event Argument Extraction by Conditional Generation [C] / / Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2021: 894-908.). On the RAMS dataset, the method of the present invention is used to extract multiple parameters in news events. The RAMS dataset contains 9124 events, covering 139 different event types and 65 argument role types. Event types include "conflict events" and "cooperation events", and role types include "locations" and "participants". The WikiEvents dataset is derived from the editing history of Wikipedia. It has 67 types of events and is widely used to verify the event parameter recognition and classification capabilities of the method. The experiment divides the above two datasets into training sets, validation sets, and test sets. The experimental results show that the performance of the method proposed in the present invention exceeds the existing baseline method, and the ablation experiment proves that each submodule in the method effectively improves the performance of the model.

[0089] In the experiment, the BERT-base-uncased model is used as a pre-trained model to encode the document. In the field of event parameter extraction, common extraction performance indicators are as follows:

[0090] Span F1 indicator requires that the predicted text segment completely matches the actual text segment.

[0091] The Head F1 indicator only focuses on the core word (head word) of the text segment. Even if the predicted text segment does not completely match the real text segment, as long as the core word is correct, the prediction is considered correct.

[0092] The Coref F1 index indicates that if the extracted parameters can be co-referenced with the true parameters (referring to the same entity or concept), the classification is considered correct.

[0093] For the RAMS dataset, the Argument Classification task (Arg CF) is evaluated, and Span F1 and Head F1 are evaluated on the validation set and test set. For the WikiEvents dataset, the Argument Identification task (Arg IF) and Argument Classification task (ArgCF) are evaluated respectively, and Head F1 and Coref F1 are used as evaluation indicators.

[0094] The baseline methods for comparison are as follows:

[0095] (1) BERT-CRF (see Ebner S, Xia P, Culkin R, et al. Multi-Sentence Argument Linking [C] / / Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 8057-8077.): It is a text sequence labeling method that combines the BERT model and the conditional random field (CRF) layer. This method uses the context encoding capability of BERT for preliminary feature extraction and optimizes the prediction of sequence labels through the CRF layer to ensure the legitimacy and consistency of the label sequence.

[0096] (2) A two-step approach for implicit event argument detection (Zhang Z, Kong X, Liu Z, et al. A two-step approach for implicit event argument detection [C] / / Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 2020: 7479-7485.): This method decomposes the event argument extraction task into two steps: argument keyword detection and keyword to complete phrase expansion, and achieved excellent performance on the RAMS dataset.

[0097] (3) Two-stream abstract meaning representation enhanced extraction model TSAR (see Xu R, Wang P, Liu T, et al. ATwo-Stream AMR-enhanced Model for Document-level Event Argument Extraction [C] / / Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. 2022: 5025-5036.): It is a method for document-level event argument extraction, which aims to solve the problems of long-distance dependencies between trigger words and arguments and interference context in documents. TSAR encodes documents from local and global perspectives through a two-stream encoding module, and constructs local and global AMR semantic graphs to capture intra-sentence and inter-sentence features. At the same time, the boundary information of text fragments is explicitly enhanced by introducing auxiliary boundary loss.

[0098] (4) BERT-QA (see Du X, Cardie C. Event Extraction by Answering (Almost) Natural Questions [C] / / Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 2020: 671-683.): It regards event parameter extraction as a question-answering task and extracts event parameters in an end-to-end manner.

[0099] (5) FEAE (see Wei K, Sun X, Zhang Z, et al. Trigger is not sufficient: Exploiting frame-aware knowledge for implicit event argument extraction [C] / / Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers). 2021: 4672-4682.): It is a frame-aware event argument extraction learning framework that identifies implicit arguments by reasoning at the event frame level. This method uses relevant arguments as clues to guide the reasoning process.

[0100] Table 1. Experimental results on the RAMS dataset

[0101]

[0102] The experimental results on the RAMS dataset are shown in Table 1. On the RAMS validation set, the proposed method outperforms all other existing methods participating in the experiment. Compared with the TSAR method, the Span F1 is improved by 0.71% and the Head F1 is improved by 0.75%. On the test set, compared with the TSAR method, the Span F1 is improved by 0.65% and the Head F1 is improved by 0.59%, which significantly improves the classification accuracy of event parameters and proves the effectiveness of the method.

[0103] Table 2. Experimental results on the WikiEvents dataset

[0104]

[0105] Head F1 and Coref F1 are used to evaluate parameter identification and classification tasks on the WikiEvents dataset, and the experimental verification results are shown in Table 2. On the WikiEvents dataset, the proposed method outperforms all other existing methods involved in the experiment. Compared with the TSAR method, the proposed method improves Head F1 by 0.31% and Coref F1 by 1.08% in the parameter identification task; Head F1 and Coref F1 of the parameter classification task are improved by 0.67% and 1.20%, respectively.

[0106] The experimental results of the above two data sets show that the method proposed in the present invention has better performance in event parameter extraction of data in different fields than the previous baseline methods.

[0107] In addition, the present invention verifies the contribution of each module of the method to the performance through ablation experiments on the WikiEvents dataset. The settings of the ablation experiment include removing ECA-Net, removing dual attention, and removing L2 regularization. The experimental results are shown in Table 3.

[0108] Table 3 Ablation experiment results on the WikiEvents dataset

[0109]

[0110] “Not removed” in Table 3 indicates that the system model of the present invention includes ECA-Net, dual attention mechanism and L2 regularization term.

[0111] The results in Table 3 show that the introduction of the ECA-Net module improves the performance of the model slightly, which indicates that the ECA module can highlight important features and suppress unimportant features by adaptively adjusting the weight of each channel, thereby making the neural network more focused on information related to a specific context.

[0112] The introduction of the dual attention module brings significant performance improvements, especially in the parameter classification task, where HeadF1 and Coref F1 are improved by 4.91% and 3.64% respectively. This result verifies the effectiveness of the dual attention module. By introducing the dual attention mechanism in the role perception module, the method can deeply interact between the fragment and the role information, so that each fragment can understand its semantic role more accurately by fusing the role-related features, which significantly enhances the method's ability to distinguish role classification.

[0113] The introduction of L2 regularization term also brings some improvement to the performance of the method. In the parameter classification task, Head F1 and Coref F1 are increased by 2.24% and 2.40% respectively. This shows that L2 regularization term can help the method prevent overfitting, thereby improving the generalization ability of the method.

[0114] These ablation experiment results further demonstrate the effectiveness of each module in the method of the present invention and verify that the method has excellent performance in the event parameter extraction task.

[0115] In an embodiment of the present invention, the BERT model is used as a pre-trained model to construct the context representation and attention weight matrix of document X, but the present invention is not limited to this. The RoBERTa model or the ALBERT model can also be used to construct the context embedding representation and attention weight matrix of document X.

[0116] Corresponding to the above method, the present invention also provides a document-level event parameter extraction system based on context and role perception, the system includes a computer device, the computer device includes a processor and a memory, the memory stores computer instructions, the processor is used to execute the computer instructions stored in the memory, when the computer instructions are executed by the processor, the system implements the steps of the method described above.

[0117] The embodiment of the present invention also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the aforementioned edge computing server deployment method are implemented. The computer-readable storage medium can be a tangible storage medium, such as a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a floppy disk, a hard disk, a removable storage disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0118] It should be understood by those skilled in the art that the exemplary components, systems and methods described in conjunction with the embodiments disclosed herein can be implemented in hardware, software or a combination of the two. Whether it is performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present invention are programs or code segments used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link via a data signal carried in a carrier.

[0119] It should be clear that the present invention is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present invention is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present invention.

[0120] In the present invention, features described and / or illustrated for one embodiment may be used in the same or similar manner in one or more other embodiments, and / or combined with features of other embodiments or replace features of other embodiments.

[0121] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the embodiments of the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A document-level event parameter extraction method based on context and role awareness, characterized in that: The method comprises the following steps: Encoding an input document using a pre-trained natural language model to generate and output a context representation and an attention weight matrix, marking trigger words in the input document to generate trigger word representations based on the tags of the trigger words, and generating an initial text fragment representation by aggregating the context representations; Obtain the attention of the text segment and the attention of the trigger word, combine the attention of the trigger word and the attention of the text segment by element-by-element product, and weight the context representation based on the combined attention as a weight to obtain a local context representation of the trigger word-segment; Based on the attention of the character to the fragment and vice versa, a bidirectional attention mechanism is used to fuse the character and fragment information to generate a character-aware text fragment representation. The initial text segment representation, the trigger word representation, the local context representation of the trigger word-segment, the role-aware text segment representation, the distance representation between the text segment and the trigger word, and the element-by-element product representation of the text segment and the trigger word are concatenated and classified after concatenation to determine the role category of the event parameter.

2. The method according to claim 1, characterized in that: The pre-trained natural language model is a BERT model, a RoBERTa model or an ALBERT model; The step of marking trigger words in the input document to generate trigger word representation based on the marks of the trigger words includes: identifying the trigger word position by inserting special marks before and after the trigger words in the input document, and generating the trigger word representation based on the marks of the trigger words.

3. The method according to claim 1, characterized in that The step of obtaining the attention of the text segment and the attention of the trigger word includes: The attention weight vector corresponding to the text segment in the attention weight matrix is ​​averaged and pooled to obtain the attention of the text segment; The attention of the token before the trigger word is taken as the attention of the trigger word.

4. The method according to any one of claims 1 to 3, characterized in that: The step of weighting the context representation based on the combined attention as a weight to obtain a local context representation of the trigger word-segment includes: The context representation is weighted based on the combined attention as a weight to obtain a local context representation of the initial trigger word-segment; The effective channel attention network ECA-Net is used to weight the feature dimensions of the local context representation of the initial trigger word-segment to obtain a weighted local context representation of the trigger word-segment.

5. The method according to claim 1, characterized in that: The method uses a bidirectional attention mechanism based on the attention of the character to the segment and the attention of the segment to the character to fuse the character and segment information to generate a character-aware text segment representation, including: Taking the role representation as the query vector and the initial text fragment representation as the key vector and value vector, a role-to-fragment attention weight matrix is ​​generated through a scaled dot product attention mechanism, and a context-aware role representation is generated based on the generated role-to-fragment attention weight matrix and the initial text fragment representation; Taking the initial text fragment representation as the query vector and the context-aware role embedding representation as the key vector and the value vector, a fragment-to-role attention weight matrix is ​​generated through a scaled dot product attention mechanism, and a role-aware text fragment representation is generated based on the generated fragment-to-role attention weight matrix and the context-aware role representation.

6. The method according to claim 1, characterized in that The initial text segment representation, the trigger word representation, the trigger word-segment local context representation, the role-aware text segment representation, the distance representation between the text segment and the trigger word, and the element-by-element product representation of the text segment and the trigger word are concatenated and classified, including: concatenating the initial text segment representation, the trigger word representation, the local context representation of the trigger word-segment, the role-aware text segment representation, the distance representation between the text segment and the trigger word, and the element-by-element product representation of the text segment representation and the trigger word representation; The concatenated representation is input into the fully connected layer and classified based on the focal loss function.

7. The method according to claim 1, characterized in that The method further includes: introducing a binary regularization term between the attention weight matrix of the character to the segment and the attention weight matrix of the segment to the character; The loss function used for classification after splicing is obtained based on the focal loss function and a weighted binary regularization term.

8. A document-level event parameter extraction system based on context and role awareness, comprising a processor, a memory, and a computer program / instruction stored in the memory, characterized in that: The processor is used to execute the computer program / instructions, and when the computer program / instructions are executed, the system implements the steps of the method according to any one of claims 1 to 7.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps of the method as claimed in any one of claims 1 to 7 are implemented.

Images