Template bridge-based machine reading comprehension event extraction method and device, and medium

Abstract

The application discloses a machine reading comprehension event extraction method and device based on template bridging, and medium, the method steps comprise: step S01. Input text and trigger word question template are obtained and combined to form an input instance with a trigger word question prompt template; step S02. Syntax analysis is performed on the input instance to form an adjacency graph, the input text and the trigger word question prompt template are connected according to the adjacency graph to obtain a link graph, and global features are extracted according to the link graph; the global features and semantic features are fused to identify the trigger word and the trigger word type; step S03. A argument question prompt template is generated, the prompt template is combined with the input text to form an input instance with a question prompt template; step S04. Argument extraction is performed to obtain argument and argument type output. The application has the advantages of simple implementation method, high event extraction efficiency and precision, good portability and the like.

Description

Technical Field

[0001] This invention relates to the field of natural language processing (NLP) technology, and in particular to a method, apparatus and medium for extracting machine reading comprehension events based on template bridging. Background Technology

[0002] Event extraction (EE) is an important and challenging task in natural language processing, widely applied in fields such as knowledge graph construction, information retrieval, question answering, and recommendation. Due to the semantic ambiguity of event mentions and the diversity of event structures, event extraction still faces significant challenges. The goal of event extraction is to extract structured event information from unstructured text. It is typically divided into two sub-tasks: event detection (ED) and argument extraction (EAE). ED aims to identify event trigger words and distinguish their types, while EAE aims to identify arguments for given event trigger words and classify their roles. Figure 1 For example, sentence S1 is a painful text containing the event subtype "START-POSITION", where the trigger word is "hired". In addition, S1 contains four entities: "1997", "company", "John D.Idol" and "chiefexecutive", which are event arguments, and they are assigned different roles such as "TIME", "ENTITY", "PERSON" and "POSITION".

[0003] Unlike traditional methods that rely on manually generated features, deep learning can automatically extract features. Therefore, deep learning models such as Convolutional Neural Networks (CNNs), Recurrent Neural Networks (RNNs), Graph Neural Networks (GNNs), and Pre-trained Language Models (PLMs) have achieved significant results in Natural Language Processing (EE). CNNs can automatically capture text features, but due to the limitations of their kernel windows, words in the information cannot contain long-distance dependencies. RNN-based models can obtain long-distance relationships between words using the order of text sequences; however, RNN models cannot effectively capture the dependencies between distant and discontinuous trigger words and their arguments. GNNs, on the other hand, apply multiple neurons to a graph structure. For event extraction tasks, GNNs typically use dependency trees between trigger words and their arguments to update node information, thus enabling GNN-based models to capture non-local grammatical relationships. PLMs can learn general language representations from large-scale unlabeled corpora; therefore, pre-training and fine-tuning paradigms are commonly used strategies in the field of natural language processing.

[0004] However, the above-mentioned event extraction method based on deep learning models has the following problems:

[0005] 1. Since event information is usually extracted directly from the text, the question template and the plain text are independent of each other. The semantic relationship between the question template and the plain text is not fully utilized, and there is a lack of prompts for the argument role by trigger words and event types. Especially for argument roles with few or no examples during the training phase, the efficiency and accuracy of actual event extraction are not high.

[0006] 2. Traditional deep learning models have poor portability. Even with pre-training and fine-tuning paradigms, different models need to be built for specific downstream tasks. Summary of the Invention

[0007] The technical problem to be solved by this invention is: in view of the technical problems existing in the prior art, this invention provides a machine reading comprehension event extraction method, device and medium based on template bridging that is simple to implement, has high event extraction efficiency and accuracy, good extraction performance and good portability.

[0008] To solve the above-mentioned technical problems, the technical solution proposed by this invention is as follows:

[0009] A method for extracting machine reading comprehension events based on template bridging, comprising the following steps:

[0010] Step S01. Trigger word question generation: Obtain the input text and the trigger word question prompt template and combine them to form an input instance with the trigger word question prompt template;

[0011] Step S02. Event Detection: Perform syntactic analysis on the input instance with trigger word prompts to form an adjacency graph describing the relationships between words in the input instance. Based on the adjacency graph, connect the input text with the trigger word prompt template to obtain a link graph, bridging the question with the context and reconstructing the relationship between the question and the text. Then, extract global features from the input instance based on the link graph. After fusing the global features with the semantic features of the input instance, identify the trigger word and its type.

[0012] Step S03. Argument Question Generation: Generate an argument question prompt template based on the identified trigger words and trigger word types, and combine the argument question prompt template with the input text to form an input instance with the argument question prompt template;

[0013] Step S04. Argument Extraction: Extract arguments from the input instance with the argument question prompt template to obtain the arguments and argument types.

[0014] Furthermore, in step S02, when connecting the input text and the trigger word question prompt template, the relationship between the word embeddings of each word in the input text and the word embeddings of the trigger word question is converted into an adjacency matrix A.n×n Where n is the number of words in the input instance QA, and the adjacency matrix A n×n The matrix elements in are Among them, v i v j These are the i-th word and the j-th word in the input instance QA, respectively. E includes poss, nsubj, dobj, prompt, and other relations. poss indicates that the possessive modifier is a possessive determiner, nsubj indicates a noun subject, dobj indicates a direct object, and prompt indicates a prompt question template for the event extraction task.

[0015] Furthermore, in step S02, GCN is used to extract global features from the input instances, wherein the following formula is used when updating node information:

[0016]

[0017] in, This represents the representation of node v at layer (l+1). Let V represent the optimized representation of node v in layer l+1, where l represents the layer number of the graph neural network, v represents a node (i.e., a word), and V represents the set of nodes. I is the identity matrix. D is the degree matrix of A, where A is the adjacency matrix obtained by transforming the relationship between the word embeddings of the input instance and the word prompt question. ii =∑ j A ij d represents the length of the question template, W (l) This represents the weight matrix of the l-th layer. This indicates that the (l+1)th layer contains overall information about the prompt question template, ReLU represents the activation function, [CLS] represents the information aggregation node, and g() represents the feature transformation function.

[0018] Furthermore, when extracting global features from input instances using GCN, the process also includes using a gating mechanism to filter out interfering terms according to the following formula:

[0019]

[0020] Among them, g(h v ) represents the gate function. Let v represent the normalized representation of node v, n represent the length of the input instance, and α and ∈ be the trainable parameters. h represents the square root of the representation of all neighboring nodes of v. u Let v represent the neighboring nodes of v, u represent the neighboring nodes of v, and N(v) represent the set of all neighboring nodes of v.

[0021] Furthermore, in step S02, the global features and the semantic features of the input instance are fused using an attention mechanism, and the calculated expression is:

[0022]

[0023] CrossAttention(S,G)=γAttention S +λAttention G

[0024] in, Represents semantic features after linear transformation. Represents global features, where M is the number of words in the sentence, and d k The dimension of the word embedding is represented by CrossAttention(S,G), which represents the feature Q. S and K G The fusion features, where γ and λ represent weighting coefficients;

[0025] Trigger words are predicted and identified during the trigger word recognition process using the following formula:

[0026]

[0027] output trigger =max(a p1 ,a p2 ,…,a pn )

[0028] Among them, a pi w represents the i-th word i It represents the probability of the trigger word, and n represents the length of the input instance. These are trainable parameters, and H is a. i The dimension, N is the number of trigger word types, output trigger This indicates the trigger words that have been predicted and identified.

[0029] Furthermore, step S04 includes:

[0030] Step S401.EAE means: Obtain the semantic features E of the input instance. A And the POE features of each word in the input instance, wherein the POE features are used to indicate whether the word is a trigger word;

[0031] Step S402. Feature Fusion: Combine the semantic features E of the input instance. A The keyword feature E is obtained by fusing the POE features of each word. C Using the semantic feature E respectively A POE features and keyword features EC Obtain the representation of the extracted event arguments;

[0032] Step S403. Argument prediction and identification: Based on the representation extracted from the event arguments, predict and identify the paper and argument type.

[0033] Furthermore, in step S402, the keyword feature E is obtained according to the following formula. C :

[0034] E C =α⊙Relu(multiply(E A T A ))+β⊙E A

[0035] Where α and β represent learnable parameters, and multiply represents the multiplication operation;

[0036] The extracted event arguments are represented as follows:

[0037]

[0038] in, This indicates the concatenation of different data types, where nor represents the normalization operation, and n represents the length of the input instance.

[0039] Furthermore, in step S403, argument prediction and identification are performed according to the following formula:

[0040] P S (w i ) = softmax(f i W s )

[0041] P E (w i ) = softmax(f i W e )

[0042] Among them, P S (w i ) and P E (w i ) represents the i-th word w i As the probability of the start and end of an argument, and These are trainable parameters;

[0043] The span of argument a includes the initial offset S a and end offset E a The initial offset S a and end offset E a The following rules must be met:

[0044] 0≤S a ≤n and 0≤E a ≤n;

[0045] E a -S a ≥0;

[0046] P S (S a )≥threshold≥max(P S ([CLS]), P S ([SEP]))

[0047] P E (E a )≥threshold≥max(P E ([CLS]), P E ([SEP]));

[0048] and E a and S a It belongs to argument a and cannot belong to any other argument.

[0049] A machine reading comprehension event extraction device based on template bridging, comprising:

[0050] The trigger word question generation module is used to obtain the input text and the trigger word question prompt template and combine them to form an input instance with the trigger word question prompt template;

[0051] The event detection module performs syntactic analysis on input instances with trigger word prompts, forming an adjacency graph describing the relationships between words in the input instance. Based on the adjacency graph, it connects the input text with the trigger word prompt template to obtain a link graph, bridging the question with the context and reconstructing the relationship between the question and the text. Then, it extracts global features from the input instance based on the link graph. After fusing the global features with the semantic features of the input instance, it identifies the trigger word and its type.

[0052] The argument question generation module is used to generate argument question prompt templates based on the identified trigger words and trigger word types, and combine the argument question prompt templates with the input text to form an input instance with argument question prompt templates;

[0053] The argument extraction module is used to extract arguments from the input instance with the argument question prompt template, and output the arguments and argument types.

[0054] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0055] Compared with existing technologies, the advantages of this invention are as follows: Based on machine reading comprehension methods, this invention reconstructs the relationship between questions and text by connecting question templates with input text during event extraction. This eliminates the independent treatment of words in the question templates as independent entities, instead providing a complete representation of the template through full connections. By employing a question-context bridging approach, the question templates and text are closely linked, thereby fully utilizing the semantic correlation between the prompt question templates and the input text to provide hints about the roles of trigger words and event types. This significantly improves the efficiency and accuracy of event detection and extraction, enhances event extraction performance, and the prompt question templates have good portability, further improving the model's portability. Attached Figure Description

[0056] Figure 1 This is a schematic diagram illustrating an example of event extraction in a specific application embodiment.

[0057] Figure 2 This is a schematic diagram illustrating the implementation process of the machine reading comprehension event extraction method based on template bridging in this embodiment.

[0058] Figure 3 This is a detailed flowchart illustrating the process of machine reading comprehension event extraction in a specific application embodiment of the present invention.

[0059] Figure 4 This is a schematic diagram illustrating the principle of event detection and argument extraction in this embodiment.

[0060] Figure 5 This is a schematic diagram illustrating an example of how the prompt question template is bridged with the context in this embodiment.

[0061] Figure 6 These are schematic diagrams illustrating the event detection ablation experimental results obtained using the present invention and the traditional model in specific application embodiments. (a) corresponds to the trigger word classification result, and (b) corresponds to the trigger word recognition result.

[0062] Figure 7 This is a schematic diagram of the Arg-C F1 score and Arg-ID F1 score results obtained from the ablation experiment in a specific application embodiment. Detailed Implementation

[0063] The present invention will be further described below with reference to the accompanying drawings and specific preferred embodiments, but this does not limit the scope of protection of the present invention.

[0064] like Figures 2-3 As shown, the steps of the machine reading comprehension event extraction method based on template bridging in this embodiment include:

[0065] Step S01. Trigger word question generation: Obtain the input text and the trigger word question prompt template and combine them to form an input instance with the trigger word question prompt template.

[0066] By configuring the question template construction strategy and designing corresponding question templates for different event domains, the event extraction model using question-and-answer formalization can be made highly versatile and well-suited for various types of event domains. In this embodiment, to achieve trigger word recognition, a set of five question templates is used: "Which word is the trigger word in the sentence?", "Which is the trigger word?", "Trigger word in the sentence?", "Trigger word", and "Verb". The trigger word question prompt template is selected from these templates and combined with the input text to form an input instance containing the trigger word question format.

[0067] by Figure 1 For example, the input text is "In 1997, the company hired John D. Idol to take over as chief executive." The second template, "which is the trigger?", is selected as the trigger word hint template. The input text and the trigger word hint template are combined to obtain the input instance. The [CLS] marker marks the starting point of the instance, and the [SEP] marker separates the input text from the trigger word hint template. The specific input instance obtained is as follows:

[0068] [CLS]which is the trigger? [SEP]In 1997, the company hired John D.Idolto take over as chief executive.[SEP]

[0069] Again Figure 3 For example, the input text is "Shaquille O'Neal's wife gave birth to a baby boy early Saturday.", which, when combined with the trigger word prompt question template "which is the trigger?", forms the input instance "[CLS]which is the trigger? [SEP]Shaquille O'Neal's wife gave birth to a baby boy early Saturday. [SEP]".

[0070] Step S02. Event Detection: Perform syntactic analysis on the input instance with trigger word prompts to form an adjacency graph describing the relationships between words in the input instance. Based on the adjacency graph, connect the input text with the trigger word prompt template to obtain a link graph, which bridges the question with the context and reconstructs the relationship between the question and the text. Then, extract global features from the input instance based on the link graph. After fusing the global features and the semantic features of the input instance, identify the trigger word and the trigger word type.

[0071] Event detection is used to identify trigger words in a single sentence and classify the identified trigger words into specific categories, that is, to identify trigger words and their types. For example... Figure 4 As shown, the event detection process is as follows: Figure 4 As shown, the event detection in this embodiment specifically includes the following steps:

[0072] Step S201. Representation of event detection: Perform syntactic analysis on the input instance with the trigger word prompt to form an adjacency graph. Based on the adjacency graph, connect the input text with the trigger word prompt template to obtain a link graph.

[0073] Syntax analysis is used to analyze the dependencies between words in a sentence. In this embodiment, syntax analysis is first performed on an input instance with trigger word hints. This allows us to obtain the syntactic dependencies between words in the input text, the information aggregation nodes in the input file, and the semantic aggregation relationships of the trigger word hint template. In specific application embodiments, NLP tools can be used to perform syntax analysis on the input instance, and the adjacency graph can be represented in the form of an adjacency matrix.

[0074] Considering the loose connection between the input text and the question in the input instance, the template of the instance should have relative completeness with respect to the corresponding text. However, the discrete state of the template may lead to the decomposition of its original semantics and the generation of more interference information. This embodiment, based on machine reading comprehension methods (i.e., question-answer pairs or prompt models), further employs a question-context bridging method after syntactic analysis. This method connects the input text and the prompt question template in the input instance according to their syntactic dependencies, treating the template as an inseparable semantic component of the text and reconstructing the relationship between the question and the text. Through this approach, words in the question template are no longer treated as independent entities, but rather a complete representation of the template is obtained through full connections. This tightly links the question template with the text, fully utilizing the semantic correlation between the prompt question template and the input text to provide hints about the roles of trigger words and event types, thereby significantly improving the efficiency and accuracy of event detection.

[0075] In this embodiment, when bridging the question with the context, the information aggregation nodes in the input text are first obtained. Specifically, the average value of all node representations can be copied into the CLS to obtain the overall semantics of the question prompt template. At the aggregation nodes, the relationship between the word embeddings of each word in the input text and the word embeddings in the prompt question template is converted into an adjacency matrix A. n×n This is achieved by performing a full join operation on the template and text, where n is the number of words in the input instance QA, determined by the adjacency matrix A. n×n This allows us to obtain the relationship between each word in the input text and the prompt question template, resulting in a link graph representing the association between the prompt question template and each word in the input text. Adjacency matrix A n×n The matrix elements in the matrix are specifically: Among them, v i , vj Let i and j be the i-th and j-th words in the input instance QA, respectively. E includes poss, nsubj, dobj, prompt, and other syntactic dependencies between words. poss indicates that the possessive modifier is a possessive determiner, nsubj represents the noun subject, dobj represents the direct object, and prompt represents the prompt question template for the Event Extraction (EE) task. A complete connection between the template and the text is achieved by setting the relation values ​​of all word nodes in the text to 1 with the CLS node in the adjacency matrix. This bridging strategy explicitly builds a close connection between the prompt question template and the input text, transforms the node relationships (syntactic dependencies) between the graphs, reshapes the semantic relationship between the prompt template and the text, and reconstructs the relationship between the question and the text, effectively enhancing the prompting effect of the prompt template.

[0076] by Figure 5 For example, the blue lines represent grammatical dependencies between words. The orange lines indicate the semantic convergence of the prompt question template, bringing information together at the [CLS] node. The green lines connect the template and text through a full join operation on the [CLS] node, i.e., by Se... i and Qe i Representing the word embeddings of the i-th word in the text and the question respectively, according to... The relationship between the input text and the prompt question template is converted into an adjacency matrix A. n×n This allows for bridging the input text with the prompt question template, resulting in a link diagram between the input text and the prompt question template.

[0077] Step S202. Feature Fusion: Extract global features from the input instance based on the link graph.

[0078] After the event detection representation in step S201, the global features of the input instance are obtained using this event detection representation. In this embodiment, after connecting the input text and the trigger word question prompt template to obtain a link graph, the global features are further extracted from the input instance based on the link graph.

[0079] Traditional graph neural network methods directly extract global features from the word relationship graph obtained through syntactic analysis, but this breaks down the semantic connection between the template and the text. In this embodiment, after the bridging strategy between the text and the question prompt template in step S201, a link graph representing the connections between nodes is formed. Based on this link graph, to obtain the global features of the input instance, this embodiment uses a GCN (Graph Convolutional Network) and improves the node update method to make it suitable for global feature extraction of instances after bridging the text and the question prompt template, thus minimizing syntactic structural information.

[0080] Specifically, the node update formula of the classic GCN model is shown in equation (1). In order to match the bridging strategy between the text and the prompt question template, this embodiment further adopts the node information update formula shown in equation (2) to update the node information:

[0081]

[0082] in, This represents the representation of node v at layer (l+1). Let V represent the optimized representation of node v in layer l+1, where l represents the layer number of the graph neural network, v represents a node (i.e., a word), and V represents the set of nodes. I is the identity matrix. D is the degree matrix of A, where A is the adjacency matrix obtained by transforming the relationship between the word embeddings of the input instance and the word prompt question. ii =∑ j A ij d represents the length of the question template, W (l) This represents the weight matrix of the l-th layer. This indicates that the (l+1)th layer contains overall information about the prompt question template, ReLU represents the activation function, [CLS] represents the information aggregation node, and g() represents the feature transformation function.

[0083] This embodiment combines the advantages of MRC and GCN. By utilizing GCN, a more effective event structure can be constructed for templates and text, enabling a more reasonable semantic representation for the template. This representation can then be fully utilized as a clue to quickly and accurately find trigger words.

[0084] Even with the GCN model described above, obtaining global features still presents issues such as redundant information in the problem template, errors in syntax analysis, and incomplete equivalence between adjacent nodes. To mitigate these problems, this embodiment further employs a gating mechanism to filter out interference terms when extracting global features from input instances using GCN. The calculation expression can be represented as:

[0085]

[0086] Among them, g(h v ) represents the gate function. Let v represent the normalized representation of node v, n represent the length of the input instance, and α and ∈ be the trainable parameters. h represents the square root of the representation of all neighboring nodes of v. u Let v represent the neighboring nodes of v, u represent the neighboring nodes of v, and N(v) represent the set of all neighboring nodes of v.

[0087] This embodiment uses the GCN model with gating mechanism described above to extract global features. This can effectively aggregate the global information of nodes while filtering out connections of irrelevant nodes, reducing redundant information in the problem template, errors in syntax analysis, and incomplete equivalence of adjacent nodes, thus effectively mitigating the impact of invalid bridging on GCN.

[0088] To achieve trigger word recognition, in addition to acquiring global features, semantic features also need to be acquired. Semantic features can be obtained by extracting contextual information from a sentence using a pre-trained language model. Semantic features and global features belong to different domains. To integrate features from different domains, this embodiment uses an attention mechanism to fuse global and semantic features, achieving cross-domain fusion. Inspired by self-attention mechanisms, this embodiment specifically employs the following attention mechanism to aggregate different features:

[0089]

[0090] CrossAttention(S,G)=γAttention S +λAttention G (8)

[0091] in, This represents the semantic features after linear transformation, and can be initialized using the same pre-trained model. Representing global features, specifically initialized using GCN, the parameter W is learnable. M is the number of words in the sentence, and d... k The dimension of the word embedding is represented by CrossAttention(S,G), which represents the feature Q.S and K G The fusion features are represented by γ and λ, which are weighting coefficients.

[0092] The output obtained after fusing in the above manner is the semantic feature with weighted assignment, where each value represents two different features Q. S and K G The tightness between them.

[0093] Step S203. Trigger word prediction and recognition:

[0094] This embodiment uses a linear classifier to predict the result of the trigger word ED. Assume A... Cross =(a1,a2,…,a n ) is a sequence related to trigger word recognition, while F EPT =(f1,f2,…,f n ) is the sequence used for argument extraction. Trigger word prediction measures the probability that each word in the text will be identified as a trigger word. Since arguments can typically consist of multiple words, argument prediction requires identifying the start and end words of the arguments. Therefore, trigger word and argument prediction require different methods. In this embodiment, trigger words are predicted and identified according to the following formula:

[0095]

[0096] output trigger =max(a p1 ,a p2 ,…,a pn (10)

[0097] Among them, a pi w represents the i-th word i It represents the probability of the trigger word, and n represents the length of the input instance. These are trainable parameters, and H is a. i The dimension, N is the number of trigger word types, output trigger This indicates the trigger words identified in the prediction.

[0098] In this embodiment, the model specifically uses the cross-entropy loss function as the objective function. For trigger word recognition, a cross-entropy loss function for multi-class scenarios is adopted:

[0099]

[0100] Where N is the number of sentences in the training dataset. M represents the number of event types in the dataset. ij =1 indicates that the label of i is j, otherwise y ij =0.

[0101] Step S03. Argument Question Generation: Generate an argument question prompt template based on the identified trigger words and trigger word types, and combine the argument question prompt template with the input text to form an input instance with the argument question prompt template.

[0102] Unlike trigger word recognition, the question templates for argument extraction depend on the results of the event extraction (EE) task. Having already obtained the event types and trigger words of the sentence through S02, this embodiment further designs a series of question templates for different event types. Figure 1 For example, the event type "START-POSITION" typically has four arguments: "TIME", "ENTITY", "PERSON", and "PLACE". This embodiment designs four question templates for this type of event: "When did the event occur?", "Where did the event occur?", "Who is the employer?", and "Who is the employee?". The combination of argument question prompt templates and sentences constitutes the input instance for argument extraction, as shown in Table 1.

[0103] Table 1: Examples of templates for argument problems

[0104] Entity types Input instances Time [CLS]When the event takes place? [SEP]the sentence[SEP] Place [CLS]Where the event takes place? [SEP]the sentence[SEP] Person [CLS]Who is the employee? [SEP]the sentence[SEP] Entity [CLS]Who is the employer? [SEP]the sentence[SEP]

[0105] And with Figure 3 For example, for the "TIME" type, the question template is "When did birth happen?", for the "PLACE" type, the question template is "When the event takes place about birth?", and for the "PERSON" type, the question templates are "who is born about birth" and "who is the person about birth". Then, the argument prompt question templates are combined with the input sentence to form the input instance for argument extraction.

[0106] Step S04. Argument Extraction: Extract arguments from the input instance with the argument question prompt template to obtain the arguments and argument types.

[0107] Step S401.EAE means: Obtain the semantic features E of the input instance. A And the POE features of each word in the input instance, which are used to indicate whether a word is a trigger word.

[0108] like Figure 4As shown, the EAE (Event Argument Extraction) representation in this embodiment consists of three elements: POS (Part-of-Speech Tagging), the position of the event trigger word (TPOE), and semantic features. POS is the part-of-speech tag for the words in the input instance. For computational convenience, a simple dictionary of POS is constructed, containing multiple POS values ​​and mapping them to unique values. POE refers to the position of the event trigger word to facilitate the recognition of event arguments. Semantic features are similar to the ED module, using a pre-trained language model to obtain the semantic information of each word in the input.

[0109] For ease of description, let W = (w1, w2, ..., w n ) is the word sequence of the instance, E A = (e1, e2, ..., e n ) is the semantic feature of W, and E is obtained by applying the BERT model. A This allows us to obtain contextualized representations of words in a text. TP A = (p1, p2, ..., p n The ) indicates the POS obtained using spaCy (an advanced NLP library). A = (t1, t2, ..., t n ) is the POE for each word in the instance, where t i ∈(0,1),t i =0 means w i t is not a trigger word in the input. i =1 means w i It is a trigger word.

[0110] Step S402. Feature Fusion: Combine the semantic features E of the input instance. A The keyword features E are obtained by fusing the POE features of each word. C Using semantic features E respectively A POE features and keyword features E C The representation of the event arguments is obtained.

[0111] The argument information of an event is typically determined by the type of the trigger word. This embodiment constructs a feature E based on the event trigger word. C This refers to keyword features, which can help semantic features focus more on information closely related to trigger words.

[0112] In a specific application embodiment, the keyword feature E can be obtained according to the following formula. C :

[0113] E C =α⊙Relu(multiply(E A T A ))+β⊙EA (12)

[0114] Here, α and β represent learnable parameters, and multiply represents the multiplication operation.

[0115] The three features mentioned above are used together to form the final representation of event argument extraction. Specifically, the representation of event argument extraction can be expressed as:

[0116]

[0117] in, This indicates the concatenation of different data types, where nor represents the normalization operation, and n represents the length of the input instance.

[0118] This embodiment constructs an information fusion mechanism centered on trigger words during the argument extraction process and adopts a feature fusion method based on core vocabulary. This can highlight the guiding role of trigger words on arguments, thereby improving the efficiency and accuracy of detection.

[0119] Step S403. Argument prediction and identification: Based on the representation extracted from the event arguments, predict and identify the paper and argument type.

[0120] In specific application examples, argument prediction and recognition can be performed according to the following formula:

[0121] P S (w i ) = softmax(f i W s (14)

[0122] P E (w i ) = softmax(f i W e (15)

[0123] Among them, P S (w i ) and P E (w i ) represents the i-th word w i As the probability of the start and end of an argument, and These are trainable parameters.

[0124] Due to the complexity of argument span candidates, this embodiment further sets several rules to determine their rationality. Assuming the length of the original text sequence is n, the span of argument a includes the starting offset S. a and end offset E a Initial offset S a and end offset E a The following rules must be met:

[0125] (1) 0 ≤ S a ≤n and 0≤E a ≤n, which means that the candidate word must belong to the original text sequence;

[0126] (2)E a -S a ≥0 indicates that the candidate words can form a correct phrase;

[0127] (3)P S (S a )≥threshold≥max(P S ([CLS]), P S ([SEP])),

[0128] P E (E a )≥threshold≥max(P E ([CLS]), P E ([SEP])) indicates that the span candidate needs high reliability.

[0129] Since [CLS] and [SEP] are non-argument parts, the candidates are more reliable than them.

[0130] (4)E a and S a It belongs to argument a and cannot belong to any other argument.

[0131] To train the argumentation model, this embodiment employs a loss function similar to that used in event detection. Since the argumentation information consists of two words: a start and an end, the loss function is calculated using both, i.e., the loss equals S. a and E a The sum of:

[0132]

[0133] like Figure 3 As shown, the machine reading comprehension event extraction device based on template bridging in this embodiment includes:

[0134] The trigger word question generation module is used to obtain the input text and the trigger word question template and combine them to form an input instance with the trigger word question prompt template;

[0135] The event detection module performs syntactic analysis on input instances with trigger word hints. It obtains the syntactic dependencies between words in the input text, the information aggregation nodes in the input file, and the semantic aggregation relationships of the trigger word hint template. Based on the information aggregation nodes, it connects the input text and the trigger word hint template to obtain a link graph. It then extracts global features from the input instance based on the link graph. Finally, it fuses the global features with the semantic features of the input instance to identify the trigger word and its type.

[0136] The argument question generation module is used to generate argument question prompt templates based on the identified trigger words and trigger word types, and combine the argument question prompt templates with the input text to form an input instance with argument question prompt templates;

[0137] The argument extraction module is used to extract arguments from input instances with argument question prompt templates, and output the arguments and argument types.

[0138] like Figure 3 As shown, this embodiment includes a trigger word question generation module, an event detection module, an argument question generation module, and an argument extraction module. The trigger word question generation module and the argument question generation module are used to generate prompt question templates for trigger words and arguments, respectively, and combine the prompt question templates with text to form an input instance with prompt information. The event detection module is used to apply the input text to identify trigger words and their types. The argument extraction model uses the input text to obtain entities related to specific events.

[0139] like Figure 4 As shown, the event detection module in this embodiment mainly comprises three parts: event detection representation, feature fusion, and predictor. The event detection representation is used to obtain the global features of the input instance. Feature fusion utilizes an attention mechanism to fuse global features and semantic features. Finally, a linear classifier is used to predict the ED result. The argument extraction module mainly comprises three parts: EAE representation, feature fusion, and predictor. The EAE representation is used to obtain different features of the event. Feature fusion is used to connect different features to integrate information related to the arguments. Then, a linear classifier is used as a predictor to extract the arguments.

[0140] The template-bridging-based machine reading comprehension event extraction device in this embodiment corresponds one-to-one with the template-bridging-based machine reading comprehension event extraction method described above, and will not be repeated here.

[0141] To verify the effectiveness of this invention, this embodiment employs the method of this invention and compares the results with traditional models (including sequence-based models, GNN-based models, and machine reading comprehension models) for event extraction. The baseline model used in this embodiment is as follows:

[0142] S-CNNs: An event extraction framework based on skip-window convolutional neural networks to capture global structural features. dbRNN: This model utilizes grammatically relevant information about words to construct dependency bridges to enhance the RNN.

[0143] Joint3EE: This model uses a bidirectional RNN to induce shared hidden representations for three subtasks: EMD, ED, and ARP. Due to the sharing of knowledge between different tasks, the performance of EE is improved.

[0144] CMM-share: This is a multi-task learning framework that uses a hard parameter sharing method at the character level between the language model and the event detection model.

[0145] JMEE: This framework improves the performance of ED by using GCN with enhanced information flow and self-attention mechanisms.

[0146] MHGEE: This model constructs a multi-granularity heterogeneous graph for R-GCN with gating mechanism to capture the representation of different nodes.

[0147] Zhang: An end-to-end event extraction network based on probabilistic syntax forest, which is represented by a two-stage GCN model.

[0148] BERT_QA: To avoid the problem of error propagation, this framework treats EE as a QA task to extract event arguments in an end-to-end manner.

[0149] RCEE: A topic-related and context-dependent unsupervised problem generation method that leverages cross-domain data to alleviate the problem of data scarcity.

[0150] MQAEE: This model uses a multi-round QA formalization to jointly model entities and relationships. The model needs to run multiple rounds of QA following the order of the relationship chain.

[0151] ONEIE: This model is an end-to-end EE framework with global constraints that evaluates interactions across subtasks and across instances.

[0152] TEXT2EVENT(T5-large): A sequence-to-structure generation paradigm that can directly extract events from text end-to-end.

[0153] Multi-Task TANL: A unified framework for solving different information extraction problems. It is framed as an augmented natural language translation task from which task-related information can be easily extracted.

[0154] DEGREE is a generative model focused on low-resource, end-to-end event extraction. DEGREE designs templates with tag semantics and generates descriptions of various event types to improve data utilization efficiency.

[0155] GTEE-DYNPREF: Compared to other cue learning models, this model applies a dynamic prefixing method by combining contextual information and type-specific prefixes to generate templates.

[0156] First, the above methods were used to perform event detection tasks on the ACE dataset, and the results are shown in Table 1.

[0157] Table 1: Event Detection Results of Each Model

[0158]

[0159]

[0160] Analyzing Table 1, the methods described above are categorized into three types: sequence-based models, GNN-based models, and MRC-based models. Table 1 shows that GNN-based models have significant advantages over other models in event classification and trigger word recognition. GNN-based models achieve improvements of approximately 4.8% and 6.2% compared to sequence-based models, and they significantly outperform MRC-based models in event classification and trigger word recognition (2.3% improvement in event classification; 4.0% improvement in trigger word recognition). Graph neural network-based models obtain long-distance dependencies between words through syntactic dependency trees and other forms. These dependencies allow trigger words to aggregate important but distant words, thereby enhancing the effective representation of trigger words. MRC-based models improve event detection performance, but their performance is heavily dependent on the designed question template, which may introduce noise and degrade performance. The MRCBEE model of this invention achieves the best results on most metrics, and compared to other models, it improves the F1 score for event classification and trigger word recognition by 5.1% and 7.5%, respectively.

[0161] The above methods were then used to perform argument extraction tasks on the ACE dataset, and the experimental results are shown in Table 2.

[0162] Table 2: Experimental results of argument extraction tasks for each model

[0163]

[0164]

[0165] In Table 2, "pre" indicates that the performance of argument extraction is directly affected by trigger word recognition, and "gold" indicates that argument extraction is based on gold trigger words to exclude false propagation in event detection. Table 2 shows that the MRCBEE of this invention has better performance than traditional models. Although DEGREE performs slightly better than the model of this invention in Arg-C and Arg-ID (gold), the performance of the MRCBEE of this invention is more stable. Furthermore, the template of the MRCBEE of this invention is more flexible and requires less additional information.

[0166] Table 2 also shows that event detection results have a significant impact on argument extraction. DEGREE experienced the largest performance drop, with Arg-C(pre) decreasing by 17.7%. DEGREE's prompts include event type descriptions, event keywords, and end-to-end templates, all of which heavily rely on event detection results. Other MRC-based models have relatively simple prompt templates, typically containing only one prompt statement. Sequence-based and GNN-based models utilize event detection information less, relying primarily on named entity recognition. Table 2 also shows the F1 scores of various models in argument extraction. The results indicate that the MRC-based model achieved relatively high performance, improving Arg-C F1(pre) by 4.0% and 2.6% compared to the other two categories. Other models treat role event extraction as a classification problem, ignoring the relationship between event type and arguments. The MRC-based model transforms argument extraction into an end-to-end approach, avoiding error propagation problems caused by intermediate steps such as named entity recognition. Most MRC-based models generate cue templates from trigger words or event types, thus these templates contain weakly supervised information, effectively revealing the connection between event detection and argument extraction.

[0167] This embodiment further employs the above model for ablation experiments. First, in the ablation experiment on the ACE2005 dataset, the event detection module of this invention has two main components: (1) a cross-domain mechanism to integrate structural and semantic information; and (2) a Q&A bridging module to explicitly mine the relationship between the question template and the text, and capture structural information. This embodiment analyzes the effectiveness of the above modules on the model of this invention through ablation experiments. This embodiment uses the following four ablation models: (1) full indicates that the model has complete modules; (2) –attention excludes the cross-domain mechanism applied to fuse different information; (3) –bridging eliminates the Q&A bridging module to remove structural information; and (4) no is the baseline model used to evaluate the performance of the original QA model.

[0168] Ablation study evaluation results such as Figure 6 As shown, by Figure 6As can be seen, the Q&A bridging module significantly improved the F1 score of the no model on the event detection task, increasing it by 3.45% and 5.9%. This is due to the more reasonable organizational structure between the text and the template, which allows the graph neural network to fully explore the inherent relationship between trigger words and text. The cross-domain mechanism also helps improve model performance, increasing the F1 score on the two sub-tasks by 2.01% and 4.98%, respectively. Compared with other fusion methods, the cross-domain fusion method used in this invention has two advantages: first, it focuses on keywords during feature fusion through an attention mechanism; second, it uses a gating mechanism to filter out invalid information.

[0169] This embodiment further evaluates the effectiveness of component extraction in argument extraction through ablation experiments, such as... Figure 7 As shown, the orange and blue lines represent the Arg-C F1 score and the Arg-ID F1 score, respectively, and the trend of the lines indicates that trigger words play an important role in argument extraction.

[0170] As shown above, compared with other components, the MRCBEE of this invention gains greater benefits from the head word feature in the EAE. Due to the removal of this feature (-HeadWord), the F1 scores of the MRCBEE of this invention decreased by 4.8% and 5.6% in argument classification and recognition, respectively. This is mainly because the trigger word is the core of the event, and most arguments appear in the subject or object position of the trigger word. Similarly, it can be observed that the POE indicates the positional information of the trigger word, thus having a significant impact on the model's performance.

[0171] Event extraction in low-resource scenarios has become a major challenge in natural language processing. This embodiment further conducts experiments in low-resource scenarios to further verify the performance of the MRCBEE model of this invention. Specifically, another set of experiments was conducted on the ERE-EN and ACE2005 datasets. The development and test sets from the ERE-EN and ACE2005 datasets remained unchanged, while different proportions (3%, 5%, 10%, 20%, and 30%) of training data were retained to simulate low-resource scenarios.

[0172] Table 3 shows the F1 scores for event classification and argument classification on the two datasets with limited training data. The results indicate that the MRCBEE of this invention outperforms all baseline models in event classification, and its advantage becomes increasingly significant with increasing training data. Furthermore, using only 3% of the training data, the F1 scores of the MRCBEE of this invention are 5.2% and 2.9% higher than the best baseline models on the two datasets, respectively.

[0173] The present invention, MRCBEE, also achieved state-of-the-art performance in argument role classification. However, when using less than 10% of the training data, DEGREE outperforms the present invention, mainly because DEGREE generates prompt templates containing label semantics. The sentence structure in the template describes the semantic relationship between trigger words and arguments, and the template includes a description of the event and a description of similar words for the arguments, which further verifies the accuracy of the prompts. The advantage of the present invention lies in its greater flexibility and the fact that it does not require complex sentence construction or more event information.

[0174] Table 3. Experimental results of small sample event extraction on training sets of different sizes

[0175]

[0176]

[0177] In summary, the present invention outperforms traditional models in all EE subtasks, and it can not only handle scenarios with few samples, but also effectively handle complex texts with multiple events.

[0178] It is understood that the method described in this embodiment can be executed by a single device, such as a computer or server, or it can be applied to a distributed scenario where multiple devices cooperate to complete the task. In a distributed scenario, one of the multiple devices may execute only one or more steps of the method described in this embodiment, and the multiple devices interact to complete the method. The processor can be implemented using a general-purpose CPU, microprocessor, application-specific integrated circuit, or one or more integrated circuits, and is used to execute relevant programs to implement the method described in this embodiment. The memory can be implemented using read-only memory (ROM), random access memory (RAM), static storage devices, and dynamic storage devices. The memory can store the operating system and other applications. When the method described in this embodiment is implemented through software or firmware, the relevant program code is stored in the memory and called and executed by the processor.

[0179] This embodiment further provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described above.

[0180] Those skilled in the art will understand that the above embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-readable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code. This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create an implementation for the process. Figure 1 One or more processes and / or boxes Figure 1 The computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to operate in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The functions specified in one or more boxes. These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable apparatus for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0181] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the invention. Therefore, any simple modifications, equivalent changes, and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention should fall within the protection scope of the present invention.

Claims

1. A method for extracting machine reading comprehension events based on template bridging, characterized in that the steps include... include: Step S01. Trigger word question generation: Obtain the input text and the trigger word question prompt template and combine them to form an input instance with the trigger word question prompt template; Step S02. Event Detection: Perform syntactic analysis on the input instance with trigger word prompts to form an adjacency graph describing the relationship between words in the input instance. Based on the adjacency graph, connect the input text with the trigger word prompt template to obtain a link graph, so as to bridge the question with the context and reconstruct the relationship between the question and the text. Then, extract global features from the input instance based on the link graph. After fusing the global features and the semantic features of the input instance, the trigger words and trigger word types are identified; Step S03. Argument Question Generation: Generate an argument question prompt template based on the identified trigger words and trigger word types, and combine the argument question prompt template with the input text to form an input instance with the argument question prompt template; Step S04. Argument Extraction: Extract arguments from the input instance with the argument question prompt template to obtain the arguments and argument types.

2. The template-bridging machine reading comprehension event extraction method according to claim 1, characterized in that, In step S02, when concatenating the input text and the trigger word question prompt template, the relationship between the word embeddings of each word in the input text and the word embeddings of the trigger word question is converted into an adjacency matrix A. n×n Where n is the number of words in the input instance QA, and the adjacency matrix A n×n The matrix elements in are Among them, v i , vj These are the i-th word and the j-th word in the input instance QA, respectively. E includes poss, nsubj, dobj, prompt, and other relations. poss indicates that the possessive modifier is a possessive determiner, nsubj indicates a noun subject, dobj indicates a direct object, and prompt indicates a prompt question template for the event extraction task.

3. The machine reading comprehension event extraction method using template bridging according to claim 1, characterized in that, In step S02, GCN is used to extract global features from the input instance, and the following formula is used when updating node information: in, This represents the representation of node v at layer (l+1). Let V represent the optimized representation of node v in layer l+1, where l represents the layer number of the graph neural network, v represents a node (i.e., a word), and V represents the set of nodes. I is the identity matrix. D is the degree matrix of A, where A is the adjacency matrix obtained by transforming the relationship between the word embeddings of the input instance and the word prompt question. ii =∑ j A ij d represents the length of the question template, W (l) This represents the weight matrix of the l-th layer. This indicates that the (l+1)th layer contains overall information about the prompt question template, ReLU represents the activation function, [CLS] represents the information aggregation node, and g() represents the feature transformation function.

4. The machine reading comprehension event extraction method using template bridging according to claim 3, characterized in that, When extracting global features from input instances using GCN, the method also includes using a gating mechanism to filter out interfering terms according to the following formula: Among them, g(h v ) represents the gate function. Let v represent the normalized representation of node v, n represent the length of the input instance, and α and ∈ be the trainable parameters. h represents the square root of the representation of all neighboring nodes of v. u Let v represent the neighboring nodes of v, u represent the neighboring nodes of v, and N(v) represent the set of all neighboring nodes of v.

5. The machine reading comprehension event extraction method using template bridging according to claim 1, characterized in that, In step S02, the global features and the semantic features of the input instance are fused using an attention mechanism, and the calculated expression is: CrossAttention(S,G)=γAttention S +λAttention G in, Represents semantic features after linear transformation. Represents global features, where M is the number of words in the sentence, and d k The dimension of the word embedding is represented by CrossAttention(S,G), which represents the feature Q. S and K G The fusion features, where γ and λ represent weighting coefficients; Trigger words are predicted and identified during the trigger word recognition process using the following formula: output trigger =max(a p1 ,a p2 ,…,a pn ) Among them, a pi w represents the i-th word i It represents the probability of the trigger word, and n represents the length of the input instance. These are trainable parameters, and H is a. i The dimension, N is the number of trigger word types, output trigger This indicates the trigger words identified in the prediction.

6. The template-bridging machine reading comprehension event extraction method according to any one of claims 1 to 5, characterized in that, Step S04 includes: Step S401.EAE means: Obtain the semantic features E of the input instance. A And the POE features of each word in the input instance, wherein the POE features are used to indicate whether the word is a trigger word; Step S402. Feature Fusion: Combine the semantic features E of the input instance. A The keyword feature E is obtained by fusing the POE features of each word. C Using the semantic feature E respectively A POE features and keyword features E C Obtain the representation of the extracted event arguments; Step S403. Argument prediction and identification: Based on the representation extracted from the event arguments, predict and identify the paper and argument type.

7. The template-bridging machine reading comprehension event extraction method according to claim 6, characterized in that, In step S402, the keyword feature E is obtained according to the following formula. C : E C =α⊙Relu(multiply(E A ,T A ))+β⊙E A Where α and β represent learnable parameters, and multiply represents the multiplication operation; The extracted event arguments are represented as follows: in, This indicates the concatenation of different data types, where nor represents the normalization operation, and n represents the length of the input instance.

8. The machine reading comprehension event extraction method using template bridging according to claim 6, characterized in that, In step S403, argument prediction and identification are performed according to the following formula: P S (w i )=softmax(f i W s ) P E (w i )=sofmtax(f i W e ) Among them, P S (w i ) and P E (w i ) represents the i-th word w i As the probability of the start and end of an argument, and These are trainable parameters; The span of argument a includes the initial offset S a and end offset E a The initial offset S a and end offset E a The following rules must be met: 0≤S a ≤n and 0≤E a ≤n; E a -S a ≥0; P S (S a )≥threshold≥max(P S ([CLS]),P S ([SEP])) P E (E a )≥threshold≥max(P E ([CLS]),P E ([SEP])); and E a and S a It belongs to argument a and cannot belong to any other argument.

9. A machine reading comprehension event extraction device based on template bridging, characterized in that, include: The trigger word question generation module is used to obtain the input text and the trigger word question prompt template and combine them to form an input instance with the trigger word question prompt template; The event detection module performs syntactic analysis on input instances with trigger word prompts to form an adjacency graph describing the relationships between words in the input instance. Based on the adjacency graph, the input text is connected with the trigger word prompt template to obtain a link graph, which bridges the question with the context and reconstructs the relationship between the question and the text. Then, global features are extracted from the input instance based on the link graph. After fusing the global features and the semantic features of the input instance, the trigger words and trigger word types are identified; The argument question generation module is used to generate argument question prompt templates based on the identified trigger words and trigger word types, and combine the argument question prompt templates with the input text to form an input instance with argument question prompt templates; The argument extraction module is used to extract arguments from the input instance with the argument question prompt template, and output the arguments and argument types.

10. A computer-readable storage medium storing a computer program, characterized in that, When the computer program is executed by a processor, it implements the method as described in any one of claims 1 to 8.

Images