Method for extracting energy power document level event argument based on dependency perception and role guidance

Abstract

The invention provides an energy power document level event argument extraction method based on dependency perception and role guidance. The method comprises the following steps: S1, acquiring data and preprocessing the data; S2, constructing a prompt template; S3, constructing a dependency perception self-attention module; according to the method, the prompt template is constructed for the event, dependency information between roles is increased, the integrity of event extraction is improved, independent modeling is carried out on the argument role slot, meanwhile, the argument boundary is limited by adopting boundary loss, the deviation of a prediction result is effectively reduced, and the accuracy of argument extraction is improved.

Description

Technical Field

[0001] The present invention belongs to the technical field of energy and power automation, and in particular relates to a method for extracting energy and power document-level event arguments based on dependency perception and role guidance. Background Art

[0002] With the rapid development of the energy and power industry, the demand for automated and intelligent power systems is increasing. This has generated a large amount of document data in power systems, such as equipment operation reports, fault records, and dispatch instructions. These documents contain rich event information, such as equipment failures, power dispatch, and load changes. Automatically extracting event arguments (such as the subject, time, location, and cause of the event) from these documents is of great significance for automated management, fault diagnosis, and decision support in power systems.

[0003] Currently, event argument extraction technology primarily relies on information extraction methods in natural language processing (NLP). Traditional event argument extraction methods are typically based on rules or statistical models. These methods have achieved certain success in extracting events at the short text or sentence level. However, in the energy and power sector, event information is often distributed across multiple sentences or even multiple documents. Traditional sentence-level event extraction methods struggle to capture long-range document-level dependencies, resulting in insufficient accuracy and completeness in the extraction results. Furthermore, existing methods often overlook the dependencies and role information between event arguments. In energy and power documents, event arguments often have complex dependencies. For example, a device failure may trigger multiple events, and the arguments of these events often have specific role associations. Existing methods lack effective modeling of these dependencies and role information, resulting in poor extraction accuracy and completeness. Summary of the Invention

[0004] The purpose of the present invention is to provide a method for extracting event arguments at the energy and power document level based on dependency perception and role guidance, so as to solve the problems of insufficient research on the dependency relationship of event argument extraction in long documents in the energy and power field, and poor extraction accuracy and completeness.

[0005] The technical solution adopted by the present invention is a method for extracting energy and power document-level event arguments based on dependency perception and role guidance, which includes the following steps:

[0006] Step S1, acquiring data and preprocessing the data;

[0007] Step S2, constructing a prompt template;

[0008] Step S3, constructing a dependency-aware self-attention module;

[0009] Step S4, constructing an argument role guided span selector;

[0010] Step S5: Calculate the boundary loss.

[0011] Furthermore, in S1, for document X, given an event trigger word x trig , in the trigger word x trig Insert front and back <t> 、< / t> , updated documentation It is expressed as follows:

[0012]

[0013] in, Represents the updated document, x trig Indicates event trigger words, x n Represents the nth word in the document, <t>and< / t> They are special symbols used to mark the trigger words before and after.

[0014] Furthermore, it is characterized in that, in said S2, for each event type e in the data set D i , construct a role containing all its arguments Tip template Among them, r is the argument role, are all argument roles included in event i.

[0015] Furthermore, the method for constructing the prompt template in S2 includes:

[0016] (1) Manual prompting: manually connect roles through natural language to form a complete semantic template. First, all argument roles are listed, and then connected through words with semantic relationships to make it semantically complete;

[0017] (2) Splicing prompts, directly concatenating the role names to form a structured template without adding other words, and only forming a template by listing all argument roles;

[0018] (3) Soft prompts, using learnable role-specific pseudo-tokens to dynamically generate templates, calculate the number of argument roles contained in the event, and supplement the same number of pseudo-tokens <v> Victor< / v> , and finally generate the event template.

[0019] Furthermore, in S3, the specific steps of constructing the dependency-aware self-attention module are as follows:

[0020] S31, the document updated by S1 Input the pre-trained encoder Encoder to generate contextual semantic representation H X , as follows:

[0021]

[0022] At the same time, the event prompt template Also input the pre-trained encoder Encoder to generate the initial representation of the prompt template The details are as follows:

[0023]

[0024] S32, strengthen the dependency between roles and design the role dependency perception mask matrix M in the self-attention layer of the decoder role , and then perform character perception self-attention calculation. The specific formula is as follows:

[0025]

[0026] in, is the role perception prompt representation, P is the event prompt template, Softmax(·) represents the activation function, T is the matrix transpose, Indicates event e i The prompt template, SelfAttention(·) represents the self-attention mechanism, Q, K, V represent the query, key, and value matrices respectively, M role represents the role-dependent perception mask matrix, and d represents the dimension scaling factor;

[0027] S33, in the cross attention layer of the decoder, the context semantic representation H X and role-aware cue representation The interaction process is as follows:

[0028]

[0029] Among them, CrossAttention is the cross attention mechanism, H p To indicate that For role perception hint representation, is the initial representation of the prompt template.

[0030] Furthermore, in S4, the method for constructing the argument role-guided span selector is as follows:

[0031] Based on S33, the representation corresponding to the kth slot in the prompt template is changed from H p Average pooling is performed to obtain the character feature θ k , the specific formula is as follows:

[0032]

[0033] Among them, MeanPooling(·) represents average pooling, Indicates the starting position index of the kth slot in the prompt template, Indicates the end position index of the kth slot in the prompt template, H p To indicate a prompt;

[0034] A role may have multiple slots, and accordingly, multiple role features and role fillers. For a given role feature θ k have:

[0035]

[0036] The resulting argument role guides the span selector Expressed as:

[0037]

[0038] Among them, w (start) and w (end) is a learnable parameter matrix shared by all argument roles, w (start) represents the starting matrix, w (end) Indicates the end matrix, ° indicates element-by-element multiplication, Indicates the feature representation of the start word of the slot, Indicates the end word feature representation of the slot.

[0039] Furthermore, in S5, the method for calculating the boundary loss is as follows:

[0040] Based on the contextual semantic representation H X and argument roles guide span selectors First calculate the position distribution of the start word and end word of each slot, each All from H X Extract an argument span and express it as (s k ,e k ), s k is the start index of the span, e k is the end index of the span. When there is no span of this argument in the context, the span is represented as (0,0). The position distribution calculation formula of the start word and end word of each slot is as follows:

[0041]

[0042] Then, the probability distribution of the slot start and end word positions is calculated. The specific formula is as follows:

[0043]

[0044] Finally, the loss function is:

[0045]

[0046] in, is the feature representation of the start word of the slot, is the feature representation of the end word of the slot, H X is the contextual semantic representation, is the probability of the starting word position of the slot, is the probability of the end word of the slot, Softmax(·) represents the activation function, and the logit function is the inverse function of the sigmoid function. is the logarithmic probability of the slot starting position, is the logarithmic probability of the slot end position probability, BoundaryLoss represents the loss function, s k Indicates the starting index of the span, e k Indicates the ending index of the span.

[0047] The beneficial effects of the present invention are:

[0048] 1. The present invention helps to increase the dependency information between roles and improve the integrity of event extraction by constructing a prompt template for the event.

[0049] 2. The present invention constructs a dependency-aware self-attention module and selects highly relevant roles for modeling, which helps to accurately focus on key role relationships and enhance the ability to capture important semantic information.

[0050] 3. The present invention uses the argument role-guided span selector module to separately model the argument role slot, predict the start and end positions of the argument, and uses boundary loss to restrict the argument boundary, thereby effectively reducing the deviation of the prediction result and improving the accuracy of argument extraction.

[0051] 4. The present invention calculates boundary loss, quickly converges the model, accurately captures argument boundaries, and improves the accuracy of argument extraction. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0053] Figure 1 It is a flow chart of the present invention.

[0054] Figure 2 It is a flow chart of event argument extraction in the present invention.

[0055] Figure 3 is the role-dependent perception mask matrix M in the present inventionrole Representation diagram. DETAILED DESCRIPTION

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention.

[0057] Example

[0058] The embodiment of the present invention provides a method for extracting energy and power document-level event arguments based on dependency perception and role guidance, as shown in the flowchart. Figure 1 As shown, the steps include:

[0059] Step S1: Get data from dataset D and pre-process the data. In the document-level event argument extraction task, insert special symbols before and after the trigger words given in the annotated corpus. <t> 、< / t> , to explicitly mark the trigger words and enhance the model's ability to perceive their positions. The process of extracting event arguments is as follows Figure 2 For example, for the trigger word "attack", the marked form is " <t> attack< / t> This method not only helps the model accurately locate trigger words, but also enhances the interaction between trigger words and contextual information, thereby more accurately capturing event arguments. By embedding the labeled trigger words into the original text, the model can better understand the event structure and improve the accuracy and robustness of event extraction. The specific method is as follows:

[0060] For document X, given an event trigger word x trig , in the trigger word x trig Insert front and back <t> 、< / t> , updated documentation It is expressed as follows:

[0061]

[0062] in, Represents the updated document, x trig Indicates event trigger words, x n Represents the nth word in the document, <t> and< / t> They are special symbols used to mark the trigger words before and after.

[0063] Step S2: Construct a prompt template for each event type e in the dataset D. i , construct a role containing all its arguments Tip template Among them, r is the argument role, For all argument roles included in event i, specifically, taking the event type "transport" as an example, its argument role set is {Importer, artifact, origin, destination, vehicle}, and the corresponding prompt template is defined as " Importer transport artifact from origin to destination by vehicle .”, where each argument role is defined as an independent “slot” (such as the underlined part in the example), and the implicit interaction relationship between roles is captured through the pre-trained language model.

[0064] To adapt to different scenario requirements, the present invention proposes three prompt construction methods:

[0065] (1) Artificial prompts: manually connect roles through natural language to form a complete semantic template. First, list all argument roles, and then connect them through words with semantic relationships to make them semantically complete and the sentence smooth (such as " Importer transport artifact from origin to destination by vehicle .”);

[0066] (2) Splicing prompts: directly concatenate the role names to form a structured template without adding other words. The template is formed by listing all the argument roles (such as "Transport: importerartifactorigindestinationvehicle ”);

[0067] (3) Soft prompts, using learnable role-specific pseudo-tokens to dynamically generate templates, calculate the number of argument roles contained in the event, and supplement the same number of pseudo-tokens <v> Victor< / v> , and finally generate an event template (such as "Transport: <v> Victor< / v> <v> Victor< / v> <v> Victor< / v> <v> Victor< / v> <v> Victor< / v> ”).

[0068] The three types of templates strengthen the model's ability to model event-role relationships from the three dimensions of semantic coherence, structural simplicity, and dynamic adaptability, providing a structured input basis for subsequent dependency perception and role guidance mechanisms.

[0069] In the present invention, step S1 and step S2 are performed simultaneously. The pre-processing of S1 is to process a specific document, and the prompt template of S2 is for the event type.

[0070] Step S3: construct a dependency-aware self-attention module. The specific steps are as follows:

[0071] S31, the document updated by S1 Input the pre-trained encoder Encoder to generate contextual semantic representation H X , as follows:

[0072]

[0073] At the same time, the event prompt template Also input the pre-trained encoder Encoder to generate the initial representation of the prompt template The details are as follows:

[0074]

[0075] S32, strengthen the dependency between roles and design the role dependency perception mask matrix M in the self-attention layer of the decoder role ,like Figure 3 As shown in Figure 2, the matrix is initialized based on the role co-occurrence frequency and is used to constrain the self-attention mechanism to establish strong connections only between highly correlated roles. It then performs role-aware self-attention calculations. The specific formula is as follows:

[0076]

[0077] in, is the role perception prompt representation, P is the event prompt template, Softmax(·) represents the activation function, T is the matrix transpose, Indicates event e i The prompt template, SelfAttention(·) represents the self-attention mechanism, Q, K, V represent the query, key, and value matrices respectively, M role represents the role-dependent perception mask matrix, and d represents the dimension scaling factor.

[0078] S33, in the cross attention layer of the decoder, the context semantic representation H X and role-aware cue representation The interaction is carried out to achieve information fusion, so that the model can simultaneously capture the global semantics of the document and the fine-grained dependencies between roles, providing multi-level feature support for subsequent argument span extraction. The process is as follows:

[0079]

[0080] Among them, CrossAttention is the cross attention mechanism, H p To indicate that For role perception hint representation, is the initial representation of the prompt template.

[0081] Step S4, constructing an argument role-guided span selector. After successfully capturing the dependency between highly correlated roles, in order to further improve the accuracy and efficiency of the event argument extraction task and deeply explore the unique semantic value of each argument role, the present invention carries out independent modeling work for each argument role slot and designs an argument role-guided span selector, which fully considers the specific requirements of different argument roles and can accurately extract event arguments.

[0082] Based on S33, the representation corresponding to the kth slot in the prompt template is changed from H p Average pooling is performed to obtain the character feature θ k , the specific formula is as follows:

[0083]

[0084] Among them, MeanPooling(·) represents average pooling, Indicates the starting position index of the kth slot in the prompt template, Indicates the end position index of the kth slot in the prompt template, H p Indicates a prompt.

[0085] A role may have multiple slots, and accordingly, multiple role features and role fillers. For a given role feature θ k have:

[0086]

[0087] The resulting argument role guides the span selector Expressed as:

[0088]

[0089] Among them, w (start) and w (end) is a learnable parameter matrix shared by all argument roles, w (start) represents the starting matrix, w (end) Indicates the end matrix, ° indicates element-by-element multiplication, Indicates the feature representation of the start word of the slot, Indicates the end word feature representation of the slot.

[0090] Step S5, calculate the boundary loss based on the context semantic representation H X and argument roles guide span selectors First calculate the position distribution of the start word and end word of each slot. All from H X Extract an argument span and express it as (sk ,e k ), s k is the start index of the span, e k is the end index of the span. When there is no span of this argument in the context, the span is represented as (0,0). The position distribution calculation formula of the start word and end word of each slot is as follows:

[0091]

[0092] Then, the probability distribution of the slot start and end word positions is calculated. The specific formula is as follows:

[0093]

[0094] Finally, the loss function is:

[0095]

[0096] in, is the feature representation of the start word of the slot, is the feature representation of the end word of the slot, H X is the contextual semantic representation, is the probability of the starting word position of the slot, is the probability of the end word of the slot, Softmax(·) represents the activation function, and the logit function is the inverse function of the sigmoid function. is the logarithmic probability of the slot starting position, is the logarithmic probability of the slot end position probability, BoundaryLoss represents the loss function, s k Indicates the starting index of the span, e k Indicates the ending index of the span.

[0097] The present invention constructs a prompt template that fits the characteristics of the task, and then strengthens the interaction and dependency between event roles through the self-attention module. With the guidance of the prompt template, the model can accurately focus on key role information, thereby improving the accuracy of event argument extraction. In addition, the present invention also designs an argument role guided span selector module. This module models each event role separately, fully considering the uniqueness of the role, and accurately matches and allocates it based on contextual information. It can efficiently identify the complex relationship between event roles and arguments, improve the accuracy of document-level event argument extraction, and provide an effective technical path for solving the problem of argument extraction in complex texts.

[0098] Experimental verification

[0099] The present invention is compared with the prior art model, and the results are shown in Table 1:

[0100] Table 1 Comparison results between the present invention and the prior art

[0101]

[0102] In Table 1, the Arg-I evaluation index represents the recognition accuracy of arguments in event argument recognition, and the Arg-C evaluation index represents the classification accuracy of arguments in event argument recognition. Bold indicates the best result, and underlined indicates the suboptimal result. The two evaluation indicators of the present invention on the three datasets of RAMS (Richly Annotated Multilingual Schema-guided Event Structure), WikiEvents, and MLEE (Multi-Level Event Extraction) are higher than those of other methods, showing that the method of the present invention has a high improvement in both recognition and classification. It proves that the present invention can deeply explore the semantic connotation of roles in different contexts through a unique role dependency perception module and argument role guided span selector, and accurately capture the subtle differences and special semantics of roles. The present invention integrates role dependency information into the model training process by constructing task-specific prompt templates, and strengthens the interactive relationship between roles with the help of the self-attention mechanism, so that the model can clearly grasp the complex implicit dependencies between roles, thereby performing well in the recognition and classification of complex arguments and improving the accuracy of document-level event argument extraction.

[0103] Each embodiment in this specification is described in a related manner. Similar parts between the various embodiments can be referred to in conjunction with each other. Each embodiment focuses on the differences between the other embodiments. In particular, the system embodiment is generally similar to the method embodiment, so the description is relatively simple. For related parts, refer to the description of the method embodiment.

[0104] The above description is only a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention are included in the scope of protection of the present invention.

Claims

1. A method for extracting event arguments from energy and power documents based on dependency perception and role guidance, characterized by the following steps: include: Step S1, acquiring data and preprocessing the data; Step S2, constructing a prompt template; Step S3, constructing a dependency-aware self-attention module; Step S4, constructing an argument role guided span selector; Step S5: Calculate the boundary loss.

2. The method for extracting energy and power document-level event arguments based on dependency perception and role guidance according to claim 1 is characterized in that: In S1, for document X, given an event trigger word x trig , in the trigger word x trig Insert front and back <t> 、< / t> , updated documentation It is expressed as follows: in, Represents the updated document, x trig Indicates event trigger words, x n Represents the nth word in the document, <t> and< / t> They are special symbols used to mark the trigger words before and after.

3. The method for extracting energy and power document-level event arguments based on dependency perception and role guidance according to claim 1 is characterized in that: In S2, for each event type e in the dataset D, i , construct a role containing all its arguments Tip template Among them, r is the argument role, are all argument roles included in event i.

4. The method for extracting energy and power document-level event arguments based on dependency perception and role guidance according to claim 1 is characterized in that: The method for constructing the prompt template in S2 includes: (1) Manual prompting: manually connect roles through natural language to form a complete semantic template. First, all argument roles are listed, and then connected through words with semantic relationships to make it semantically complete; (2) Splicing prompts, directly concatenating the role names to form a structured template without adding other words, and only forming a template by listing all argument roles; (3) Soft prompts, using learnable role-specific pseudo-tokens to dynamically generate templates, calculate the number of argument roles contained in the event, and supplement the same number of pseudo-tokens <v> Victor< / v> , and finally generate the event template.

5. The method for extracting energy and power document-level event arguments based on dependency perception and role guidance according to claim 1 is characterized in that: In S3, the specific steps of constructing the dependency-aware self-attention module are as follows: S31, the document updated by S1 Input the pre-trained encoder Encoder to generate contextual semantic representation H X , as follows: At the same time, the event prompt template Also input the pre-trained encoder Encoder to generate the initial representation of the prompt template The details are as follows: S32, strengthen the dependency between roles and design the role dependency perception mask matrix M in the self-attention layer of the decoder role , and then perform character perception self-attention calculation. The specific formula is as follows: in, is the role perception prompt representation, P is the event prompt template, Softmax(·) represents the activation function, T is the matrix transpose, Indicates event e i The prompt template, SelfAttention(·) represents the self-attention mechanism, Q, K, V represent the query, key, and value matrices respectively, M role represents the role-dependent perception mask matrix, and d represents the dimension scaling factor; S33, in the cross attention layer of the decoder, the context semantic representation H X and role-aware cue representation The interaction process is as follows: Among them, CrossAttention is the cross attention mechanism, H p To indicate that For role perception hint representation, is the initial representation of the prompt template.

6. The method for extracting energy and power document-level event arguments based on dependency perception and role guidance according to claim 1 is characterized in that: In S4, the method for constructing the argument role-guided span selector is as follows: Based on S33, the representation corresponding to the kth slot in the prompt template is changed from H p Average pooling is performed to obtain the character feature θ k , the specific formula is as follows: Among them, MeanPooling(·) represents average pooling, Indicates the starting position index of the kth slot in the prompt template, Indicates the end position index of the kth slot in the prompt template, H p To indicate a prompt; A role may have multiple slots, and accordingly, multiple role features and role fillers. For a given role feature θ k have: The resulting argument role guides the span selector Expressed as: Among them, w (slart) and w (end) is a learnable parameter matrix shared by all argument roles, w (start) represents the starting matrix, w (end) represents the end matrix, represents element-wise multiplication, Indicates the feature representation of the start word of the slot, Indicates the end word feature representation of the slot.

7. The method for extracting energy and power document-level event arguments based on dependency perception and role guidance according to claim 1 is characterized in that: In S5, the method for calculating the boundary loss is as follows: Based on the contextual semantic representation H X and argument roles guide span selectors First calculate the position distribution of the start word and end word of each slot, each All from H X Extract an argument span and express it as (s k ,e k ), s k is the start index of the span, e k is the end index of the span. When there is no span of this argument in the context, the span is represented as (0,0). The position distribution calculation formula of the start word and end word of each slot is as follows: Then, the probability distribution of the slot start and end word positions is calculated. The specific formula is as follows: Finally, the loss function is: in, is the feature representation of the start word of the slot, is the feature representation of the end word of the slot, H X is the contextual semantic representation, is the probability of the starting word position of the slot, is the probability of the end word of the slot, Softmax(·) represents the activation function, and the logit function is the inverse function of the sigmoid function. is the logarithmic probability of the slot starting position, is the logarithmic probability of the slot end position probability, BoundaryLoss represents the loss function, s k Indicates the starting index of the span, e k Indicates the ending index of the span.

Images