An energy power document-level event argument extraction method based on dependency perception and role guidance

Abstract

The application provides an energy power document-level event argument extraction method based on dependency perception and role guidance, and steps include: S1, data is acquired and preprocessed, S2, a prompt template is constructed, S3, a dependency perception self-attention module is constructed, S4, an argument role guidance span selector is constructed, and S5, boundary loss is calculated. The application increases the dependency information between roles by constructing a prompt template for an event, improves the integrity of event extraction, separately models argument role slots, simultaneously adopts boundary loss to limit argument boundaries, effectively reduces the deviation of a prediction result, and improves the precision of argument extraction.

Description

Technical Field

[0001] This invention belongs to the field of energy and power automation technology, and in particular relates to a method for extracting document-level event arguments in energy and power based on dependency awareness and role guidance. Background Technology

[0002] With the rapid development of the energy and power industry, the demand for automation and intelligence in power systems is increasing. This has led to the generation of a large amount of document data in power systems, such as equipment operation reports, fault records, and dispatch instructions. These documents contain rich event information, such as equipment failures, power dispatch, and load changes. Automatically extracting event arguments (such as the subject, time, location, and cause of the event) from these documents is of great significance for the automated management, fault diagnosis, and decision support of power systems.

[0003] Currently, event argument extraction techniques primarily rely on information extraction methods from Natural Language Processing (NLP). Traditional event argument extraction methods are typically based on rule-based or statistical models, which have achieved some success in extracting events from short texts or at the sentence level. However, in the energy and power sector, event information is often distributed across multiple sentences or even multiple documents. Traditional sentence-level event extraction methods struggle to capture long-distance dependencies at the document level, leading to insufficient accuracy and completeness in the extraction results. Furthermore, existing methods often neglect the dependencies and role information between event arguments. In energy and power documents, complex dependencies often exist between event arguments; for example, a device malfunction may trigger multiple events, and these event arguments may have specific role associations. Existing methods lack effective modeling of these dependencies and role information, resulting in poor accuracy and completeness in the extraction results. Summary of the Invention

[0004] The purpose of this invention is to provide a method for extracting event arguments from energy and power documents based on dependency awareness and role guidance, in order to solve the problems of insufficient research on dependency relationships in the extraction of event arguments from long documents in the energy and power field, as well as poor extraction accuracy and completeness.

[0005] The technical solution adopted in this invention is a method for extracting document-level event arguments in the energy and electricity sector based on dependency awareness and role guidance, comprising the following steps:

[0006] Step S1: Acquire data and preprocess the data;

[0007] Step S2, construct the prompt template;

[0008] Step S3: Construct a dependency-aware self-attention module;

[0009] Step S4: Construct an argument role-guided span selector;

[0010] Step S5: Calculate the boundary loss.

[0011] Furthermore, in S1, for document X, given an event trigger word x trig In the trigger word x trig Insert at the front and back respectively <t> 、< / t> The updated document It is expressed as follows:

[0012]

[0013] in, Indicates the updated document, x trig Indicates the event trigger word, x n This represents the nth word in the document. <t>and< / t> These are special symbols used to mark the period before and after the trigger word.

[0014] Furthermore, the characteristic is that, in S2, for each event type e in the dataset D... i Construct a system that includes all its argument roles. prompt template Where r is the argument role, For all argument roles included in event i.

[0015] Furthermore, the method for constructing the prompt template in S2 includes:

[0016] (1) Manual prompts: The roles are manually connected through natural language to form a complete semantic template. First, all argument roles are listed, and then words with semantic relationships are connected to make the semantics complete.

[0017] (2) Spelling prompts: directly link the character names to form a structured template without adding other words, and only by listing all the argument characters to form the template;

[0018] (3) Soft hints: A learnable role-specific pseudo-tokens are dynamically generated templates to calculate the number of argument roles contained in the event and supplement an equal number of pseudo-tokens. <v> Victor< / v> Finally, an event template is generated.

[0019] Furthermore, in S3, the specific steps for constructing the dependency-aware self-attention module are as follows:

[0020] S31, the document updated by S1 Input the pre-trained encoder to generate the contextual semantic representation H X The details are as follows:

[0021]

[0022] At the same time, the event notification template Also input into the pre-trained encoder to generate the initial representation of the cue template. Specifically as follows:

[0023]

[0024] S32, strengthen the dependencies between roles by designing a role dependency-aware mask matrix M in the self-attention layer of the decoder. role Then, the self-attention calculation for role perception is performed, using the following formula:

[0025]

[0026] in, Let P be the event cue template, Softmax(·) be the activation function, and T be the matrix transpose. Indicates event e i The prompt template, SelfAttention(·) represents the self-attention mechanism, Q, K, and V represent the query, key, and value matrices, respectively, and M role d represents the role dependency perception mask matrix, and d represents the dimension scaling factor;

[0027] S33, in the cross-attention layer of the decoder, the contextual semantic representation H is... X Representing character perception cues The interaction process is as follows:

[0028]

[0029] CrossAttention is a cross-attention mechanism, where H... p As a prompt, This is a cue for character perception. This is the initial representation of the prompt template.

[0030] Furthermore, in S4, the method for constructing the argument role-guided span selector is as follows:

[0031] Based on S33, the representation corresponding to the kth slot in the prompt template will be changed from H. p Average pooling is used to obtain the character feature θ. k The specific formula is as follows:

[0032]

[0033] MeanPooling(·) represents average pooling. This indicates the starting index of the k-th slot in the prompt template. H represents the end index of the k-th slot in the prompt template. p This is to indicate a prompt;

[0034] A character may have multiple slots, and correspondingly, multiple character features and character fillers, for a given character feature θ. k have:

[0035]

[0036] The obtained argument role-guided span selector Represented as:

[0037]

[0038] Among them, w (start) and w (end) It is a learnable parameter matrix shared by all argument roles, w (start) Let w represent the starting matrix. (end) This indicates the ending matrix, and ° indicates element-wise multiplication. The feature representation of the start word of the slot. The end-word feature of the slot is represented.

[0039] Furthermore, in S5, the method for calculating the boundary loss is as follows:

[0040] H based on context semantic representation X And the role guide span selector First, calculate the positional distribution of the start and end words for each slot. All from H X Extract an argument span from the given information and represent it as (s). k ,e k ), s k e is the starting index of the span. k The span is represented as (0,0) when there is no span for this argument in the context. The formula for calculating the positional distribution of the start and end words of each slot is as follows:

[0041]

[0042] Then, calculate the probability distribution of the positions of the start and end words of the slot, using the following formula:

[0043]

[0044] Finally, the loss function is:

[0045]

[0046] in, The feature representation of the starting word of the slot. H represents the end-word feature of the slot. X For contextual semantic representation, The probability of the position of the starting word of the slot. Let represent the probability of the end word position of the slot, Softmax(·) denotes the activation function, and the logit function is the inverse function of the sigmoid function. Let be the logarithmic probability of the starting position of the slot. Let s be the log odds of the probability of the slot ending position, and let BoundaryLoss represent the loss function. k The starting index of the span, e k Indicates the end index of the span.

[0047] The beneficial effects of this invention are:

[0048] 1. This invention helps to increase the dependency information between roles and improve the completeness of event extraction by constructing prompt templates for events.

[0049] 2. By constructing a dependency-aware self-attention module and selecting highly relevant roles for modeling, this invention helps to accurately focus on key role relationships and enhances the ability to capture important semantic information.

[0050] 3. This invention models the argument role slots separately through the argument role-guided span selector module, predicts the start and end positions of arguments, and uses boundary loss to restrict the argument boundaries, thereby effectively reducing the deviation of the prediction results and improving the accuracy of argument extraction.

[0051] 4. This invention calculates boundary loss, quickly converges the model, accurately captures argument boundaries, and improves the accuracy of argument extraction. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0053] Figure 1 This is a flowchart of the present invention.

[0054] Figure 2 This is a flowchart of event argument extraction in this invention.

[0055] Figure 3 The role dependency-aware mask matrix M in this inventionrole Representation diagram. Detailed Implementation

[0056] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0057] Example

[0058] Embodiments of the present invention provide a method for extracting document-level event arguments in the energy and power sector based on dependency awareness and role guidance, as shown in the flowchart below. Figure 1 As shown, the steps include:

[0059] Step S1: Obtain data from dataset D and preprocess the data. In the document-level event argument extraction task, insert special symbols before and after the trigger words given in the annotated corpus. <t> 、< / t> To explicitly tag trigger words and enhance the model's awareness of their location, the process of extracting event arguments is as follows: Figure 2 As shown. For example, for the trigger word "attack", the tagged form is " <t> attack< / t> This method not only helps the model accurately locate trigger words, but also enhances the interaction between trigger words and contextual information, thereby capturing event arguments more precisely. By embedding the tagged trigger words into the original text, the model can better understand the event structure, improving the accuracy and robustness of event extraction. The specific method is as follows:

[0060] For document X, given an event trigger word x trig In the trigger word x trig Insert at the front and back respectively <t> 、< / t> The updated document It is expressed as follows:

[0061]

[0062] in, Indicates the updated document, x trig Indicates the event trigger word, x n This represents the nth word in the document. <t> and< / t> These are special symbols used to mark the period before and after the trigger word.

[0063] Step S2, construct a prompt template for each event type e in dataset D. i Construct a system that includes all its argument roles. prompt template Where r is the argument role, For all argument roles included in event i, specifically, taking the event type "transport" as an example, its argument role set is {Importer, artifact, origin, destination, vehicle}, and the corresponding prompt template is defined as " Importer transport artifact from origin to destination by vehicle The . symbol is used to capture implicit interaction relationships between arguments, where each argument is defined as an independent "slot" (as shown in the underlined part in the example).

[0064] To adapt to different scenario requirements, this invention proposes three methods for constructing prompts:

[0065] (1) Manual prompting: Roles are manually connected using natural language to form a complete semantic template. First, all argument roles are listed, then connected by words with semantic relationships to make the meaning complete and the sentence fluent (e.g., "... Importer transport artifact from origin to destination by vehicle .");

[0066] (2) Concatenation prompts: directly string together the character names to form a structured template, without adding other words, only by listing all argument characters to form the template (e.g., "Transport:"). importerartifactorigindestinationvehicle ”);

[0067] (3) Soft hints: A learnable role-specific pseudo-tokens are dynamically generated templates to calculate the number of argument roles contained in the event and supplement an equal number of pseudo-tokens. <v> Victor< / v> Finally, an event template (such as "Transport:") is generated. <v> Victor< / v> <v> Victor< / v> <v> Victor< / v> <v> Victor< / v> <v> Victor< / v> ”).

[0068] The three types of templates enhance the model's ability to model the relationship between event roles from three dimensions: semantic coherence, structural simplicity, and dynamic adaptability, providing a structured input foundation for subsequent dependency perception and role guidance mechanisms.

[0069] In this invention, steps S1 and S2 are performed simultaneously. The preprocessing in S1 involves processing specific documents, while the prompt template in S2 is for event types.

[0070] Step S3, construct the dependency-aware self-attention module, the specific steps are as follows:

[0071] S31, the document updated by S1 Input the pre-trained encoder to generate the contextual semantic representation H X The details are as follows:

[0072]

[0073] At the same time, the event notification template Also input into the pre-trained encoder to generate the initial representation of the cue template. Specifically as follows:

[0074]

[0075] S32, strengthen the dependencies between roles by designing a role dependency-aware mask matrix M in the self-attention layer of the decoder. role ,like Figure 3 As shown, this matrix is ​​initialized based on the co-occurrence frequency of roles and is used to constrain the self-attention mechanism to establish strong connections only between highly relevant roles before performing role-aware self-attention calculation. The specific formula is as follows:

[0076]

[0077] in, Let P be the event cue template, Softmax(·) be the activation function, and T be the matrix transpose. Indicates event e i The prompt template, SelfAttention(·) represents the self-attention mechanism, Q, K, and V represent the query, key, and value matrices, respectively, and M role d represents the role-dependent perception mask matrix, and d represents the dimension scaling factor.

[0078] S33, in the cross-attention layer of the decoder, the contextual semantic representation H is... X Representing character perception cues Interaction is achieved to realize information fusion, enabling the model to simultaneously capture the global semantics of the document and the fine-grained dependencies between roles, providing multi-level feature support for subsequent argument span extraction. The process is as follows:

[0079]

[0080] CrossAttention is a cross-attention mechanism, where H... p As a prompt, This is a cue for character perception. This is the initial representation of the prompt template.

[0081] Step S4: Construct an argument role-guided span selector. After successfully capturing the dependency relationship between highly relevant roles, in order to further improve the accuracy and efficiency of the event argument extraction task and deeply explore the unique semantic value of each argument role, this invention conducts independent modeling work for each argument role slot and designs an argument role-guided span selector. It fully considers the specific needs of different argument roles and can accurately extract event arguments.

[0082] Based on S33, the representation corresponding to the kth slot in the prompt template will be changed from H. p Average pooling is used to obtain the character feature θ. k The specific formula is as follows:

[0083]

[0084] MeanPooling(·) represents average pooling. This indicates the starting index of the k-th slot in the prompt template. H represents the end index of the k-th slot in the prompt template. p This is a prompt / indication.

[0085] A character may have multiple slots, and correspondingly, multiple character features and character fillers, for a given character feature θ. k have:

[0086]

[0087] The obtained argument role-guided span selector Represented as:

[0088]

[0089] Among them, w (start) and w (end) It is a learnable parameter matrix shared by all argument roles, w (start) Let w represent the starting matrix. (end) This indicates the ending matrix, and ° indicates element-wise multiplication. The feature representation of the start word of the slot. The end-word feature of the slot is represented.

[0090] Step S5, calculate the boundary loss based on the contextual semantic representation H. X And the role guide span selector First, calculate the positional distribution of the start and end words for each slot. All from H X Extract an argument span from the given information and represent it as (s).k ,e k ), s k e is the starting index of the span. k The span is represented as (0,0) when there is no span for this argument in the context. The formula for calculating the positional distribution of the start and end words of each slot is as follows:

[0091]

[0092] Then, calculate the probability distribution of the positions of the start and end words of the slot, using the following formula:

[0093]

[0094] Finally, the loss function is:

[0095]

[0096] in, The feature representation of the starting word of the slot. H represents the end-word feature of the slot. X For contextual semantic representation, The probability of the position of the starting word of the slot. Let represent the probability of the end word position of the slot, Softmax(·) denotes the activation function, and the logit function is the inverse function of the sigmoid function. Let be the logarithmic probability of the starting position of the slot. Let s be the log odds of the probability of the slot ending position, and let BoundaryLoss represent the loss function. k The starting index of the span, e k Indicates the end index of the span.

[0097] This invention constructs prompt templates tailored to task characteristics, and then strengthens the interaction and dependency relationships between event roles through a self-attention module. With the guidance of the prompt templates, the model can accurately focus on key role information, thereby improving the accuracy of event argument extraction. Furthermore, this invention also designs an argument role-guided span selector module. This module models each event role individually, fully considering the uniqueness of each role, and performs precise matching and assignment based on contextual information. This can efficiently identify complex relationships between event roles and arguments, improving the accuracy of document-level event argument extraction and providing an effective technical path to solve the challenge of argument extraction in complex text.

[0098] Experimental verification

[0099] The results of comparing the present invention with existing technology models are shown in Table 1:

[0100] Table 1 Comparison results between the present invention and the prior art

[0101]

[0102] In Table 1, the Arg-I evaluation metric represents the accuracy of argument recognition in event argument identification, and the Arg-C evaluation metric represents the accuracy of argument classification in event argument identification. Bold text indicates the best result, and underlined text indicates the second-best result. This invention outperforms other methods in both evaluation metrics on the RAMS (Richly Annotated Multilingual Schema-guided Event Structure), WikiEvents, and MLEE (Multi-Level Event Extraction) datasets, demonstrating significant improvements in both recognition and classification. This proves that this invention, through its unique role dependency awareness module and argument role-guided span selector, can deeply explore the semantic connotations of roles in different contexts and accurately capture subtle differences and special semantics of roles. This invention integrates role dependency information into the model training process by constructing task-specific prompt templates and strengthens the interaction relationships between roles through a self-attention mechanism. This allows the model to clearly grasp the complex implicit dependencies between roles, resulting in excellent performance in the recognition and classification of complex arguments and improving the accuracy of document-level event argument extraction.

[0103] The various embodiments in this specification are described in a related manner. Similar or identical parts between embodiments can be referred to mutually. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions of the method embodiments.

[0104] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention are included within the scope of protection of the present invention.

Claims

1. A method for extracting document-level event arguments in the energy and electricity sector based on dependency awareness and role guidance, characterized by the following steps: include: Step S1: Acquire data and preprocess the data; Step S2, construct the prompt template; Step S3: Construct a dependency-aware self-attention module; Step S4: Construct an argument role-guided span selector; Step S5: Calculate the boundary loss; In step S3, the specific steps for constructing the dependency-aware self-attention module are as follows: S31, the document updated by S1 Input the pre-trained encoder to generate a contextual semantic representation. The details are as follows: At the same time, the event notification template Also input into the pre-trained encoder to generate the initial representation of the cue template. The details are as follows: S32, strengthen the dependencies between roles by designing a role dependency-aware mask matrix in the decoder's self-attention layer. Then, the self-attention calculation for role perception is performed, using the following formula: in, This is a cue for character perception. This is an event notification template. This represents the activation function. For matrix transpose, Indicates an event The prompt template This represents the self-attention mechanism, where Q, K, and V represent the query, key, and value matrices, respectively. This represents the role-dependent perception mask matrix. Indicates the dimension scaling factor; S33, in the cross-attention layer of the decoder, the contextual semantic representation is... Representing character perception cues The interaction process is as follows: CrossAttention is a cross-attention mechanism. As a prompt, This is a cue for character perception. This is the initial representation of the prompt template.

2. The method for extracting document-level event arguments in the energy and power sector based on dependency awareness and role guidance, as described in claim 1, is characterized in that... In S1, for the document Given an event trigger word In the trigger word Insert at the front and back respectively Updated document It is expressed as follows: in, This indicates the updated document. Indicates event trigger words, Indicates the first in the document One word, and These are special symbols used to mark the period before and after the trigger word.

3. The method for extracting document-level event arguments for energy and power based on dependency awareness and role guidance according to claim 1, characterized in that, In S2, for the dataset Each event type in Construct a system that includes all its argument roles. prompt template ,in, For the sake of the meta-role, For the event All argument roles included in it.

4. The method for extracting document-level event arguments in the energy and power sector based on dependency awareness and role guidance, as described in claim 1, is characterized in that... The method for constructing the prompt template in S2 includes: (1) Manual prompts: The roles are manually connected by natural language to form a complete semantic template. First, all argument roles are listed, and then words with semantic relationships are connected to make the semantics complete. (2) Spelling prompts: directly link the character names to form a structured template, without adding other words, and only by listing all the argument characters to form the template; (3) Soft hints: A learnable role-specific pseudo-tokens are used to dynamically generate templates, calculate the number of argument roles contained in the event, and supplement an equal number of pseudo-tokens. Finally, an event template is generated.

5. The method for extracting document-level event arguments for energy and power based on dependency awareness and role guidance according to claim 1, characterized in that, In S4, the method for constructing the argument role-guided span selector is as follows: Based on S33, the first in the prompt template Each slot corresponds to a representation from Average pooling is used to obtain character features. The specific formula is as follows: in, Indicates average pooling. This indicates the first one in the prompt template. The starting position index of each slot. This indicates the first one in the prompt template. The end position index of each slot. This is to indicate a prompt; A character may have multiple slots, and correspondingly, multiple character traits and character fillers. For a given character trait... have: The obtained argument role-guided span selector Represented as: in, and It is a learnable parameter matrix shared by all argument roles. Indicates the starting matrix. Indicates the ending matrix, This represents element-wise multiplication. The feature representation of the start word of the slot. The end-word feature of the slot is represented.

6. The method for extracting document-level event arguments for energy and power based on dependency awareness and role guidance according to claim 1, characterized in that, In S5, the method for calculating the boundary loss is as follows: Context-based semantic representation And the role guide span selector First, calculate the positional distribution of the start and end words for each slot. All from Extract an argument span and represent it as The starting index for the span, The span is represented as (0,0) when there is no span for this argument in the context. The formula for calculating the positional distribution of the start and end words of each slot is as follows: Then, calculate the probability distribution of the positions of the start and end words of the slot, using the following formula: Finally, the loss function is: in, The feature representation of the starting word of the slot. The probability of the position of the end word of the slot. This represents the activation function. The function is The inverse function of a function Let be the logarithmic probability of the starting position of the slot. Let be the logarithmic probability of the end position of the slot. Represents the loss function. Indicates the starting index of the span. Indicates the end index of the span.

Images