Autoregression event argument extraction method based on multi-view prompt learning template

Abstract

The invention discloses an autoregression event argument extraction method based on a multi-view prompt learning template, and belongs to the technical field of event argument extraction. The method aims at solving the problems that an existing event argument extraction technology excessively depends on a single template, and information interaction between event elements in the template is insufficient. And through the multi-view prompt learning template, learning the interdependence relationship among the event elements. The model uses an encoder-decoder architecture sharing a word vector embedding layer, and an input text firstly captures semantic information through an encoder and transmits the semantic information to a decoder for sequence generation. In the reasoning stage, a decoder is combined with a constraint decoding strategy to guide a model to generate structured event element sequences in different event prompt template formats, and finally, output results from different perspectives are voted through an integration strategy, so that the robustness of event argument extraction results is ensured, and more accurate event arguments are obtained. In addition, the method is also suitable for the field of event argument extraction in a low-resource environment.

Description

Technical Field

[0001] The present invention relates to the field of event argument extraction, and in particular to an autoregressive event argument extraction method based on a multi-view prompt learning template. Background Art

[0002] Event argument extraction technology is to extract argument information corresponding to a specific event from a large amount of heterogeneous data, and realize the function of converting massive unstructured text into event argument data with a unified format. In recent years, more and more researchers have used prompt learning as a bridge between unstructured text and structured event argument data, and improved the performance of event argument extraction by designing prompt learning templates. Existing research, such as manual prompt templates written by humans, soft prompt templates based on pseudo-words, and hard prompt templates that directly splice event argument roles, mainly set a single event argument prompt template with a fixed argument role order based on the argument roles corresponding to each event type. This makes the model results heavily dependent on the design of a single template. The error accumulation of a single perspective will seriously affect the subsequent prediction results of the autoregressive model, resulting in weak model stability. At the same time, event elements are not independent, and the fixed event argument prompt template limits the information interaction between event elements in the template.

[0003] Therefore, the present invention proposes an autoregressive event argument extraction method based on a multi-perspective prompt learning template. At the same time, from the three perspectives of event trigger words, event arguments, and argument roles, a prompt learning template with multiple event element combination modes is designed, which effectively promotes the information interaction between event elements in the template. At the same time, voting is performed based on the event argument prediction results generated by the model from various perspectives to enhance the robustness of the model. In addition, a constrained decoding strategy is introduced in the model reasoning stage to improve the context relevance and accuracy of the model output. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that the existing autoregressive event argument extraction model based on prompt learning is limited by a single, fixed prompt template design, resulting in excessive reliance on a single template prompt and insufficient information interaction between event elements in the fixed template, which leads to overfitting of the learned semantic representation to the training data, and then leads to poor generalization of the model. Therefore, the present invention simulates the process of humans solving problems from different angles, and uses a multi-perspective prompt learning template to allow the model to learn the information interaction between event elements from the dynamic event element combination pattern, thereby generating a more complete word unit representation, and comprehensively considering the event argument prediction results generated by the model from various perspectives, alleviating the model's potential error accumulation for a single template prompt, and then improving the robustness of event argument extraction, thereby solving the problems mentioned in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] An autoregressive event argument extraction method based on a multi-view prompt learning template, wherein the event argument extraction method has the following specific steps:

[0007] Preferably, the specific steps of step S10 are as follows:

[0008] Step S101: the original sentence set sentenCe is known set The representation of the I-th sentence in is S I =[w1, w2, ... w a ,…,w b ,…,w c ,…,w d ,…,w e ,…,w f ,…,w n ], n is the sentence S I The number of tokens in w is the length of the sentence. a ,…,w b is the event trigger word t of the sentence, and the corresponding event type is e; w c ,…,w d With w e ,…,w f are the two event arguments a1 and a2 of the sentence, and their corresponding argument roles are r1 and r2 respectively. If the argument role set R corresponding to the event type e is known e =[r1, r2, r3], for the event trigger word, event argument, argument role, special marks [T], [A], [R] are used to indicate the order of appearance. According to the order of appearance, the following four perspective element prompt sequences can be constructed: [T]→[A]→[R], [T]→[R]→[A], [A]→[R]→[T], [R]→[A]→[T]. Construct a multi-perspective event argument prompt template based on the prompt sequence and the shuffled argument role arrangement order in,

[0009]

[0010]

[0011]

[0012]

[0013] In the template, [SSEP] represents the separator between elements.

[0014] Step S102: construct a multi-perspective target text based on the order of occurrence of elements of the event argument prompt template:

[0015]

[0016]

[0017]

[0018]

[0019] Among them, null means that the event argument corresponding to the argument role r3 does not appear in the sentence. The target text is indexed by the vocabulary to obtain the target sequence Index() represents the word-index mapping function based on the model vocabulary;

[0020] Step S20, concatenating the multi-view event argument prompt template obtained in S10 with the sentence to construct the input data of the model;

[0021] Preferably, the specific steps of step S20 are as follows:

[0022] Step S201: Sentence S I Combined with the event argument prompt template to form a new sentence representation As the input of the model. Among them, Represents the concatenation operation of adjacent sequences.

[0023] Step S30, constructing a sentence semantic information encoder Φ based on an autoregressive pre-trained language model s , denote the above sentence as S′ I Input to the semantic information encoder Φ s Generate sentence vector representation in Where n is the length of the original sentence encoding, p is the length of the event argument prompt template encoding, d is an integer, (n+p)*d represents Dimensions;

[0024] Preferably, the specific steps of step S40 are as follows:

[0025] Step S401, construct a sequence-to-sequence decoder based on the autoregressive pre-trained language model. The decoder is constructed by H′ I Initialize the hidden state. In order to maintain the consistency of the mapping from the index word list to the shallow word vector feature space, the decoder uses the word vector embedding layer Emb that shares weight parameters with the encoder. vocab , to establish a shared mapping space from the index word table to the word vector feature weights, and then generate word units to obtain the output sequence Where T represents the length of the output sequence.

[0026] Preferably, the specific steps of step S50 are as follows:

[0027] Step S501, in the training phase, in the original sentence set senten|e set Extract the training dataset Train set When , according to the number of resources required by the task scenario, for the training data set Train set Set the sampling coefficient α∈(0, 1] from the training data set Train set Random sampling to obtain a training dataset with a specific amount of resources Number() represents the number of sample sentences in the dataset. If α∈(0, 1), it is a low-resource training scenario. If d=1, it is a regular training scenario. Then, the training data is passed into the model to obtain the output sequence Y generated by the model. I .

[0028] Step S502: Pair the target sequence-output sequence obtained in S10 and S501. And minimize the negative log-likelihood loss function in, represents the expected operation, M is the length of the target sequence, y ＜m are all the tokens generated before the mth token, p(y m |g,y ＜m ) represents the word sequence y generated by the model before the mth step in the given target sequence during the model training phase ＜m Under the condition that m probability.

[0029] Preferably, the specific steps of step S60 are as follows:

[0030] Step S601, in the reasoning stage, a constraint decoding strategy for constraining the model to generate structured sequences is designed based on the permutation and combination patterns of trigger words, event arguments, and argument roles, and the structured information of different element permutations and combinations is introduced into the decoding process of the sequence decoder.

[0031] The constraint decoding strategy is as follows: In the inference phase, the model dynamically adjusts the generation list of the next word based on the word generated in the current state. If the current word is decoded as "[", the word generated in the next state will be selected from [T], [A], and [R]; if the word generated in the current state is [T] and [A], the word generated in the next state will be selected from the sentence S I and [SSEP]; if the word generated in the current state is [R], the word generated in the next state will be selected from the argument role set R eand [SSEP].

[0032] The beneficial effects of the present invention are:

[0033] The purpose of the present invention is to solve the problem that the existing autoregressive event argument extraction model based on prompt learning is limited by a single, fixed prompt template design, resulting in excessive reliance on a single template prompt and insufficient information interaction between event elements in the fixed template, which leads to overfitting of the learned semantic representation to the training data, and then leads to poor model generalization. Therefore, the present invention simulates the process of humans solving problems from different angles, and uses a multi-perspective prompt learning template to allow the model to learn the information interaction between event elements from the dynamic event element combination pattern, thereby generating a more complete word unit representation, and comprehensively considering the event argument prediction results generated by the model from various perspectives, alleviating the model's potential error accumulation for a single template prompt, and then improving the robustness of event argument extraction, thereby solving the problems mentioned in the above background technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 The figure is a flow chart of the steps of the method of the present invention. DETAILED DESCRIPTION

[0035] The following is a detailed description of the embodiments of the present invention. Although the present invention will be described and illustrated in conjunction with some specific embodiments, it should be noted that the present invention is not limited to these embodiments. On the contrary, modifications or equivalent substitutions made to the present invention should all be included in the scope of the claims of the present invention.

[0036] In addition, in order to better illustrate the present invention, numerous specific details are given in the following specific embodiments. It will be understood by those skilled in the art that the present invention can also be implemented without these specific details.

[0037] See also Figure 1 , the present invention provides a technical solution:

[0038] The present invention provides the following technical solution: a dual-coding pre-training small sample relationship extraction method based on contrastive learning. The specific steps take the small sample relationship extraction of news as an example. The specific steps of applying the small sample relationship extraction method to the field of news relationship extraction are as follows:

[0039] Step 1: Based on the event types, event arguments, and argument roles contained in the Wikipedia event dataset, a multi-view event argument prompt learning template and target sequence are constructed for each sentence;

[0040] Step 1-1, obtain the event-argument role set label on the Wiki event dataset set, and split the sentences containing multiple events into sentences of a single event news original sentence set sentence set . The original representation of each news sentence is:

[0041] S I =[w1, w2, ... w a ,…,w b ,…,w c ,…,w d ,…,w e ,…,w f ,…,w n ],0<I<N

[0042] Where n is the number of words in the sentence, N is the number of sentences in the news sentence set, and in sentence S I In, w a ,…,w b is the event trigger word t, and the corresponding event type is e; w c ,…,w d With w e ,…,w f They are the two event arguments a1 and a2 corresponding to the event type, and their argument roles are r1 and r2 respectively.

[0043] Step 1-2: Construct a multi-perspective event argument prompt learning template P for each event type e involved in each sentence I For event trigger words, event arguments, and argument roles, special markers [T], [A], and [R] are used to indicate the order of occurrence. According to the order of occurrence, the following four perspectives of element prompt sequences can be constructed: [T]-→[A]-→[R], [T]→[R]→[A], [A]-→[R]→[T], [R]→[A]→[T]. set Get all argument role sets R corresponding to event type e e = [r1, r2, r3], construct a multi-perspective event argument prompt template based on the prompt order and the disrupted argument role arrangement order:

[0044] in,

[0045]

[0046]

[0047]

[0048]

[0049] In the template, [SSEP] represents the separator between elements.

[0050] Steps 1-3: Construct the corresponding target sequence based on each sentence and its multi-perspective event argument prompt learning template. Construct the multi-perspective target text based on the order of occurrence of elements in the event argument prompt template.

[0051]

[0052]

[0053]

[0054]

[0055] In the sentence, null indicates that the event argument corresponding to the argument role r3 does not appear in the sentence. The target text is indexed by the vocabulary to obtain the target sequence Index() represents a term-index mapping function based on the model vocabulary.

[0056] Step 2: Construct the input data of the model. I and event argument prompt template P I Fusion to form new sentence representation As the input of the model. Among them, Represents the concatenation operation of adjacent sequences.

[0057] Step 3: Construct a sentence semantic information encoder Φ based on the autoregressive pre-trained language model s , and design the word vector embedding layer Emb with shared weight parameters for the encoder vocab , denote the above sentence as S′ I Input to the semantic information encoder Φ s Generate sentence vector representation in Among them, θ e represents the encoder model parameters, n is the length of the original sentence encoding, p is the length of the event argument prompt template encoding, d is an integer, (n+p)*d represents The dimension of k represents the serial number of the event argument prompt template. Encoder(*) is an encoder function that converts the input sequence into a vector representation. Emb vocab (*) is the mapping function from word unit to word vector. Obviously, after the encoder Φ s The generated sentence vector representation H′ I It consists of two parts, namely the original sentence S I The corresponding vector represents h i (1≤i≤n), and event argument prompt template The corresponding vector representation

[0058] Step 4: Construct a sequence-to-sequence decoder based on the autoregressive pre-trained language model. The decoder processes the sentence vector representation to obtain the decoder initial hidden state s0 = f(H′ I θ d ), where f is a learnable transformation function and θ d represents the decoder model parameters. At each time step m, the decoder relies on the previously generated word and the previous hidden state s t-1 Generate the next token Finally, we get the output sequence Where T represents the length of the output sequence.

[0059] Step 5: During the training phase, the decoder minimizes the target sequence g under each view I With the output sequence Y I The negative log-likelihood loss function of ;

[0060] Step 5-1, in the training phase, in the original sentence set sentence set Extract the training dataset Train set When , according to the number of resources required by the task scenario, for the training data set Train set Set the sampling coefficient α∈(0, 1] from the training data set Train set Random sampling to obtain a training dataset with a specific amount of resources Number() represents the number of sample sentences in the dataset. If α∈(0,1), it is a low-resource training scenario. If α=1, it is a regular training scenario. Then, the training data is passed into the model to obtain the output sequence Y generated by the model. I .

[0061] Step 5-2, obtain the target sequence-output sequence pair from step 1 and step 5-1 And calculate its negative log-likelihood loss function in, represents the expected operation, M is the length of the target sequence, y ＜m are all the tokens generated before the mth token. m |g,y ＜m ) represents the word sequence y generated by the model before the mth step in the given target sequence during the model training phase ＜m Under the condition that m probability.

[0062] Step 6: In the inference phase, the decoder generates a structured sequence that conforms to a specific perspective based on the order of arguments based on the constraint decoding strategy.

[0063] Step 6-1, design the constrained decoding strategy as follows: In the inference stage, the model dynamically adjusts the generation list of the next word based on the word generated in the current state. If the current word is decoded as "[", the word generated in the next state will be selected from [T], [A], [R]; if the word generated in the current state is [T], [A], the word generated in the next state will be selected from the sentence S I and [SSEP]; if the word generated in the current state is [R], the word generated in the next state will be selected from the argument role set R e and [SSEP]. Through the constrained decoding strategy, the decoder will confirm the candidate sequence according to the above three provisions when using the beam search algorithm to maintain the candidate sequence at each time step during the inference phase.

[0064] An autoregressive event argument extraction device based on a multi-view prompt learning template includes a data preprocessing module for generating a multi-view prompt learning template, a sentence semantic information representation encoder module, a sequence-to-sequence autoregressive decoder module, a constrained decoding strategy for constraining the beam search process of the autoregressive decoder in the inference phase, and a voting strategy based on the output results of the multi-view template, specifically:

[0065] Based on the event-argument role set, a multi-perspective prompt learning template and the corresponding target sequence are designed for each sentence, and the position of event elements is adjusted in multiple ways to guide the model to generate sequences from different perspectives.

[0066] Create a sentence semantic information representation encoder to generate sentence semantic information representation based on semantics and prompt templates;

[0067] A sequence-to-sequence decoder based on an autoregressive pre-trained language model is created, and structured text sequences under different perspective prompts are generated based on a constrained decoding strategy. A voting strategy is then used to screen the generated event argument results to improve the event argument extraction performance.

[0068] Since the present invention adopts the above scheme, it has the following characteristics:

[0069] By using multi-perspective prompting to learn templates, the present invention solves the problems of over-reliance on a single template and insufficient information interaction of event elements within the template. Specifically, by designing event argument prompt templates from four different perspectives, information interaction between event elements within the template is promoted, thereby improving the model's ability to learn the interdependencies between event elements. This multi-perspective design enables the model to understand and generate event arguments from different perspectives, thereby improving the accuracy and robustness of the extraction. Compared with traditional single-perspective templates, this method significantly improves the performance of event argument extraction.

[0070] In the encoder-decoder architecture, the input text first captures semantic information through the encoder, and then the decoder uses this information to generate a structured sequence of event elements. This design effectively combines the semantic understanding ability of the encoder and the sequence generation ability of the decoder, improving the efficiency and quality of event argument extraction.

[0071] In the inference phase, the decoder combines the constrained decoding strategy to generate structured event element sequences in different event prompt template formats. This strategy improves the contextual relevance and accuracy of the output sequence by limiting the model's choices during the generation process. At the same time, the robustness of the event argument extraction results is further enhanced by voting on the output results from different perspectives through an integrated strategy.

[0072] In addition, the present invention also considers the application in low-resource environment, and effectively adapts to the resource-constrained scenario by setting the sampling coefficient α to sample a training data set of a specific resource amount from the training data. This design makes the present invention applicable not only to conventional training scenarios, but also to low-resource environments, which improves the scope of application and flexibility of the method.

[0073] In summary, the present invention effectively improves the performance of event argument extraction through the combination of multi-view prompt learning templates, encoder-decoder architecture, constrained decoding strategy and integration strategy, solves the problems in the prior art, and provides new ideas and methods for the field of event argument extraction.

[0074] Although the present invention has been described in detail with reference to the aforementioned embodiments, it is still possible for those skilled in the art to modify the technical solutions described in the aforementioned embodiments, or to make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the protection scope of the present invention.

Claims

1. A method for extracting autoregressive event arguments based on multi-view prompt learning templates, characterized in that: The following steps are involved: step S10: Based on the event arguments involved in the event in the sentence, construct a multi-perspective event argument prompt learning template and target sequence g I , used to calculate the loss function in step 5; Step S20: Concatenate the sentence with the multi-view event argument prompt template to form a new sentence representation S′ I , as the encoder Φ in step 3 s Input; Step 830, construct a sentence semantic information encoder Φ based on the autoregressive pre-trained language model s , generate sentence vector representation H′ I , as the initial hidden state of the decoder in step 4; Step S40: Create an autoregressive sequence decoder, process the sentence vector representation to generate the decoder output sequence Y I ; Step S50: In the training phase, in the original sentence set sentence set Extract the training dataset Train set When , according to the number of resources required by the task scenario, for the training data set Train set Set the sampling coefficient α∈(0, 1] from the training data set Train set Random sampling to obtain a training dataset with a specific amount of resources Number() represents the number of sample sentences in the dataset. If α∈(0,1), it is a low-resource training scenario. If α=1, it is a regular training scenario. Then, the decoder minimizes the target sequence g under each view I With the output sequence Y I The negative log-likelihood loss function of ; Step S60: In the inference phase, the decoder generates a predicted output sequence Y based on the constraint decoding strategy and the learning template according to the event arguments of different perspectives I , and vote on the event arguments generated from different perspectives. If an event argument is generated under the prompt of each perspective, the final output of the model includes the result of the event argument.

2. According to the method of claim 1, the method is characterized in that: Step 10 includes the following steps: Step S101: the original sentence set sentence is known set The representation of the I-th sentence in is S I =[w1, w2, ... w a ,…,w b ,…,w c ,…,w d ,…,w e ,…,w f ,…,w n ],w a ,…,w b is the event trigger word t of the sentence, and the corresponding event type is e; w c ,…,w d With w e ,…,w f are the two event arguments a1 and a2 of the sentence, and their corresponding argument roles are r1 and r2 respectively. If the argument role set R corresponding to the event type e is known, e = [r1, r2, r3], for the event trigger word, event argument, argument role, special marks [T], [A], [R] are used to indicate the order of appearance, and the element prompt order of the following four perspectives is constructed according to their order of appearance: [T]→[A]→[R], [T]→[R]→[A], [A]→[R]→[T], [R]→[A]→[T]; Constructing a multi-perspective event argument prompt template based on prompt order and shuffled argument role arrangement order in In the template, [SSEP] represents the separator between elements. The event argument prompt template of the k-th perspective corresponding to the I-th sentence, where I represents the sentence number and k represents different perspective numbers ranging from 1 to 4; Step S102: construct a multi-perspective target text based on the order of occurrence of elements of the event argument prompt template: Among them, null means that the event argument corresponding to the argument role r3 does not appear in the sentence. The target text is indexed by the vocabulary to obtain the target sequence in Represents the target text of the k-th perspective of the I-th sentence, where I is the sentence number, k is the different perspective numbers ranging from 1 to 4, and Index(*) is the word-to-index mapping function based on the model vocabulary.

3. According to the method of claim 1, the method is characterized in that: Step S20 includes: Step S201: Sentence S I Combined with the event argument prompt template to form a new sentence representation As the input of the model. Among them, Represents the concatenation operation of adjacent sequences.

4. The method for extracting autoregressive event arguments based on multi-view prompt learning template according to claim 1, characterized in that: Step S30 includes: Constructing sentence semantic information encoder Φ based on autoregressive pre-trained language model s , and design the word vector embedding layer Emb with shared weight parameters for the encoder vocab , let the sentence be S′ I Input to the semantic information encoder Φ s Generate sentence vector representation in Among them, θ e represents the encoder model parameters, n is the length of the original sentence encoding, p is the length of the event argument prompt template encoding, d is an integer, (n+p)*d represents The dimension of k represents the serial number of the event argument prompt template. Encoder(*) is an encoder function that converts the input sequence into a vector representation. Emb vocab (*) is the mapping function from word unit to word vector, which is obtained by encoder Φ s The generated sentence vector representation H′ I It consists of two parts, namely the original sentence S I The corresponding vector represents h i (1≤i≤n), and event argument prompt template The corresponding vector representation 5. The method for extracting autoregressive event arguments based on multi-view prompt learning template according to claim 1, characterized in that: Step 40 includes: Step S401, construct a sequence-to-sequence decoder based on the autoregressive pre-trained language model. The decoder uses H′ I Initialize the hidden state. In order to maintain the consistency of the mapping from the index word list to the shallow word vector feature space, the decoder uses the word vector embedding layer Emb that shares weight parameters with the encoder. vocab , to establish a shared mapping space from the index word table to the word vector feature weights, and then generate word units to obtain the output sequence Where T represents the length of the output sequence, Represents the output sequence Y I The mth word in , where m represents the position in the sequence.

6. The method for extracting autoregressive event arguments based on multi-view prompt learning template according to claim 1, characterized in that: Step 50: Wrapping: Step S501, in the training phase, in the original sentence set sentenCe set Extract the training dataset Train set When , according to the number of resources required by the task scenario, for the training data set Train set Set the sampling coefficient α∈(0, 1] from the training data set Train set Random sampling to obtain a training dataset with a specific amount of resources Number() represents the number of sample sentences in the dataset. If α∈(0,1), it is a low-resource training scenario. If α=1, it is a regular training scenario. Then, the training data is passed into the model to obtain the output sequence Y generated by the model. I ; Step S502, obtaining a target sequence-output sequence pair And calculate its negative log-likelihood loss function: Where g represents the target sequence g I , Y represents the decoder output sequence Y I , represents the target sequence from the k-th perspective of the ,th sentence, shows the decoder output sequence of the k-th view of the sentence, represents the expected operation, M is the length of the target sequence, y ＜m are all the words generated before the mth word, p(Y|g) represents the probability of the model outputting the sequence Y given the target sequence g, and p(y m |g,y ＜m ) represents the word sequence y generated by the model before the mth step in the given target sequence during the model training phase ＜m Under the condition that m probability.

7. The method for extracting autoregressive event arguments based on multi-view prompt learning template according to claim 1, characterized in that: Step 60 includes: Based on the permutation and combination patterns of trigger words, event arguments, and argument roles, a constrained decoding strategy is designed to constrain the model to generate structured sequences, and the structural information of different element permutations and combinations is introduced into the decoding process of the sequence decoder; The algorithm of the constrained decoding strategy is as follows: In the inference phase, the model dynamically adjusts the generation list of the next word based on the word generated in the current state. If the current word is decoded as "[", the word generated in the next state will be selected from [T], [A], and [R]; If the word unit generated in the current state is [T], [A], then the word unit generated in the next state will be from sentence S I and [SSEP]; If the word generated in the current state is [R], the word generated in the next state will be from the argument role set R e and [SSEP]; Through the constrained decoding strategy, the decoder will first confirm the candidate sequence according to the above rules when maintaining the candidate sequence at each time step using the beam search algorithm in the inference phase.

Images