Method and device for extracting chapter-level event based on prompt learning and generative model

Abstract

The embodiment of the invention discloses a chapter-level event extraction method and device based on prompt learning and a generative model, and the method comprises the steps: carrying out the serialization of a to-be-extracted chapter-level text through the chapter-level event extraction method based on the prompt learning and the generative model, and obtaining a serialized text comprising at least one event set; matching the serialized text with a preset event prompt template to obtain an event sequence; and processing the event sequence by applying a pre-trained event generation model to obtain an event extraction result. Through the mode, event extraction can be accurately carried out, the problem of performance reduction caused by entity overlapping and argument scattering is relieved, and the event extraction method and device are suitable for small sample scenes.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of natural language processing technology, and more particularly to a method and device for extracting chapter-level events based on prompt learning and generative models. Background Art

[0002] Event extraction is one of the three major subtasks of information extraction in natural language processing, aiming to convert unstructured text into structured text. It is widely used in intelligence work in fields such as commerce and the military, and is a key technology for building knowledge graphs. Event extraction tasks are generally divided into sentence-level event extraction and paragraph-level event extraction. Paragraph-level event extraction involves extracting the main components of an event from a paragraph and converting it into structured text. Because paragraph-level event extraction often faces the problem of multiple events within the same paragraph, with event arguments distributed across different sentences, paragraph-level event extraction is more challenging than sentence-level event extraction. However, in the current large amount of text information in fields such as the internet, industrial production, and finance, a single event often requires multiple sentences to describe, necessitating paragraph-level event extraction. Therefore, paragraph-level text event extraction is a meaningful and challenging task.

[0003] The "pre-training + prompt learning" paradigm has achieved remarkable results in various areas of natural language processing. However, applying this paradigm to the task of paragraph-level event extraction still presents some challenges. First, how to model the extracted event list and design a prompt template based on a text generation model. Second, how to select appropriate fine-tuning methods to adapt the model to the task at hand. Paragraph-level event extraction differs from sentence-level event extraction in that it presents two major challenges. First, multiple events in the same document require identifying the affiliation between argument mentions and events. Second, arguments are distributed across different sentences, a problem often referred to as argument scattering. Recent paragraph-level event extraction methods typically address this problem in three steps. The first step is entity extraction, which involves extracting entity mentions from the document. The second step is event detection, which involves extracting events from the document. The third step is populating an event template with the extracted entities, which consists of predefined argument roles for each event.

[0004] For the task of extracting textual events at the paragraph level, existing technologies use sequence labeling methods based on deep neural network models to extract entities, and have proposed a series of solutions to the problem of argument dispersion. However, deep neural network-based methods cannot effectively address entity overlap and nesting, and lack domain adaptability. When the model is applied to a new vertical domain, a large number of samples must be retrained. Summary of the Invention

[0005] In view of the above problems, an embodiment of the present invention provides a method and device for extracting chapter-level events based on prompt learning and generative models, which overcomes the above problems or at least partially solves the above problems.

[0006] According to one aspect of an embodiment of the present invention, a chapter-level event extraction method based on prompt learning and generation model is provided, the method comprising: serializing the chapter-level text to be extracted to obtain serialized text including at least one event set; matching the serialized text with a preset event prompt template to obtain an event sequence; applying a pre-trained event generation model to process the event sequence to obtain an event extraction result.

[0007] Optional,

[0008] The method of serializing the chapter-level text to be extracted to obtain serialized text including at least one event set includes: preprocessing the chapter-level text to be extracted to generate an event list of at least one event, wherein the event list includes event type, event trigger word, event argument and corresponding argument role; constructing a directed acyclic graph according to the event list of at least one event; and serializing the events using a depth-first traversal method according to the directed acyclic graph to obtain serialized text.

[0009] Optionally, matching the serialized text with a preset event prompt template to obtain an event sequence includes: matching the preset event prompt template with arguments related to event type, trigger words and argument roles according to the serialized text to obtain an event sequence.

[0010] Optionally, the application of a pre-trained event generation model to process the event sequence and obtain the event extraction result includes: training the event generation model, and the objective function of the model training is:

[0011]

[0012] Where x' is the output of the event generation model, P(y|x'; θ; φ) is the probability that the output of the model with model parameters θ and φ is y when the input is x', x is the original input text, and h <i is the output of the hidden layer, θ, φ are the model parameters of the event generation model; the training process of the event generation model is to fine-tune the model based on low-rank decomposition to obtain a pre-trained event generation model.

[0013] Optionally, the training process of the event generation model is based on low-rank decomposition to perform model fine-tuning to obtain a pre-trained event generation model, including: during the training process of the event generation model, calculating the hidden vector based on the output of the multi-layer transformer encoder processing in the event generation model and the first low-rank decomposition result, the first low-rank decomposition result being the result of the model input after low-rank decomposition; obtaining the output and decoder state of the event generation model after the hidden vector is processed by the decoder and low-rank processing in the event generation model; calculating the loss function of the event generation model, and adjusting the parameters of the event generation model according to the loss function until the convergence conditions are met; obtaining the parameters of the event generation model that meet the convergence conditions to obtain a pre-trained event generation model.

[0014] Optionally, the output and decoder state of the event generation model are obtained after the latent vector is passed through the decoder and low-rank processing in the event generation model, including: applying the decoder for processing according to the latent vector, the historical state of the decoder and the output of the decoder in the previous step to obtain the current output of the decoder; performing low-rank processing on the latent vector, the historical state of the decoder and the output of the decoder in the previous step to obtain a second low-rank decomposition result; and superimposing the current output of the decoder with the second low-rank decomposition result to obtain the output and decoder state of the event generation model.

[0015] Optionally, the applying a pre-trained event generation model to process the event sequence to obtain an event extraction result also includes: decoding the output using a beam search algorithm to obtain a final output sequence.

[0016] Based on the same inventive concept, a chapter-level event extraction device based on prompt learning and generation model is provided, including: a serialization unit, used to serialize the chapter-level text to be extracted, and obtain a serialized text including at least one event set; a template matching unit, used to match the serialized text with a preset event prompt template to obtain an event sequence; an event extraction unit, used to apply a pre-trained event generation model to process the event sequence and obtain an event extraction result.

[0017] Based on the same inventive concept, an embodiment of the present invention further proposes an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the aforementioned method when executing the program.

[0018] Based on the same inventive concept, an embodiment of the present invention further proposes a computer storage medium, in which at least one executable instruction is stored, and the executable instruction enables a processor to execute the aforementioned method.

[0019] The beneficial effects are: the embodiment of the present invention serializes the chapter-level text to be extracted to obtain a serialized text including at least one event set; matches the serialized text with a preset event prompt template to obtain an event sequence; applies a pre-trained event generation model to process the event sequence to obtain an event extraction result, which can accurately extract events and alleviate the performance degradation problem caused by entity overlap and argument scattering, and is suitable for small sample scenarios.

[0020] The above description is only an overview of the technical solutions of the embodiments of the present invention. In order to more clearly understand the technical means of the embodiments of the present invention, they can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the embodiments of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are specifically listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] Various other advantages and benefits will become apparent to those skilled in the art upon reading the detailed description of the preferred embodiment below. The accompanying drawings are for illustration purposes only and are not to be considered as limiting the present invention. The same reference symbols are used throughout the drawings to represent the same components. In the drawings:

[0022] Figure 1 A flow chart of a method for extracting chapter-level events based on prompt learning and generative modeling provided by an embodiment of the present invention is shown;

[0023] Figure 2 A schematic diagram of a directed acyclic graph according to an embodiment of the present invention is shown;

[0024] Figure 3 A schematic diagram of an event serialization process according to an embodiment of the present invention is shown;

[0025] Figure 4 FIG2 shows a schematic diagram of fine-tuning a model based on Lora according to an embodiment of the present invention;

[0026] Figure 5 An example diagram of the first step of decoding of the beam search algorithm according to an embodiment of the present invention is shown;

[0027] Figure 6 An example diagram of the second step of beam search algorithm decoding according to an embodiment of the present invention is shown;

[0028] Figure 7 An example diagram of the third step of beam search algorithm decoding according to an embodiment of the present invention is shown;

[0029] Figure 8 An overview diagram of a chapter-level event extraction method based on prompt learning and generative model according to an embodiment of the present invention is shown;

[0030] Figure 9 An example diagram showing a method for extracting chapter-level events based on prompt learning and generative model according to an embodiment of the present invention;

[0031] Figure 10 A schematic diagram showing the convergence curves of the chapter-level event extraction method based on prompt learning and generative model according to an embodiment of the present invention on the uee-Fin dataset and the FNDEE dataset;

[0032] Figure 11 A schematic diagram of a convergence curve of training a new vertical domain dataset on a trained model according to an embodiment of the present invention is shown;

[0033] Figure 12 A schematic diagram of the structure of a chapter-level event extraction device based on prompt learning and generative model provided by an embodiment of the present invention is shown;

[0034] Figure 13 A schematic diagram of an electronic device in an embodiment of the present invention is shown. DETAILED DESCRIPTION

[0035] Exemplary embodiments of the present invention will be described in more detail below with reference to the accompanying drawings. Although exemplary embodiments of the present invention are shown in the accompanying drawings, it should be understood that the present invention can be implemented in various forms and should not be limited by the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present invention and to fully convey the scope of the present invention to those skilled in the art.

[0036] Figure 1 The flow chart of the chapter-level event extraction method based on prompt learning and generative model provided by the embodiment of the present invention is shown. Figure 1 As shown in Figure 2, the chapter-level event extraction method based on prompt learning and generative model includes:

[0037] Step S11: Serialize the chapter-level text to be extracted to obtain a serialized text including at least one event set.

[0038] Event extraction is one of the three major information extraction tasks in natural language processing, aiming to transform structured text into unstructured text. An event, as a form of information, is defined as an objective fact involving the interaction of specific people or objects at a specific event and location. The components of an event typically include four elements: event type, trigger, argument role, and argument. Taking the Duee-Fin dataset for text-level event extraction as an example, Duee-Fin defines 13 event types. Event triggers are the core words that indicate the occurrence of an event, typically verbs or nouns. Event arguments are participants in an event and primarily consist of entities, values, and time. Values are non-physical event participants, such as job titles. Argument roles are the roles played by event arguments within the event, with 35 types of roles, such as attacker and victim.

[0039] For chapter-level event extraction, there is often a paragraph of text containing multiple events.

[0040] In an embodiment of the present invention, optionally, the chapter-level text to be extracted is preprocessed to generate an event list of at least one event, wherein the event list includes event type, event trigger words, event arguments and corresponding argument roles; a directed acyclic graph is constructed based on the event list of at least one event; and the events are serialized using a depth-first traversal method based on the directed acyclic graph to obtain serialized text.

[0041] The input and output of the subsequent event generation model are in the form of sequences, so the output text needs to be adjusted to the sequence form. Considering that in the chapter-level event extraction task, the output of the event generation model is multiple events, and each event contains multiple elements, there are complex associations between the semantics of words, which cannot be fully represented by simply using a tree or hierarchical segmentation. A directed acyclic graph is a descriptive form that can contain richer information. Therefore, the embodiment of the present invention first uses a directed acyclic graph to represent the event record table, and then uses a depth-first traversal method to serialize it. The specific representation method is as follows. Figure 2 As shown, the dotted lines represent event relationships.

[0042] For example, the following passage contains two events: "pledge" and "unpledge." On July 31st, Wabei.com reported that Shuangta Food (002481) announced that its chairman, Yang Junmin, had released 15 million shares pledged ahead of schedule on July 29, 2019. The unpledged shares represent 8,057% of his total shareholding. As of the date of this announcement, the chairman held 175,060,633 shares, representing 13.86% of the company's total shares. Following this pledge, the number of shares remaining pledged is 71,335,200, representing 40.75% of his total shares and 5.74% of the company's total share capital.

[0043] To describe an "unpledge" event, there are seven elements: the pledger, the number of pledged stocks / shares, the event time, the pledged assets, and the disclosure time. These seven elements are called argument roles, and the instances of these elements in a specific event are the arguments mentioned above. The event serialization process based on a directed acyclic graph is as follows: Figure 3 shown.

[0044] Step S12: Match the serialized text with a preset event prompt template to obtain an event sequence.

[0045] Optionally, a preset event prompt template is matched with arguments related to event type, trigger words and argument roles according to the serialized text to obtain an event sequence.

[0046] Hint learning is a learning method that evolved from fine-tuning. In the fine-tuning framework, the pre-trained parameters φ are initialized. Here is a trainable language model distribution, performing gradient updates on the following log-likelihood objective:

[0047] h i =LM φ (z i ,h <i )

[0048]

[0049] Where x is the input, y is the output, z is the concatenation of x and y, and h i is the output of the hidden layer and is used to calculate the distribution of the next token: W φ It is the pre-training parameter matrix, and token refers to the smallest element in the chapter-level text to be extracted.

[0050] Hint learning involves adding "hint information" to the input, transforming the downstream task into a text generation task, without significantly changing the pre-trained language model's structure and parameters. Its formal definition involves using a template to modify the original input x into a text string prompt x', which contains some unfilled slots. The language model is then used to probabilistically fill in the unfilled information, resulting in the final string x, from which the final output y can be derived.

[0051] Step S13: Apply the pre-trained event generation model to process the event sequence to obtain event extraction results.

[0052] In the embodiment of the present invention, before step S13, the event generation model is trained. The event generation model is modeled using an encoder-decoder architecture. φ (y|x), where x is encoded by the bidirectional encoder and the decoder autoregressively predicts y (conditioned on the encoded x and its left context). Using the same index and activation signs, h i For all i∈Xidxis is calculated by the bidirectional Transformer encoder; for all i∈Yidx, h i By using the same equation h i =LM φ (z i ,h <i ) is calculated using the autoregressive decoder.

[0053] Let Prefix = "event extraction:", then x' = ["event extraction:":x], where x is the original input text and x' is the output after the template function is constructed. The objective function of the model training is:

[0054]

[0055] Where P(y|x'; θ; φ) is the probability that the output of the model with model parameters θ and φ is y when the input is x', x' is the output of the model generated by the event, x is the original input text, and h <i is the output of the hidden layer, h <i represents h1,h2…,h i-1 .

[0056] The training process of the event generation model is to fine-tune the model based on low-rank decomposition to obtain a pre-trained event generation model. The event generation model fine-tunes model parameters based on Lora to adapt to the multi-event extraction method at the chapter level.

[0057] In step S13, optionally, during the training of the event generation model, Figure 4As shown, first, a hidden vector is calculated based on the output processed by the multi-layer transformer encoder in the event generation model and the first low-rank decomposition (LoRA) result. The first low-rank decomposition result is the result after the model input is subjected to low-rank decomposition.

[0058] H = Encoder(x1, ……, x |x| ) + BA(x1, ……, x |x| )

[0059] Among them, the weight matrix is W0 ∈ R d×k , B ∈ R d×r , A ∈ R r×k And r << min(d, k). Each layer of Encoder(·) is a transformer block with a multi-head attention mechanism. After encoding the input token sequence, the decoder uses the hidden vector of the sequential input tokens to predict the output structure token by token.

[0060] Then, based on the hidden vector, the output of the event generation model and the decoder state are obtained through the decoder and low-rank processing in the event generation model. Optionally, continue to refer to Figure 4 , and the decoder is applied for processing according to the hidden vector, the decoder historical state, and the decoder output of the previous step to obtain the current output of the decoder; low-rank processing is performed on the hidden vector, the decoder historical state, and the decoder output of the previous step to obtain the second low-rank decomposition result; the current output of the decoder is superimposed with the second low-rank decomposition result to obtain the output of the event generation model and the decoder state.

[0061]

[0062] Among them, y i is the decoder output, is the decoder state, is the decoder historical state, y i-1 is the decoder output of the previous step, is the second low-rank decomposition result. Each layer of the decoder Decoder(·) is a transformer block, which includes the self-attention of the decoder state and the cross-attention of the encoder state H. The self-attention decoder predicts the i-th token y i in a linearized form.

[0063] Finally, the loss function of the event generation model is calculated, and the parameters of the event generation model are adjusted according to the loss function until the convergence condition is satisfied; the parameters of the event generation model that satisfy the convergence condition are obtained to get the pre-trained event generation model. The loss function of the event generation model is:

[0064]

[0065] Among them, X is the event sequence input to the event generation model, y c Generate m outputs of the model for the event.

[0066] In the embodiment of the present invention, the output of the event generation model is a sequence consisting of arguments of all events extracted from the chapter-level text to be extracted.

[0067] In this embodiment of the present invention, the event generation model uses a beam search algorithm to decode the output and obtain the final output sequence. The specific algorithm is as follows:

[0068]

[0069] The output of the event generation model is:

[0070]

[0071] Among them, y <i =y1,…,y i-1 ,P(y i |y <i ,x) is the probability of the target vocabulary V normalized by softmax(·).

[0072] The following example illustrates how to use a beam search algorithm to decode the final output sequence.

[0073] See also Figure 5 , in the initial time step, A and C become the best choices, resulting in two results: [A] and [C]. Therefore, the remaining three options will be discarded.

[0074] See also Figure 6 The second step is to continue the generation process based on the two previous results. In the branch corresponding to option A, five candidate solutions were generated: [AA], [AB], [AC], [AD], and [AE], while the same number of candidate solutions were generated in the branch corresponding to option C. At this point, the model uniformly ranks all ten candidate solutions and retains the two best solutions: [AB] and [CE].

[0075] See also Figure 7 The third step is the same. The best two words will be retained from the new 10 candidate words, and finally the two results [ABD] and [CED] will be obtained.

[0076] The beam search algorithm has time and space complexity that depends on the beam size K and the length of the input sequence n. Regarding the algorithm's time complexity, at each time step, for each hypothesis being expanded, a score must be calculated and the top K hypotheses selected. Therefore, the computational complexity per time step is O(K). The total time complexity is O(nK).

[0077] In the embodiment of the present invention, Figure 8 As shown in the figure, we first serialize events based on a directed acyclic graph, then formalize the chapter-level event extraction task into a sequence-to-sequence task based on the prompt template, and then model the task through a generative model. We complete the chapter-level event extraction task through autoregressive decoding, and perform Lora fine-tuning during the process to finally obtain the event extraction result. For example, see Figure 9 The aforementioned chapter-level text to be extracted, including the two events of "pledge" and "unpledge", obtains the corresponding event extraction results through the chapter-level event extraction method based on prompt learning and generative model in an embodiment of the present invention.

[0078] The following experiments are conducted on the chapter-level event extraction method based on prompt learning and generative model of an embodiment of the present invention. The Chinese chapter-level event extraction dataset DuEE-Fin in the financial field is used to test the domain transfer ability of the method; and the dataset NFDEE in the military field is used for few-sample and zero-sample tests.

[0079] DuEE-Fin is Baidu's latest dataset for extracting chapter-level events from the financial sector. It contains 11,700 chapters across 13 categories, including some non-target chapters as negative examples. The chapters in the dataset are from news and announcements in the financial sector, covering many difficult problems in real-world application scenarios. Since the released dataset does not include annotations for the test set, the results of the development set and online testing are compared. FNDEE contains nine event types and approximately 17,000 texts containing event information (each text may contain multiple events). The data is sourced from public military news websites such as Sina Military, China Military Network, and Huanqiu.com. The annotations include event mentions (trigger words, event types, and event elements) and a list of coreference arguments. The nine event types and their corresponding event elements are shown in Table 1. The event trigger words and event types, event arguments, and argument roles are the expected outputs. Since the test set annotations for this dataset are also not released, we only compare the test results on the development set. An analysis of the distributions of the two datasets above revealed that both datasets follow a long-tail distribution, with the evaluation and test sets maintaining the same distribution. No data balancing was performed during training. For predicted events of a specific event type, the most similar real event of the same event type was selected without replacement. The micro-averaged role-level precision (P), recall (R), and F1 score were then calculated for the predicted event and the selected golden event.

[0080] In the Doc2Event experiment, we used the mT5 and T5-PEGASUS models. mT5 is a multilingual model released by Google in 2021. It uses a corpus covering 101 languages for self-supervised training. It is a text-to-text generative model. We trained the model using Adam (Kingma and Ba, 2015) as the optimizer for 80 epochs, with learning rates set at 5e-05 and 5e-06, respectively, weight_decay = 0.01, and a batch size of 1.

[0081] Several existing basic methods are applied for comparison with the method of the embodiment of the present invention. The basic methods include Doc2EDAG, GIT, PTPCG and REDEE. Among them, Doc2EDAG uses a transformer encoder to obtain sentence and entity embeddings, and then uses another transformer to fuse cross-sentence contexts and extract multiple events at the same time. GIT replaces the Transformer-based document-level encoder in Doc2EDAG with a GCN-based encoder, and establishes a memory-based tracker to simulate the interaction between different events during decoding. PTPCG designs a new event parameter combination strategy through a pruned complete graph non-autoregressive decoding algorithm, which is constructed under the guidance of automatically selected pseudo triggers. REDEE is a chapter-level event extraction framework based on a customized Transformer proposed for the argument scattering and multi-event problems in chapter-level event extraction, which can be expanded to capture multi-scale and multi-amount parameter relationships.

[0082] First, a comparative analysis is performed with the basic method on the Duee-Fin dataset and FNDEE dataset. Figure 10 Schematic diagram of the convergence curve on Duee-Fin dataset and FNDEE dataset. Figure 10 a and Figure 10 b are the average error and F1 score trained on the NFDEE dataset; Figure 10 c and Figure 10 d are the average error and F1 score of training on Duee-Fin dataset. Figure 10 As shown in the figure, when training on the NFDEE dataset and the Duee-Fin dataset, the model can converge within 30 epochs.

[0083]

[0084]

[0085] Table 1 Experimental comparison between DuEE-Fin dataset and basic methods

[0086] Table 2 Online test results of DuEE-Fin dataset

[0087]

[0088] As shown in Table 1 and Table 2, Dev represents the experimental results of the evaluation set, Online test is the online test result, F1(S.) is the F1 score of a single sentence, and F1(M.) is the F1 score of an event whose arguments are distributed in multiple sentences. T5-base(full-tuning)The method (Doc2Event) and Doc2Event of the embodiment of the present invention for full-tuning are shown. T5-base(lora-tuning) This represents the Lora-based tuning method (Doc2Event) of an embodiment of the present invention. On the DuEE-Fin dataset, a classic dataset in the financial field, Doc2Event performed well, outperforming the SOTA algorithm by 3.5% in the evaluation set and 6% in the online test. Furthermore, the results show that Doc2Event can effectively improve performance (6.7%) in event scenarios where arguments are distributed across multiple sentences.

[0089] Table 3 Experimental results of NFDEE dataset

[0090]

[0091] As shown in Table 3, on the latest NFDEE dataset, Doc2Event still achieves very good results compared with other chapter-level event extraction datasets. The experimental results show that the F1 value exceeds the SOTA algorithm by 16.7%.

[0092] The above results show that on the datasets in the financial and military fields, the chapter-level multi-event extraction task is converted into a sequence-to-sequence generation task in the form of a directed acyclic graph. By modeling the encoder-decoder architecture model in the embodiment of the present invention, the empirical knowledge accumulated by the generative model in the pre-training stage and the long document modeling ability can be effectively utilized to solve the problems in the chapter-level multi-event extraction task.

[0093] In order to verify the domain transfer capability of this method, the present invention also set up experiments using a model trained on the Duee-FIN dataset with 50%, 25%, and 10% of the sample size in the NFDEE dataset, and compared and analyzed the performance indicators and model convergence speed. Figure 11 As shown, Figure 11 a and Figure 11 b represents the mean error and F1 score, respectively. Using a new vertical domain dataset on the trained model, the model converges in about 10 epochs, effectively reducing the number of training batches and shortening the training time.

[0094] Table 4. Experimental results of domain adaptation on NFDEE dataset

[0095]

[0096] As shown in Table 4, NFDEE (Dev) w / o is the result obtained by training directly on the native mt5 model under this framework, and NFDEE (Dev) w is the experimental result of training on the Duee-fin dataset and then using 100%, 50%, 25%, and 10% of the total samples of the NFDEE dataset (a total of 8,000). The experimental results show that with a sample size of 10%, an F1 score of 85% of the direct training of the original model can be achieved. The above results show that after adapting to the task mode proposed in the embodiment of the present invention, the attenuation of the F1 value caused by the reduction in the number of samples can be effectively reduced. This shows that by adding "prompt information" to the input and matching the task mode of the generated model, task adaptation can be performed through cross-domain training, so that the method of the embodiment of the present invention can solve scenarios with a small number of samples.

[0097] This embodiment of the present invention also uses mT5-base as the basic model. Experiments are conducted on the Duee-Fin and FNDEE datasets for four fine-tuning schemes: full-tuning, p-tuning, prefix-tuning, and Lora-based tuning. Performance indicators are compared and analyzed. The performance, fine-tuning parameter count, and video memory usage of different fine-tuning schemes are compared and analyzed on the DuEE-Fin dataset.

[0098] Table 5 Experimental results of DuEE-Fin dataset (comparison of different fine-tuning methods)

[0099]

[0100]

[0101] Table 6 Experimental results of NFDEE dataset (comparison of different fine-tuning methods)

[0102]

[0103] As shown in Table 5 and Table 6. In the method proposed in the embodiment of the present invention, compared with full-tuning, p-tuning, and prefix-tuning, using LoRa can achieve better results in fine-tuning.

[0104] Table 7 Comparison of parameter amount, video memory and running time under different fine-tuning methods

[0105]

[0106] It can be seen that in terms of video memory usage, Lora has no advantage over Prefix-tuning and P-tuning.

[0107] The embodiment of the present invention also conducted experiments on the Duee-Fin dataset using two fine-tuning methods: full-tuning and Lora-based tuning, and conducted comparative analysis on models of different sizes.

[0108] Table 8 Experimental results of DuEE-Fin dataset (model size comparison based on full-tuning)

[0109]

[0110]

[0111] Table 9 Experimental results of DuEE-Fin dataset (model size comparison based on lora)

[0112]

[0113] The experimental results show that both LoRa and full parameter fine-tuning methods can achieve better experimental results on larger models, but as the number of model parameters increases, the gain brought by LoRa fine-tuning decreases. This is because the size of the model affects the prior knowledge stored in the pre-trained language model, which further illustrates that the performance of the method in the embodiment of the present invention effectively utilizes the prior knowledge of the pre-trained language model and can bring better performance when the pre-trained language model has richer knowledge.

[0114] In summary, the chapter-level event extraction method based on prompt learning and generation model in an embodiment of the present invention serializes the chapter-level text to be extracted to obtain a serialized text including at least one event set; matches the serialized text with a preset event prompt template to obtain an event sequence; applies a pre-trained event generation model to process the event sequence to obtain an event extraction result. It can accurately extract events, alleviates the performance degradation problem caused by entity overlap and argument scattering, and is suitable for small sample scenarios.

[0115] The foregoing description is of specific embodiments of the present invention. In some cases, the actions or steps described in the embodiments of the present invention may be performed in an order different from that shown in the embodiments and still achieve the desired results. In addition, the processes depicted in the accompanying drawings do not necessarily require the specific order or sequential order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0116] Based on the same concept, the embodiment of the present invention also provides a chapter-level event extraction device based on prompt learning and generation model. Applied to the server. Figure 12As shown in Figure 1, the chapter-level event extraction based on prompt learning and generative model includes: serialization unit, template matching unit and event extraction unit.

[0117] A serialization unit, configured to serialize the chapter-level text to be extracted, and obtain serialized text including at least one event set;

[0118] A template matching unit, configured to match the serialized text with a preset event prompt template to obtain an event sequence;

[0119] The event extraction unit is used to apply the pre-trained event generation model to process the event sequence and obtain event extraction results.

[0120] For the convenience of description, the above devices are described as being divided into various modules according to their functions. Of course, when implementing the embodiments of the present invention, the functions of each module can be implemented in the same or multiple software and / or hardware.

[0121] The apparatus of the above embodiment is applied to the corresponding method of the above embodiment and has the beneficial effects of the corresponding method embodiment, which will not be described in detail here.

[0122] Based on the same inventive concept, an embodiment of the present invention further provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein when the processor executes the program, the method described in any one of the above embodiments is implemented.

[0123] An embodiment of the present invention provides a non-volatile computer storage medium, wherein the computer storage medium stores at least one executable instruction, and the computer executable instruction can execute the method described in any one of the above embodiments.

[0124] Figure 13 13 shows a more specific hardware structure diagram of an electronic device provided in this embodiment. The device may include: a processor 1301, a memory 1302, an input / output interface 1303, a communication interface 1304, and a bus 1305. The processor 1301, the memory 1302, the input / output interface 1303, and the communication interface 1304 are communicatively connected to each other within the device via the bus 1305.

[0125] The processor 1301 can be implemented as a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided by the method embodiments of the present invention.

[0126] The memory 1302 can be implemented in the form of ROM (Read Only Memory), RAM (Random Access Memory), static storage devices, dynamic storage devices, etc. The memory 1302 can store an operating system and other application programs. When the technical solutions provided by the method embodiments of the present invention are implemented through software or firmware, the relevant program codes are stored in the memory 1302 and called and executed by the processor 1301.

[0127] The input / output interface 1303 is used to connect input / output modules to implement information input and output. The input / output modules can be configured as components within the device (not shown in the figure) or can be externally connected to the device to provide corresponding functions. Input devices may include a keyboard, mouse, touch screen, microphone, various sensors, etc., and output devices may include a display, speaker, vibrator, indicator light, etc.

[0128] The communication interface 1304 is used to connect to a communication module (not shown) to enable communication between the device and other devices. The communication module can communicate via a wired method (such as USB, network cable, etc.) or a wireless method (such as mobile network, WIFI, Bluetooth, etc.).

[0129] The bus 1305 comprises a pathway for transmitting information between the various components of the device (eg, the processor 1301 , the memory 1302 , the input / output interface 1303 , and the communication interface 1304 ).

[0130] It should be noted that although the above device only shows the processor 1301, the memory 1302, the input / output interface 1303, the communication interface 1304, and the bus 1305, in a specific implementation, the device may also include other components necessary for normal operation. In addition, those skilled in the art will understand that the above device may only include the components necessary to implement the embodiments of the present invention, and does not necessarily include all the components shown in the figure.

[0131] Those skilled in the art should understand that the discussion of any of the above embodiments is merely illustrative and is not intended to imply that the scope of the present disclosure is limited to these examples. Based on the concept of the present disclosure, the technical features in the above embodiments or different embodiments may be combined, the steps may be implemented in any order, and there are many other variations of the different aspects of the embodiments of the present invention as described above, which are not provided in detail for the sake of simplicity.

[0132] This application is intended to cover all such substitutions, modifications, and variations that fall within the broad scope of all embodiments. Therefore, any omissions, modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the embodiments of the present invention should be included in the scope of protection of this disclosure.

Claims

1. A chapter-level event extraction method based on prompt learning and generative model, characterized by: The chapter-level event extraction method includes: Serializing the chapter-level text to be extracted to obtain a serialized text including at least one event set; Matching the serialized text with a preset event prompt template to obtain an event sequence; The pre-trained event generation model is applied to process the event sequence to obtain an event extraction result.

2. The method according to claim 1, characterized in that The step of serializing the chapter-level text to be extracted to obtain the serialized text including at least one event set includes: Preprocessing the chapter-level text to be extracted to generate an event list of at least one event, wherein the event list includes an event type, an event trigger word, an event argument, and a corresponding argument role; constructing a directed acyclic graph from the event list of at least one event; The events are serialized using a depth-first traversal method according to the directed acyclic graph to obtain serialized text.

3. The method according to claim 2, characterized in that The matching of the serialized text with a preset event prompt template to obtain an event sequence includes: According to the serialized text, a preset event prompt template is matched with arguments related to event type, trigger words and argument roles to obtain an event sequence.

4. The method according to claim 3, characterized in that The application of the pre-trained event generation model to process the event sequence and obtain the event extraction result includes: The event generation model is trained, and the objective function of the model training is: Where x' is the output of the event generation model, P(y|x'; θ; φ) is the probability that the output of the model with model parameters θ and φ is y when the input is x', x is the original input text, and h <i is the output of the hidden layer, θ, φ are the model parameters of the event generation model; The training process of the event generation model is to fine-tune the model based on low-rank decomposition to obtain a pre-trained event generation model.

5. The method according to claim 4, characterized in that The training process of the event generation model is to fine-tune the model based on low-rank decomposition to obtain a pre-trained event generation model, including: During the training process of the event generation model, a hidden vector is calculated based on the output of the multi-layer transformer encoder processing in the event generation model and a first low-rank decomposition result, where the first low-rank decomposition result is a result of the low-rank decomposition of the model input; Obtaining the output and decoder state of the event generation model after passing the latent vector through a decoder and low-rank processing in the event generation model; Calculating a loss function of the event generation model, and adjusting parameters of the event generation model according to the loss function until a convergence condition is met; Parameters of the event generation model that meet the convergence conditions are obtained to obtain a pre-trained event generation model.

6. The method according to claim 5, characterized in that The step of obtaining the output of the event generation model and the decoder state after passing the latent vector through a decoder and low-rank processing in the event generation model includes: Applying the decoder to perform processing based on the hidden vector, the decoder's historical state, and the decoder output of the previous step to obtain a current output of the decoder; Performing low-rank processing on the hidden vector, the decoder history state, and the decoder output of the previous step to obtain a second low-rank decomposition result; The current output of the decoder is superimposed with the second low-rank decomposition result to obtain the output of the event generation model and the decoder state.

7. The method according to claim 1, characterized in that The applying a pre-trained event generation model to process the event sequence to obtain an event extraction result further includes: The output is decoded using a beam search algorithm to obtain the final output sequence.

8. A chapter-level event extraction device based on prompt learning and generative model, characterized by: The device comprises: A serialization unit, configured to serialize the chapter-level text to be extracted, and obtain serialized text including at least one event set; A template matching unit, configured to match the serialized text with a preset event prompt template to obtain an event sequence; The event extraction unit is used to apply the pre-trained event generation model to process the event sequence and obtain event extraction results.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method according to any one of claims 1 to 7 is implemented.

10. A computer storage medium, characterized in that: The storage medium stores at least one executable instruction, and the executable instruction enables the processor to execute the method according to any one of claims 1 to 7.

Images