Event element extraction method, device, electronic device and storage medium based on semantic analysis and prompt learning

Abstract

The present invention discloses a method, apparatus, device, and storage medium for event element extraction based on semantics-based prompt learning. The method performs linguistic analysis on event elements to form a bidirectional conversion module between traditional event element labels and event element semantic labels. An event element classification model is constructed and trained using an event element dataset based on event element semantic labels. Event text is input into a prompt learning event element summary model in conjunction with an event prompt template to extract a summary text containing the event elements. The summary text is then decoded in conjunction with the bidirectional conversion module of the event element classification model to complete the extraction of event elements. By combining the linguistic characteristics of event elements and utilizing prompt learning to mine potential knowledge from a pre-trained language model, the present invention improves the accuracy of event element extraction in the case of few samples.

Description

Technical Field

[0001] The present invention relates to the field of natural language processing in the field of information technology, and in particular to technical fields such as deep learning and information extraction. Specifically, a method, device, electronic device and storage medium for extracting event elements based on semantic analysis and prompt learning are provided. Background Art

[0002] With the rapid development of information technology, the internet generates massive amounts of text data daily, much of which is unstructured and difficult to directly utilize. As an important form of information representation, events describe the interactions between certain people or objects at a specific time and place, helping us understand the world. Event feature extraction is the task of extracting key information such as time, location, people, impact, and cause from unstructured text containing event information, making it easier for computers to process and more efficient for users to access information. Event feature extraction is the foundation for the automatic construction of event graphs and has been widely used in fields such as finance, intelligence, and journalism, offering broad research and application prospects.

[0003] However, event element extraction also faces significant challenges. The first challenge is the difficulty of acquiring data. Existing public datasets are relatively few, and event element extraction is characterized by high annotation difficulty, high professional knowledge requirements, and high manual annotation costs, further increasing the difficulty of data acquisition. The second challenge is the difficulty of domain migration. Different fields focus on different event types, and the event elements of different event types also vary. For example, a visit event in the political field may include five event elements: time, place, person, location, and purpose of the visit, while a bankruptcy event in the financial field focuses on three event elements: time, company, and reason for bankruptcy. These two event elements do not completely correspond, and the corresponding syntactic dependency structures also differ significantly, making algorithm migration even more difficult. Therefore, how to use a small number of labeled samples to develop an event element extraction model with good domain migration is a key issue that needs to be addressed.

[0004] In recent years, prompt learning has emerged as a new paradigm in natural language processing, achieving excellent results in tasks such as text classification, knowledge mining, machine translation, and information extraction with only a small number of labeled samples. Prompt learning adds prompt text to the input text, allowing the model to generate the corresponding answer text in a cloze format. After decoding, the target result is obtained. In addition, the event element extraction task has significant linguistic characteristics. Event elements have similar semantic features and specific syntactic dependencies between event elements, providing a linguistic foundation for prompt learning. Combining prompt learning with the linguistic characteristics of event element extraction can fully tap into the implicit knowledge of pre-trained language models and effectively address the difficulties of data acquisition and domain transfer for event element extraction. Summary of the Invention

[0005] The present invention provides an event element extraction method, device, electronic device, and storage medium based on linguistic analysis and prompt learning. A linguistic analysis is performed on event elements to form a bidirectional conversion module from traditional event element labels to event element semantic labels; an event element classification model is constructed and trained using an event element dataset based on event element semantic labels; the event text is input into a prompt learning event element summary model in conjunction with an event prompt template to extract a summary text containing the event elements; the summary text is then decoded in conjunction with the bidirectional conversion module of the event element classification model to complete the extraction of event elements. By combining the linguistic characteristics of event elements and utilizing prompt learning to mine potential knowledge from a pre-trained language model, the present invention improves the accuracy of event element extraction in the case of few samples.

[0006] The event element extraction method based on linguistic analysis and prompt learning described in the present invention is carried out according to the following steps:

[0007] a. Based on linguistic analysis, a two-way label conversion is formed between traditional event element labels and event element semantic labels;

[0008] b. Constructing an event element semantic dataset: First, obtain an event element extraction dataset based on traditional event element labels, and re-label the dataset using the label conversion method in step a to form an event element semantic dataset;

[0009] c. Constructing an event element semantic classification model: This model includes an event text encoding model and an event element classification model to implement semantic classification of event elements;

[0010] d. For each event type, construct an event prompt template according to the natural language expression method;

[0011] e. Build a prompt learning event element summary model: take the event prompt template and event text generated in step d as model input, and output a natural language sequence of event elements;

[0012] f. Event element summary text decoding: First, the event element concatenation text output in step e is segmented and pre-filtered to form a group of event elements, and the event element semantic classification model in step c is used to semantically classify the group of event elements;

[0013] g. Use the bidirectional label conversion method of traditional event element label-event element semantic label in step a to transform the event element semantic classification result in step f, so as to realize event element extraction based on the original event annotation rules.

[0014] The bidirectional label conversion of traditional event element labels and event element semantic labels described in step a is specifically as follows: the linguistic analysis means are event element semantic analysis and event element dependency analysis, and the event element types of all event types are classified to form cause, time, place, subject, object, result and event element semantic labels that are weakly related to the event type. The bidirectional mapping relationship between the original traditional event element annotation label and the new event element semantic label is the bidirectional label conversion of traditional event element label and event element semantic label.

[0015] The event element semantic dataset described in step b is a dataset labeled using event element semantic labeling rules. The event element semantic labels are relabeled from the labels of the event element extraction dataset based on traditional event element labels. The relabeling uses a bidirectional label conversion of traditional event element labels and event element semantic labels to map the original traditional event element extraction labels.

[0016] The event element semantic classification model described in step c is composed of an event text encoding model and an event element classification model connected in series and trained together. The event text encoding model is a pre-trained language model with the classification layer removed. The event element classification model includes an event element feature conversion module and an event element classification module. The event element feature conversion module first performs spatial conversion of event element features, and then the event element classification module outputs the event element classification results.

[0017] The event prompt template described in step d refers to the cloze pre-training task of the pre-trained language model. It is based on the natural language expression and the event element dependency structure, and consists of prompt text and blank prompts. The number of blank prompts is consistent with the number of event element types, and each blank prompt corresponds to an event element.

[0018] The prompt learning event element summary model described in step e is a pre-trained language model. The event text is encoded and decoded by the pre-trained language model. The event prompt template can be input in series with the event text or input from the pre-trained language model decoder. The model output is a continuous and natural event element summary text composed of event elements and other texts concatenated in sequence.

[0019] The event element summary text decoding described in step f, wherein the pre-filtering includes stop word filtering, part of speech filtering, and trigger word filtering; the semantic classification method is to pre-process the event element summary text according to the requirements of the event element semantic classification model, and then send it into the event element semantic classification model for classification prediction, and the predicted event element semantic label is the semantic classification result of the event element.

[0020] An event element extraction device based on linguistic analysis and prompt learning, the device is composed of an event prompt template module, a prompt learning event element summary module and an event element decoding module, wherein

[0021] Event prompt template module: used to provide a corresponding event prompt template for each event type. Based on the natural language expression and event element dependency structure, a corresponding event prompt template is provided for each event type to prompt the pre-trained language model to generate event elements.

[0022] The prompt learning event element summary module is used to summarize the event elements corresponding to the event type from the event text and output the natural language text of the event elements. It takes the event prompt template output by the event element prompt template module and the event text as input, and uses the text generation ability acquired by the pre-trained language model in the pre-training stage to summarize the event elements corresponding to the event type from the event text and output the natural language text of the event elements.

[0023] Event element decoding module: used to decode event elements from the event element concatenation text output by the prompt learning event element summary module, including event summary word segmentation unit, event element classification unit, and event element conversion unit. The event summary word segmentation unit is used to segment and pre-filter the event element concatenation text output by the prompt learning event element summary module, and then the event element classification unit is used to classify the event elements output by the event summary word segmentation unit based on event semantic labels. Finally, the event element conversion unit is used to convert the event semantic labels of the event elements into traditional event element labels, thereby realizing event element extraction based on traditional event element labels.

[0024] An electronic device comprises at least one processor; at least one GPU computing card; and a memory communicatively connected to the processor; wherein: the memory stores instructions executed by at least one processor, and the instructions are executed by at least one processor or at least one GPU computing card to enable the at least one processor or at least one GPU computing card to execute the described method.

[0025] A non-transitory computer-readable storage medium storing computer instructions for causing the computer to execute the method.

[0026] A processor; at least one GPU computing card; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor or the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to execute any one of the methods described in the examples of the present application.

[0027] The technology of this application can complete the task of event element extraction in a small number of sample scenarios and improve the accuracy of event element extraction in a small number of sample scenarios.

[0028] Other features and advantages of the present invention will be described in the following description and, in part, will become apparent from the description or be learned through practice of the present invention. The objectives and other advantages of the present invention may be achieved and obtained through the structures particularly pointed out in the written description and accompanying drawings. The technical solutions of the present invention are further described in detail below with reference to the accompanying drawings and examples. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Figure 1 Flowchart of the event element extraction method based on linguistic analysis and prompt learning provided by the present invention;

[0030] Figure 2 A flowchart of a bidirectional label conversion method for forming traditional event element labels and event element semantic labels based on linguistic analysis provided by the present invention;

[0031] Figure 3 Flowchart of the event element semantics dataset method provided by the present invention;

[0032] Figure 4 A flowchart of a method for semantic classification of event elements provided by the present invention;

[0033] Figure 5 A structural diagram of an event element semantic classification model provided by the present invention;

[0034] Figure 6 A prediction flow chart of an event element semantic classification model provided by the present invention;

[0035] Figure 7 A structural diagram of a method for summarizing elements of learning events provided by the present invention;

[0036] Figure 8 A structural diagram of a learning event element summary model provided by the present invention;

[0037] Figure 9 A structural diagram of a learning event element summary model provided by the present invention;

[0038] Figure 10 A flowchart of decoding an event element summary text provided by the present invention;

[0039] Figure 11 A structural diagram of a device for extracting elements from a prompt learning event provided by the present invention;

[0040] Figure 121 is a block diagram of an electronic device of the present invention. DETAILED DESCRIPTION

[0041] To more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention is further described in detail below with reference to the accompanying drawings. Various details of the embodiments of the present application are included to facilitate understanding, and they should be considered as merely exemplary. Therefore, it should be appreciated by those skilled in the art that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present application. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description. Example

[0042] The event element extraction method based on linguistic analysis and prompt learning described in the present invention is carried out according to the following steps:

[0043] a. Based on linguistic analysis, a bidirectional label conversion is formed from traditional event element labels to event element semantic labels. Specifically, the linguistic analysis methods are event element semantic analysis and element dependency analysis. The event element types of all event types are classified to form event element semantic labels that are weakly related to the event type, such as cause, time, location, subject, object, and result. The mapping relationship between the original traditional event element annotation labels and the new event element semantic labels is the conversion of traditional event element labels to event element semantic labels.

[0044] b. Constructing an event element semantic dataset: first, obtaining an event element extraction dataset based on traditional event element labels, and re-labeling the dataset using the label conversion method in step a, thereby forming an event element semantic dataset; the event element semantic dataset is a dataset labeled using event element semantic labeling rules, and the event element semantic labels are re-labeled from the labels of the event element extraction dataset based on traditional event element labels. The re-labeling uses the traditional event element label-event element semantic label conversion to map the original traditional event element extraction labels;

[0045] c. Construct an event element semantic classification model, which includes an event text encoding model and an event element classification model to achieve semantic classification of event elements; the event element semantic classification model is a series connection of the event text encoding model and the event element classification model and is trained together. The event text encoding model is a pre-trained language model (such as ERNIE, T5, BART) with the classification layer removed. The event element classification model includes an event element feature conversion module and an event element classification module, and first performs spatial conversion of event element features, and then outputs the event element classification result through the event element classification module. The event element feature conversion can effectively retain the implicit knowledge of the pre-trained language model, and the implicit knowledge helps to improve the few-sample effect of event element extraction;

[0046] d. For each event type, construct an event prompt template according to the natural language expression method. The event prompt template refers to the cloze pre-training task (MLM) of the pre-trained language model. Based on the natural language expression method and the dependency structure of event elements, it consists of prompt text and a blank prompt ([MASK]). For example, the prompt template for an assistance event can be designed as time: [MASK], helper [MASK] assistance recipient [MASK] supplies [MASK], where [MASK] is a blank prompt marker (which may be different for different pre-trained language models). The pre-trained language model can generate the corresponding text at this [MASK].

[0047] e. Construct a prompt learning event element summary model, use the event prompt template and event text generated in step d as model input, and output a natural language sequence of event elements in series; the prompt learning event element summary model is a pre-trained language model (such as ERNIE, T5, BART), the event text is encoded and decoded by the pre-trained language model, the event prompt template can be input in series with the event text or input from the pre-trained language model decoder, and the model output is a continuous and natural event element summary text in which event elements and other texts are connected in a certain order. The blank prompts in the event prompt template correspond to the order of event elements. If the event text lacks some event elements, the event element summary text does not contain the event element type text. The other text includes trigger words and necessary punctuation and preposition text to keep the event summary text continuous and natural. The pre-trained language model is usually pre-trained using a cloze pre-training task (MLM). The cloze pre-training task (MLM) is a mainstream pre-training task for pre-trained language models. It replaces part of the text in the input text with [MASK]. After training, the model has the ability to predict the text at [MASK], that is, the cloze ability. By passing the event prompt template to the event text, and the pre-trained language model clozes the blank prompt [MASK], the cloze ability of the pre-trained language model can be brought into play, and the small sample effect of event element extraction can be improved;

[0048] f. decoding the event element summary text, first performing word segmentation and pre-filtering on the event element concatenation text output in step e to form a group of event elements, and semantically classifying the group of event elements using an event element semantic classification model; the event element summary text decoding, wherein the pre-filtering includes stop words, part-of-speech filtering, and trigger word filtering; wherein the event element semantic classification model is the event element semantic classification model, and the semantic classification method is to pre-process the event text or event element summary text according to the requirements of the event element semantic classification model, and then feed the model into the classification prediction, and the predicted event element semantic label is the semantic classification result of the event element;

[0049] g. Use the bidirectional label conversion method of traditional event element label-event element semantic label in step a to transform the event element semantic classification result in step f, so as to realize event element extraction based on the original event annotation rules.

[0050] An event element extraction device based on linguistic analysis and prompt learning, the device is composed of an event prompt template module, a prompt learning event element summary module and an event element decoding module, wherein

[0051] Event prompt template module: used to provide a corresponding event prompt template for each event type to prompt the pre-trained language model to generate event elements. Based on the natural language expression and event element dependency structure, a corresponding event prompt template is provided for each event type to prompt the pre-trained language model to generate event elements, thereby better mining the implicit knowledge of the pre-trained language model. The implicit knowledge of the pre-trained language model is a non-salient knowledge obtained after the pre-trained language model is pre-trained using a large amount of text. It contains knowledge such as objective world knowledge and linguistic knowledge, which helps to improve the small sample effect of event element extraction;

[0052] Prompt Learning Event Element Summarization Module: This module is used to summarize the event elements corresponding to the event type from the event text and output a natural language sequence of the event elements. It takes the event prompt template and event text output by the event element prompt template module as input, and uses the text generation capability acquired by the pre-trained language model in the pre-training phase to summarize the event elements corresponding to the event type from the event text and output a natural language sequence of the event elements. The text generation capability of the pre-trained language model helps improve the small sample effect of event element extraction.

[0053] Event element decoding module: used to decode event elements from the event element concatenation text output by the prompt learning event element summary module. It includes an event summary word segmentation unit, an event element classification unit, and an event element conversion unit. The event summary word segmentation unit is used to segment and pre-filter the event element concatenation text output by the prompt learning event element summary module. The event element classification unit is then used to classify the event elements based on event semantic labels. Finally, the event element conversion unit is used to convert the event semantic labels into traditional event element labels, thereby realizing event element extraction based on traditional event element labels.

[0054] An electronic device, comprising at least one processor; at least one GPU computing card; and a memory communicatively connected to the processor; wherein: the memory stores instructions executed by the at least one processor, and the instructions are executed by the at least one processor or the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to execute the method;

[0055] Figure 1 This is a flow chart of an event element extraction method based on linguistic analysis and prompt learning provided by an embodiment of the present application. This embodiment is applicable to the case of event element extraction with a small number of labeled samples. The method can be executed by a prompt learning event element extraction device, which can be implemented by software and / or hardware. Figure 1 The event element extraction method provided in the embodiment of the present application includes:

[0056] A bidirectional label conversion method based on linguistic analysis to form traditional event element labels and event element semantic labels;

[0057] For the specific method of constructing the bidirectional label conversion method, see Figure 2 , the specific steps include:

[0058] Obtain an event element set containing all event types; illustratively, add the event elements corresponding to the assistance event and the listing event into the event element set to form {assistance time, assister, assisted person, assistance item, listing time, listed company, listing location;

[0059] Semantic analysis and classification of event elements;

[0060] Analysis and classification of event element dependencies;

[0061] A bidirectional label conversion method for traditional event element labels to event element semantic labels is constructed, wherein, based on the classification results of the traditional event element labels formed after linguistic analysis steps b and c, a bidirectional mapping conversion relationship is formed between the traditional event element labels and the classified event element semantic labels. For example, the traditional event element label "assistant" is analyzed and classified as the event element semantic label "agent-subject", and the two labels form a bidirectional mapping conversion relationship;

[0062] Among them, the traditional event element label is that each event type corresponds to a set of event elements, and the event elements of different events are different;

[0063] For example, the event element sets corresponding to the assistance event and the listing event are {assistance time, assister, assisted person, assistance item} and {listing time, listed company, listing location} respectively. Although both assistance time and listing time represent events, they use different labels.

[0064] The linguistic analysis includes semantic analysis of event elements and dependency analysis of event elements;

[0065] The event element semantic analysis is to analyze the semantics of the event element set included in all event types and classify event element types with similar semantics into one category;

[0066] Exemplarily, the assistance time element and the listing time element have similar semantics and are categorized as time elements;

[0067] The event element dependency analysis is to analyze the dependency between event elements of the same event type, and classify event element types with similar dependency in different event types into one category;

[0068] For example, the donor element and the listed company element are generally the subjects in the elements, have similar subject dependency characteristics, and are classified as agent subject elements.

[0069] The event element semantic label is an event element semantic label generated by event element semantic analysis and event element dependency analysis, and is weakly related to the event type. For example, it includes cause, time, place, agent, object, and result.

[0070] To construct an event element semantic dataset, we first obtain an event element extraction dataset and relabel it using a bidirectional label conversion method to form an event element semantic dataset.

[0071] The event element extraction dataset is a dataset that uses a traditional event element extraction and annotation method. The annotation method specifically includes: based on the event type and trigger words of the event text, for a set of event element types corresponding to the event type, the event element text corresponding to each event element type is annotated from the event text in sequence; wherein the event element type is characterized in that the set of event elements for each event type is generally different, but includes event element types of time and location;

[0072] For the specific method of the event element semantic dataset method, see Figure 3 , the specific steps include:

[0073] Obtaining an event element extraction dataset. In one embodiment, a publicly available event element extraction dataset can be first collected, and then a data subset can be selected based on the requirements. In another embodiment, an event text dataset containing the required event type can be first collected, and then the event text can be manually annotated with event elements.

[0074] Among them, the event element extraction dataset is a dataset that adopts the traditional event element extraction and annotation method. The annotation method is as follows: according to the event type and trigger words of the event text, for a set of event element types corresponding to the event type, the event element text corresponding to each event element type is annotated from the event text in turn;

[0075] For example, for the event text "On August 20, China's first batch of emergency food aid arrived in Sri Lanka", its event type is aid event, the trigger word is aid, and the event element types corresponding to the aid event include: time, donor, recipient, and aid items. The corresponding event elements marked in sequence are August 20, China, Sri Lanka, and food in the event text;

[0076] Relabeling the dataset according to a bidirectional label conversion method; the relabeling method uses a bidirectional label conversion method to map and replace event element labels in the event element extraction dataset using traditional event element labels with event element semantic labels;

[0077] Construct an event element semantic classification model, which includes an event text encoding model and an event element classification model to achieve semantic classification of event elements;

[0078] In one embodiment, the specific method of constructing the event element semantic classification model is described in Figure 4 , the specific steps include:

[0079] Build an event element semantic classification model, the model structure diagram is as follows Figure 5 As shown, the event text encoding model uses the parameters of the pre-trained language model, and the event feature classification model is randomly initialized;

[0080] Preprocess the event text of the event element semantics dataset. First, encode the event text into a token sequence. Then, add a special marker [CLS] at the beginning and a special marker [SEP] at the end of the token sequence. If the event text length after adding the markers exceeds the maximum field length allowed by the pre-trained language model, the event text needs to be truncated until it meets the field length requirement of the pre-trained language model.

[0081] Event element semantic classification model training and update: First, the preprocessed event text is input into the event element semantic classification model, and the loss value between the predicted event element semantic category probability distribution and the label is calculated using a loss function. The loss is back-propagated using an optimizer algorithm and the model parameters are updated. The loss function can be cross entropy, focal loss, etc., and the optimizer algorithm can be Adam or SGD.

[0082] The structure diagram of the event element semantic classification model is as follows: Figure 5 As shown in the figure, the event element semantic classification model is composed of an event text encoding model and an event element classification model in series, and the gradients thereof can be transferred and obtained through joint training. The event text encoding model is a pre-trained language model (such as ERNIE, T5, BART) with the classification layer removed. The event element classification model includes an event element feature conversion module and an event element classification module.

[0083] The prediction process of the event element semantic classification model is as follows: Figure 6 As shown, the original event text is first preprocessed and then input into the event text encoding model. After encoding, the corresponding field features of the event elements are selected in turn, and then the event element feature conversion module converts the event element features into another feature space, and finally the event element classification network is used to predict the event element semantic category of the converted event element features; the event element feature conversion module realizes the separation of the feature spaces of the event text encoding model and the event element classification model by converting the feature space of the event text encoding model and then having the event element classification network perform classification; the event text encoding model is a pre-trained language model, which has potential implicit knowledge such as objective world knowledge and grammatical knowledge after pre-training on a large amount of text, which helps to improve the semantic classification effect of event elements and improve the few-sample effect of event element semantic classification; therefore, the event element feature conversion module realizes the separation of the feature space, further retains the implicit knowledge of the event text encoding model, i.e., the pre-trained language model, and helps to improve the accuracy and few-sample effect of the event element semantic classification model;

[0084] In one embodiment, the event element feature conversion module is composed of a global pooling layer, several fully connected layers, a batch normalization layer, and an activation function layer; the global pooling layer is used to convert the field dimension of the field feature of the event element into 1; the event element classification module is a fully connected layer, the input feature dimension of the fully connected layer is equal to the feature dimension of the event element feature after passing through the event element feature conversion module, and the output feature dimension is the label number Z of the event element semantic label, which represents the probability distribution on the predicted event element semantic label;

[0085] For example, the event text "On August 20, China's first batch of emergency food aid arrived in Sri Lanka" is preprocessed to [CLS] and "On August 20, China's first batch of emergency food aid arrived in Sri Lanka [SEP]. After the event text encoding model is applied, the field features corresponding to the agent-subject event element "China" are selected. The dimension is 2*K, where K is the feature encoding dimension in the event text encoding model. After the global pooling layer, the feature dimension becomes K. After the event element classification model is applied, a Z-dimensional vector is output. After training, the event element semantic classification model should predict the agent-subject semantic label of the Chinese element with a higher probability than other semantic labels (such as time and location).

[0086] For each event type, an event prompt template is constructed according to natural language expression methods. The prompt template refers to the cloze pre-training task (MLM) of the pre-trained language model. Based on the natural language expression method and the dependency structure of event elements, it consists of prompt text and a blank prompt ([MASK]). For example, the prompt template for an assistance event can be designed as time: [MASK], helper [MASK] assistance recipient [MASK] supplies [MASK], where [MASK] is a blank prompt marker (which may be different for different pre-trained language models). The pre-trained language model can generate the corresponding text at this [MASK].

[0087] Construct a prompt learning event element summary model, take the generated event prompt template and event text as model input, and output a natural language sequence of event elements;

[0088] In one embodiment, the specific method of constructing the summary of the prompt learning event elements is as follows: Figure 7 , the specific steps include:

[0089] Construct a prompt learning event element summary dataset, combine the event elements in the event element extraction dataset with other texts according to certain rules into a continuous text as the label of the event text, that is, the labeled event element summary text; the rules include the text continuity and naturalness and the prompt template order consistency rules; the text continuity and naturalness means that the text should follow grammatical fluency and coherence, and the prompt template order consistency means that the arrangement order of event elements should be consistent with the event element type corresponding to the blank prompt mark in the prompt template; if the input event text lacks some event elements, the labeled event element summary text also does not contain the event element type text; the other text includes trigger words and necessary punctuation marks, prepositions, and other texts to keep the event summary text continuous and natural;

[0090] For example, for the event text "On August 20, China's first batch of emergency food aid arrived in Sri Lanka," the event type is aid, and the event elements include {August 20, China, food, Sri Lanka}. If the prompt template is designed as time: [MASK], donor [MASK] aid recipient [MASK] supplies [MASK], according to the template order consistency principle, the labels {August 20, China, Sri Lanka, food} are consistent in order with the prompt template and can be filled in the blank prompt marks in the prompt template in sequence; according to the principle of text continuity and naturalness, the label "August 20, China aids Sri Lanka food" is more natural than the label "August 20, China Sri Lanka food" and is more suitable as a label.

[0091] Build a prompt learning event element summary model; the prompt learning event element summary model is a pre-trained language model (such as ERNIE, T5, BART). In one embodiment, its structure is as follows Figure 8 As shown, the event trigger word, event prompt template and event text are connected in series, and the connection is not limited to Figure 8 In the order shown, the event element summary text is output after encoding and decoding in the pre-trained language model; in another embodiment, the event text and the prompt template are input separately, and its structure is as follows Figure 9 As shown, the event text and the event trigger word are input into the encoder of the pre-trained language model together. The order of the event trigger word and the event text can be reversed, and the event trigger word can be removed. The event prompt template is input from the decoder of the pre-trained language model. The input can also add the event trigger word in series before or after the event prompt template. Finally, the event element summary text is output;

[0092] Prompt learning event element summary model training and updating; according to the model input requirements, first pre-process the event text and event prompt template, then input them into the prompt learning event element summary model, use the loss function to calculate the loss value between the summary text predicted by the model and the generated label summary text, and finally use the optimizer algorithm to perform reverse gradient propagation on the loss and update the model parameters; the loss function shown can adopt the text summary loss function, etc., and the optimizer algorithm can adopt Adam, SGD;

[0093] The pre-trained language model is typically pre-trained using a cloze pre-training task (MLM), which is a mainstream pre-training task for pre-trained language models. The MLM replaces part of the input text with [MASK]. After training, the model is equipped with the ability to predict the text at [MASK], i.e., cloze capability. By adding the event prompt template generated in step d to the event text and having the pre-trained language model cloze the blank prompt [MASK], the cloze capability of the pre-trained language model can be brought into play, thereby improving the small-sample effect of event element extraction.

[0094] Optionally, the event element summary model can be pre-trained before training and updating the model. The pre-training uses a text summarization task, where the input of the text summarization task is a piece of text (not limited to event text) and the output is a summary of the text. Although the input text of the text summarization task is not limited to event text, the performance of the event element summary model can still be improved because text summaries and event summaries have certain similarities. The text summarization task can use a public text summary dataset or a self-collected text summary dataset.

[0095] Performing word segmentation and pre-filtering on the event element summary text output in step e to form a group of event elements, and inputting the event text or event element summary text into the event element semantic classification model to implement semantic classification of the group of event elements;

[0096] In one embodiment, the specific method of constructing the semantic classification of event element summaries is described in Figure 10 , the specific steps include:

[0097] Use a word segmentation tool to segment the event element summary text generated in step E into multiple phrases; the word segmentation tool can use an open source word segmentation tool such as Jieba;

[0098] Pre-filter the segmented phrases; this pre-filtering includes stop words (e.g., in, on), part-of-speech filtering (e.g., verbs, prepositions, adjectives), and trigger word filtering (e.g., trigger words for aid events like aid, support, and funding).

[0099] Using an input event element semantic classification model to predict the semantic labels of the group of event elements; the event element semantic classification model, the semantic classification method is to pre-process the event text or event element summary text according to the model requirements, and then input it into the model for classification prediction, and the predicted event element semantic label is the semantic classification result of the event element;

[0100] Using the bidirectional label conversion method of traditional event element labels and event element semantic labels, the event element semantic classification results are converted to achieve event element extraction based on the original event annotation rules;

[0101] Figure 11 This is a schematic diagram of the structure of a linguistics-based prompt learning event element extraction device of the present invention, which is composed of an event prompt template module, a prompt learning event element summary module and an event element decoding module, wherein

[0102] Event prompt template module: used to provide corresponding event prompt templates for each event type to prompt the pre-trained language model to generate event elements;

[0103] Prompt learning event element summary module: used to summarize the event elements corresponding to the event type from the event text and output the natural language sequence of the event elements;

[0104] Event element decoding module: used to segment event element concatenation text, and also used to semantically classify event elements and convert traditional event element labels;

[0105] Furthermore, the event prompt template module includes:

[0106] Event prompt template unit: Based on natural language expression and event element dependency structure, it provides corresponding event prompt templates for each event type to prompt the pre-trained language model to generate event elements;

[0107] Furthermore, the learning event element summary module is prompted to include:

[0108] The prompt learning event element summary unit takes the event prompt template output by the event element prompt template module and the event text as input, uses the text generation ability acquired by the pre-trained language model in the pre-training phase, summarizes the event elements corresponding to the event type from the event text, and outputs a natural language sequence of the event elements.

[0109] Furthermore, the event element decoding module includes:

[0110] Event element decoding unit: This unit is used to segment the event element concatenation text output by the prompt learning event element summary module, and then classify the event elements based on event semantic labels in conjunction with the event element classification module. Finally, in conjunction with the event element conversion module, the event semantic labels are converted into traditional event element labels, thereby realizing event element extraction based on traditional event element labels.

[0111] Event element classification unit: This unit is used to perform semantic classification of one or more event elements contained in the event text. It uses the event text as input and classifies the one or more event elements contained therein. The labels generated by the classification are based on the semantics of the event elements. The model is trained using a small amount of event element semantics dataset.

[0112] The event element semantic dataset is obtained by first obtaining an event element extraction dataset based on traditional event element labels, and then using an event element conversion unit to map and replace event element labels using traditional event element labels in the event element extraction dataset with event element semantic labels;

[0113] Event element conversion unit: used to realize bidirectional label conversion from traditional event element labels to event element semantic labels, used to convert event element semantic labels into traditional event element labels, and used to re-label the dataset based on traditional event element labels to convert it into an event element semantic dataset based on event element semantic labels, which can be used to train the event element classification model;

[0114] The present invention provides an electronic device and a readable storage medium;

[0115] like Figure 12 FIG. 1 is a block diagram of an electronic device of the present invention. The electronic device refers to various modern electronic digital computers, including personal computers, portable computers, and various server devices. The components and their interconnections and functions shown herein are for illustrative purposes only.

[0116] like Figure 12 As shown, the electronic device includes: one or more multi-core processors, one or more GPU computing cards, and memory. In order to enable electronic devices to interact, it should also include: input devices and output devices. Various devices are interconnected and communicated through a bus;

[0117] The memory is a non-transitory computer-readable storage medium provided in this application, wherein the memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor or the at least one GPU computing card to enable the at least one processor or the at least one GPU computing card to perform any one of the methods in the embodiments of this application;

[0118] Input devices provide and receive user input into electronic devices. These include keyboards that generate numeric or character information and mice used to control devices and generate other key signals. Output devices provide feedback to users about electronic devices and include displays that print execution results or processes.

[0119] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the disclosure disclosed herein. This application is intended to cover any variations, uses, or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art not disclosed herein. It should be understood that the present invention is not limited to the precise structure described above and shown in the accompanying drawings, and various modifications and variations may be made without departing from the scope thereof.

Claims

1. A method for extracting event elements based on linguistic analysis and prompt learning, characterized in that: Follow these steps: a. Based on linguistic analysis, a two-way label conversion is formed between traditional event element labels and event element semantic labels; b. Constructing an event element semantic dataset: First, obtain an event element extraction dataset based on traditional event element labels, and re-label the dataset using the label conversion method in step a to form an event element semantic dataset; c. Constructing an event element semantic classification model: This model includes an event text encoding model and an event element classification model to implement semantic classification of event elements. The event element semantic classification model is composed of the event text encoding model and the event element classification model in series and trained together. The event text encoding model is a pre-trained language model with the classification layer removed. The event element classification model includes an event element feature conversion module and an event element classification module. The event element feature conversion module first performs spatial conversion of event element features, and then the event element classification module outputs the event element classification results. d. For each event type, construct an event prompt template according to the natural language expression method. The event prompt template refers to the cloze pre-training task of the pre-trained language model. Based on the natural language expression method and the event element dependency structure, it consists of prompt text and blank prompts. The number of blank prompts is consistent with the number of event element types, and each blank prompt corresponds to an event element. e. Constructing a prompt learning event element summary model: using the event prompt template and event text generated in step d as model input, and outputting a natural language sequence of event elements in series. The prompt learning event element summary model is a pre-trained language model, and the event text is encoded and decoded by the pre-trained language model. The event prompt template can be input in series with the event text or input from the pre-trained language model decoder. The model output is a continuous and natural event element summary text that is sequentially connected by the event elements and other texts. f. Event element summary text decoding: First, the event element concatenation text output in step e is segmented and pre-filtered to form a group of event elements, and the event element semantic classification model in step c is used to semantically classify the group of event elements; g. Use the bidirectional label conversion method of traditional event element label-event element semantic label in step a to transform the event element semantic classification result in step f, so as to realize event element extraction based on the original event annotation rules.

2. The event element extraction method based on linguistic analysis and prompt learning according to claim 1 is characterized in that The bidirectional label conversion of traditional event element labels and event element semantic labels described in step a is specifically as follows: the linguistic analysis means are event element semantic analysis and event element dependency analysis, and the event element types of all event types are classified to form cause, time, place, subject, object, result and event element semantic labels that are weakly related to the event type. The bidirectional mapping relationship between the original traditional event element annotation label and the new event element semantic label is the bidirectional label conversion of traditional event element label and event element semantic label.

3. The event element extraction method based on linguistic analysis and prompt learning according to claim 1 is characterized in that The event element semantic dataset described in step b is a dataset labeled using event element semantic labeling rules. The event element semantic labels are relabeled from the labels of the event element extraction dataset based on traditional event element labels. The relabeling uses a bidirectional label conversion of traditional event element labels and event element semantic labels to map the original traditional event element extraction labels.

4. The event element extraction method based on linguistic analysis and prompt learning according to claim 1 is characterized in that The event element summary text decoding described in step f, wherein the pre-filtering includes stop word filtering, part of speech filtering, and trigger word filtering; the semantic classification method is to pre-process the event element summary text according to the requirements of the event element semantic classification model, and then send it into the event element semantic classification model for classification prediction, and the predicted event element semantic label is the semantic classification result of the event element.

5. An event element extraction device based on linguistic analysis and prompt learning, used to implement the method according to any one of claims 1 to 4, characterized in that: The device is composed of an event prompt template module, a prompt learning event element summary module and an event element decoding module, wherein Event prompt template module: used to provide a corresponding event prompt template for each event type. Based on the natural language expression and event element dependency structure, a corresponding event prompt template is provided for each event type to prompt the pre-trained language model to generate event elements. The prompt learning event element summary module is used to summarize the event elements corresponding to the event type from the event text and output the natural language text of the event elements. It takes the event prompt template output by the event element prompt template module and the event text as input, and uses the text generation ability acquired by the pre-trained language model in the pre-training stage to summarize the event elements corresponding to the event type from the event text and output the natural language text of the event elements. Event element decoding module: used to decode event elements from the event element concatenation text output by the prompt learning event element summary module, including event summary word segmentation unit, event element classification unit, and event element conversion unit. The event summary word segmentation unit is used to segment and pre-filter the event element concatenation text output by the prompt learning event element summary module, and then the event element classification unit is used to classify the event elements output by the event summary word segmentation unit based on event semantic labels. Finally, the event element conversion unit is used to convert the event semantic labels of the event elements into traditional event element labels, thereby realizing event element extraction based on traditional event element labels.

6. An electronic device, characterized in that: The device includes at least one processor; at least one GPU computing card; and a memory communicatively connected to the processor; wherein: the memory stores instructions executed by at least one processor, and the instructions are executed by at least one processor or at least one GPU computing card to enable at least one processor or at least one GPU computing card to execute the method described in any one of claims 1-4.

7. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to enable the computer to execute the method according to any one of claims 1 to 4.

Images