Multi-task event relationship extraction method and system based on relative time analysis

Abstract

The present invention provides a multi-task event relationship extraction method and system based on relative time analysis, which relates to the field of natural language processing. By acquiring text data to be analyzed; inputting the text data into a pre-trained language model ERNIE, and combining the Masked Language Modeling task to enhance the pre-trained language model ERNIE's understanding of events and context semantics; constructing a multi-task learning framework based on relative time prediction; through the multi-task learning framework, jointly training event relationship classification tasks and event relative time prediction tasks, optimizing the model parameters of the pre-trained language model ERNIE; outputting the time relationship between the at least two events. Reduce the gap between the classification task and the pre-training task, thereby improving the model's effect on classifying relationships between events.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to a method and system for extracting multi-task event relations based on relative time analysis. Background Art

[0002] Event relationship extraction technology refers to obtaining the temporal relationship between events in natural language text, such as determining whether an event occurs before, after, or at the same time as another event. Existing research also mentions inclusion and inclusion as well as uncertain temporal relationships, that is, events will have a duration, and then there will be a relationship of inclusion and inclusion, as well as a temporal order that cannot be analyzed in combination with the context, that is, an uncertain relationship. Relationship extraction between events can help understand the dynamics of complex events in natural language, which is beneficial to various downstream tasks, including the construction of event graphs, the analysis and construction of time-series knowledge graphs, the prediction of future events, and question-answering and summarization, information retrieval tasks, and the timeline of events can be sorted out by combining the relationship between events. By understanding the temporal relationship of events that have occurred, constructing event graphs and event-related time-series knowledge graphs, it is possible to predict future events, and construct event timelines based on the constructed sequence, thereby improving the text quality in generation tasks. Existing research on this task mainly includes the following methods: converting the problem into a multi-label classification task, using an encoder to obtain the representation of event pairs, and then inputting them into a multi-layer perceptron for prediction. This method also includes research on the representation of event pairs. Better representation can improve the classification effect, as well as the construction of classification models, including the adjustment of features and attention, so that it is more suitable for the task of analyzing the temporal relationship between events and achieves better classification results, such as the application of pre-trained language models to improve the representation of events; other research methods build graph models to learn the semantic connection between different event trigger words, or use different attention mechanisms to focus on the content between events and before and after events, to focus on event trigger words and the content in between. Text fragments allow for a more fine-grained consideration of the relationship between sentence features and events, thereby achieving relational classification. Other studies have considered the introduction of external knowledge. The use of common sense knowledge in the task of extracting event relations based on end-to-end neural architectures is quite limited. Especially in some common sense issues, the model is easily misled by the semantics of the context. Therefore, in the face of this problem, it is necessary to explore the integration of external knowledge to alleviate the scarcity of event annotations, thereby improving the effect of extracting event relations between events. However, this situation often occurs in highly professional text representations, and often requires a professional knowledge base to help the model understand different events, rather than just the representation of the context. Existing research mostly focuses on the temporal relationship between events in the representation of the context.

[0003] Although the event temporal relationship extraction technology relying on pre-trained language models has made significant progress and achieved good results, there are still some deficiencies in existing research. Most existing technologies model this task as a multi-label classification task, using pre-trained language models to obtain the contextual representation of events, and then using classifiers to classify the temporal relationship between events. However, this method lacks the pre-trained language model's full understanding of event-related corpora. Although the pipeline model combined with the MLM task can continuously learn on the corpus and thus improve the model's understanding of the corpus, it also needs to fine-tune the model to combine it with downstream classification tasks. Therefore, there will still be problems in training caused by poor results due to mismatch and poor test results due to overfitting, that is, poor semantic understanding and insufficient generalization ability.

[0004] Therefore, how to enhance the model's understanding of event context in the task to improve the representation of events, build a more detailed event time prediction task to adapt to complex time sequence relationships, reduce the gap between the pre-trained language model and the classification task to improve the classification effect for a variety of different time sequence relationships has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to improve the generalization ability of the model on specific downstream tasks and reduce the performance difference between pre-training tasks and downstream tasks, the present application provides a multi-task event relationship extraction method and system based on relative time analysis.

[0006] In the first aspect, the present application provides a method for extracting multi-task event relations based on relative time analysis, which adopts the following technical solutions:

[0007] A multi-task event relationship extraction method based on relative time analysis, comprising:

[0008] Acquire text data to be analyzed, wherein the text data contains at least two events;

[0009] Input the text data into the pre-trained language model ERNIE, and enhance the pre-trained language model ERNIE's understanding of events and contextual semantics of the events by combining the Masked Language Modeling task;

[0010] Constructing a multi-task learning framework based on relative time prediction, the multi-task learning framework includes an event relationship classification task and an event relative time prediction task, the event relative time prediction task includes relative time prediction of event start time and event end time;

[0011] Through the multi-task learning framework, jointly train the event relationship classification task and the event relative time prediction task to optimize the model parameters of the pre-trained language model ERNIE;

[0012] The time relationship between the at least two events is output, wherein the time relationship includes at least one of a sequence of occurrence of the events, synchronous occurrence, inclusion relationship, or uncertain relationship.

[0013] Optionally, the step of combining the Masked Language Modeling task to enhance the pre-trained language model ERNIE's understanding of the event and the contextual semantics of the event includes:

[0014] Randomly mask the specified event words or label words in the input text and require the model to predict the masked words;

[0015] A template is constructed through the MLM task, which includes mask tasks for label words, event 1 and event 2. The specific form is:

[0016] ;

[0017] Among them, MLM label 、MLM event1 、MLM event2 They represent the mask task templates for the labels in the template and event 1 and event 2 respectively. Text, event1, [MASK], label, and event2 represent the analysis text where the event is located, event 1 to be analyzed, the mask symbol in the ERNIE model, the label word, and event 2 to be analyzed respectively.

[0018] The MLM task enhances the model's ability to capture event trigger words and their contextual relationships, thereby improving the accuracy of event representation.

[0019] Optionally, the event relative time prediction task includes relative time prediction of event start time and event end time, including:

[0020] Predicting the relative time of the start time and the end time of event 1 and event 2 respectively through a feed-forward network layer, wherein the feed-forward network layer uses a tanh activation function to map the relative time to a range of [-1, 1];

[0021] According to the relative time difference between the start time and the end time of the event, the enhanced representation is obtained by combining the event features, and the event relationship classification is realized through the classification layer.

[0022] Optionally, the step of obtaining an enhanced representation based on the relative time difference between the start time and the end time of the event in combination with event features, and implementing event relationship classification through a classification layer includes:

[0023] Optimize the relative time prediction task using different boundary conditions based on the relationship labels between events;

[0024] The loss of relative time prediction is calculated by the following formula:

[0025] ;

[0026] Among them, r represents the data label relationship between two events, represents the start time of event j, Indicates the start time of event i, Indicates the end time of event j, Indicates the end time of event i; Represents a conditional function. According to the labels in the training data, the relationship between events can be obtained. i ,event j The relationship between two events i and j, if the condition is met, then the value of the function is 1, otherwise it is 0. It represents the loss of relative time prediction, with the help of this loss to achieve training optimization of this task.

[0027] Optionally, the multi-task learning framework is trained by a joint loss function, and the joint loss function is expressed as:

[0028] ;

[0029] in, represents the loss of the label prediction task, Represent the loss of the mask task for event 1 and event 2 respectively, Represents the overall loss of the multi-task learning, , and As a hyperparameter to adjust the weight of the auxiliary task, and satisfy conditions.

[0030] Optionally, the step of enhancing the pre-trained language model ERNIE's understanding of the event and the contextual semantics of the event includes:

[0031] The MaskedLM class of the ERNIE model is used for mask task training. The MaskedLM class masks some words in the input text and requires the model to predict the masked words, thereby improving the model's ability to capture event trigger words and their contextual relationships.

[0032] The cross entropy loss is used to calculate the difference between the prediction and the true label of the mask position and optimize the model parameters.

[0033] Optionally, the method further includes:

[0034] In the inference phase, the temporal relationship labels of event pairs are generated by beam search;

[0035] The complete sequence with the highest probability is selected as the final output, and the temporal relationship labels between event pairs are output.

[0036] In a second aspect, the present application provides a multi-task event relationship extraction system based on relative time analysis, comprising:

[0037] A data acquisition module, used to acquire text data to be analyzed, wherein the text data contains at least two events;

[0038] A data input module, used to input the text data into the pre-trained language model ERNIE, and enhance the pre-trained language model ERNIE's understanding of events and contextual semantics of the events by combining the Masked Language Modeling task;

[0039] A framework construction module, used to construct a multi-task learning framework based on relative time prediction, wherein the multi-task learning framework includes an event relationship classification task and an event relative time prediction task, wherein the event relative time prediction task includes relative time prediction of event start time and event end time;

[0040] An optimization module, used to optimize the model parameters of the pre-trained language model ERNIE by jointly training the event relationship classification task and the event relative time prediction task through the multi-task learning framework;

[0041] The output module is used to output the time relationship between the at least two events, where the time relationship includes at least one of the sequence of event occurrence, synchronous occurrence, inclusion relationship or uncertain relationship.

[0042] In a third aspect, the present application provides a computer device, comprising: a memory and a processor, wherein the processor executes the method described above when running computer instructions stored in the memory.

[0043] In a fourth aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0044] In summary, the present application includes the following beneficial technical effects:

[0045] This application obtains text data to be analyzed; inputs the text data into a pre-trained language model ERNIE, and enhances the pre-trained language model ERNIE's understanding of events and the contextual semantics of the events by combining the Masked Language Modeling task; constructs a multi-task learning framework based on relative time prediction, the multi-task learning framework includes an event relationship classification task and an event relative time prediction task, the event relative time prediction task includes the relative time prediction of the event start time and the event end time; through the multi-task learning framework, jointly trains the event relationship classification task and the event relative time prediction task to optimize the model parameters of the pre-trained language model ERNIE; and outputs the time relationship between the at least two events. Reduce the gap between the classification task and the pre-training task, thereby improving the model's effect on classifying the relationship between events. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application;

[0047] Figure 2 It is a flowchart of the first embodiment of the multi-task event relationship extraction method based on relative time analysis of the present application;

[0048] Figure 3 This is a conceptual diagram of the extraction process of the first embodiment of the multi-task event relationship extraction method based on relative time analysis of the present application;

[0049] Figure 4 This is a schematic diagram of the time sequence relationship between events in the first embodiment of the multi-task event relationship extraction method based on relative time analysis of the present application;

[0050] Figure 5 This is a task flow chart of relative time prediction between events in the first embodiment of the multi-task event relationship extraction method based on relative time analysis of the present application;

[0051] Figure 6 It is a structural block diagram of the first embodiment of the multi-task event relationship extraction system based on relative time analysis of the present application. DETAILED DESCRIPTION

[0052] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below through the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0053] Reference Figure 1 , Figure 1 A schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0054] like Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0056] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and a multi-task event relationship extraction program based on relative time analysis.

[0057] exist Figure 1 In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the present application can be set in the computer device, and the computer device calls the multi-task event relationship extraction program based on relative time analysis stored in the memory 1005 through the processor 1001, and executes the multi-task event relationship extraction method based on relative time analysis provided in the embodiment of the present application.

[0058] The present application embodiment provides a method for extracting multi-task event relations based on relative time analysis. Figure 2 , Figure 2 This is a flowchart of the first embodiment of the multi-task event relationship extraction method based on relative time analysis of the present application.

[0059] In this embodiment, the multi-task event relationship extraction method based on relative time analysis includes the following steps:

[0060] Step S10: Acquire text data to be analyzed, where the text data contains at least two events.

[0061] It should be noted that in this embodiment, the temporal relationship extraction technology between events is dedicated to analyzing the temporal relationship between specified events from the analysis text, that is, the relationship between the time when an event occurs or lasts and another event, which may be the time of occurrence, or simultaneous occurrence, or the inclusion relationship between durations. Of course, it may be difficult to distinguish the relationship between events due to the limited understanding of the context, which is defined as an uncertain time relationship. The relationship extraction between events can help understand the dynamics of complex events in natural language, which is beneficial to various downstream tasks, including the construction of event graphs, the analysis and construction of time series knowledge graphs, the prediction of future events, and question-answering and summarization, information retrieval tasks, etc. Combined with existing research, the challenges faced by this technology include how to enhance the understanding of the event context of the model in the task to improve the representation of the event, how to construct a more detailed event time prediction task to adapt to the various different time sequence relationships corresponding to the more complex labels, and how to reduce the gap between the pre-trained language model and the event relationship classification task to improve the classification effect of various different time sequence relationships. To solve these problems, this embodiment proposes a multi-task event relationship extraction technology based on relative time analysis. This technology is based on a pre-trained language model and combines the MLM task to enhance the model's understanding of events and contextual semantics, and implements the classification of time relationships based on the MLM task. In the relative time prediction of events, two methods are used to process the relative time of the start and the relative time of the end of the event respectively, so that the more complex time relationship between events can be better represented. The above tasks are constructed into a unified joint task framework through multi-task learning, and trained with a unified loss. The innovation of this technology lies in combining advanced natural language processing technology and building a multi-task framework with the help of more similar tasks. The time prediction task focuses on more detailed event time relationships by modeling the relative start time and relative end time of events and uses it as an auxiliary task. The MLM task is combined with prompts to enhance the model's understanding of events and their contexts, and the prompt-related MLM is used as the main task to reduce the gap between the classification task and the pre-training task, thereby improving the model's effect on classifying relationships between events.

[0062] like Figure 3As shown, this embodiment takes the prediction of label words as the main task, combines the analyzed sentences or longer texts, adds the event words and [MASK] to be analyzed at the end to realize the prediction, and combines the MLM task with prompt learning to reduce the gap between the downstream tasks and the pre-training tasks of the pre-trained language model, thereby improving the training effect. In the entire model, the ERNIE pre-trained language model is used, because the model adopts mask pre-training as one of its pre-training tasks, which is the same as the MLM task. The difference is that the downstream task of this embodiment restricts the position of the mask, and does not adopt a random masking method. Instead, a fixed word is masked on each sample to enhance the model's understanding of these events, the understanding of the temporal relationship between events, and the understanding of the temporal relationship between events and the connection between label words after combined prompt learning. The ERNIE model is a pre-trained language model developed by Baidu, based on the encoder structure of Transformer, in which a variety of masking methods are used in the pre-training tasks, including masks at the character, phrase, and entity levels. Code pre-training, richer masking strategies enable the model to achieve better results in multiple natural language understanding tasks. This masking method is helpful for masked understanding of events and labels in subsequent tasks, and has a good adaptation effect on the technical tasks faced by this embodiment. In addition, the model has corresponding Chinese and English versions, and can be combined with other modules to achieve better application and low-cost switching in Chinese event relationship extraction and English event relationship extraction tasks. In addition, ERNIE has a good matching degree for event relationship extraction tasks. Since ERNIE combines a large amount of cross-domain and cross-language knowledge in the pre-training process, the model can better understand the semantics and contextual relationships in the text, which is conducive to accurately extracting the relationship between events. In this embodiment, ernie-2.0-large-en and ernie-2.0-large-zh models can be used for event relationship extraction tasks on English and Chinese data respectively. Of course, other types of ERNIE models can also be used as long as they meet the language type of the data.

[0063] Step S20: Input the text data into the pre-trained language model ERNIE, and enhance the pre-trained language model ERNIE's understanding of events and the contextual semantics of events by combining the Masked Language Modeling task.

[0064] It should be noted that the steps of enhancing the understanding of events and event context semantics of the pre-trained language model ERNIE by combining the Masked Language Modeling task include: randomly masking the specified event words or label words in the input text, and requiring the model to predict the masked words; building a template through the MLM task, the template includes the masking tasks for the label words, event 1 and event 2, and the specific form is:

[0065] ;

[0066] Among them, MLM label 、MLM event1 、MLM event2 They represent the mask task templates for the labels in the template and event 1 and event 2 respectively. Text, event1, [MASK], label, and event2 represent the analysis text where the event is located, event 1 to be analyzed, the mask symbol in the ERNIE model, the label word, and event 2 to be analyzed respectively.

[0067] The MLM task enhances the model's ability to capture event trigger words and their contextual relationships, thereby improving the accuracy of event representation.

[0068] In a specific implementation, the steps to enhance the pre-trained language model ERNIE's understanding of events and their contextual semantics include: using the MaskedLM class of the ERNIE model to perform mask task training. The MaskedLM class masks some words in the input text and requires the model to predict the masked words, thereby improving the model's ability to capture event trigger words and their contextual relationships; and calculating the difference between the prediction of the masked position and the true label through the cross entropy loss to optimize the model parameters.

[0069] In the specific implementation, for the MLM task, the sample data is a combination of existing training data and sample templates. Different from previous studies, this embodiment combines prompt learning, constructs templates in the MLM task, and performs masking on the template. The goal is to train the model to prompt its understanding of events and labels based on the sentence content. The data construction forms of the three different tasks are shown as follows:

[0070] ;

[0071] Among them, MLM label 、MLM event1 、MLM event2 They represent the mask task templates for the labels in the template and event 1 and event 2 respectively. Text, event1, [MASK], label, and event2 represent the analysis text of the event, event 1 to be analyzed, the mask symbol in the ERNIE model, the label word, and event 2 to be analyzed respectively. The above method is used to construct data samples as the input of the model. text Represents the complete sample template, without masking, and uses it as the label data for the MLM task. For the mask label task MLM label , construct a sample sequence with added template , and record the location of the event that needs to be analyzed in the source text event pos , input the processed samples into the Tokenizer of the ERNIE model to obtain the corresponding sample sequence and , and get the label text of :

[0072] ;

[0073] Input into the model to get the predicted representation logits of the [MASK] position label and the coding sequence combined with the complete sample Calculated loss , the loss calculation here uses the cross entropy loss CrossEntropyLoss, which is calculated inside the model, as shown in the following formula, and the loss output can be set directly:

[0074] ;

[0075] ERNIE MaskedLM Indicates the use of the MaskedLM class of the ERNIE model. This class is a model class specifically used for mask tasks and can be called directly. In the loss calculation, set the other positions in the sequence to -100, so that the calculation of other positions can be ignored in the calculation, and only the prediction of the mask position can be calculated. and mask labels The loss between.

[0076] Similarly, the other two MLM tasks, MLM event1 and MLM event2 Mission, Process and MLM label The tasks are similar, and the training data is referenced to the template to construct the sample sequence

[0077] and , and record the location of the event that needs to be analyzed in the source text event pos , input the processed samples into the Tokenizer of the ERNIE model to obtain the corresponding sample sequence and , the sequence obtained after encoding the complete label text has been processed previously , no further processing is needed here.

[0078] ;

[0079] Input into the model to get the predicted representation logits of the [MASK] positionevent1 , logits event2 Combined with the coding sequence of the complete sample Calculated loss , , the loss calculation here uses the cross entropy loss CrossEntropyLoss, which is calculated inside the model, as shown in the following formula, and the loss output can be set directly: ; ; ; ;

[0080] ERNIE MaskedLM Indicates the use of the MaskedLM class of the ERNIE model. This class is a model class specifically used for mask tasks and can be called directly. In the loss calculation, set the other positions in the sequence to -100, so that the calculation of other positions can be ignored in the calculation, and only the prediction of the mask position can be calculated. and mask labels The loss between.

[0081] Step S30: construct a multi-task learning framework based on relative time prediction, the multi-task learning framework includes an event relationship classification task and an event relative time prediction task, and the event relative time prediction task includes relative time prediction of event start time and event end time.

[0082] In the specific implementation, the task of event relative time prediction is to analyze the relative time between the start and end of event 1 and event 2. The analysis of the start and end time of events can adapt them to more complex time relationships. The start and end time of events are modeled separately, which is more in line with the requirements of complex time sequence relationships and is not easy to confuse. Clear relationship conditions and rules are determined for each time sequence relationship. The specific division method is as follows: Figure 4As shown in the figure, the relationship between event 1 and event 2 is not determined by just one time point, but by combining the start and end time. In addition, combined with the definition of relationship labels in existing commonly used datasets, more types of labels are selected, such as the six different labels representing the time sequence relationship shown in the figure. After the detailed division of the start time and the end time, a clearer boundary condition can be obtained. Simply relying on a single time point of an event (such as the start time or the end time) may ignore the duration of the event. Combining the start time and the end time can more accurately capture the relative time relationship between events and avoid misjudgment. If only the start time is used, it may not be possible to distinguish the situation where one event contains another event. Combining the start time and the end time can accurately describe the complex relationship of events such as overlap, inclusion, and simultaneous occurrence. Combining the start time and the end time of an event for relative time analysis can provide a more accurate and comprehensive description of the time relationship, enhance the accuracy and generalization ability of the model, handle complex time relationships, and improve the use of context information. Combined with the definition of the time sequence relationship between events, the prediction of the relative time between events is subdivided into two relative time analyses: the start time of the event and the end time of the time. The classification of the relationship is achieved through comparative prediction between the start time and the end time.

[0083] In the specific implementation, Figure 5 As shown, as an auxiliary task, this part needs to be based on the event pos Get the event representation that needs to be analyzed. Because the analyzed events are repeated, the same event word may appear multiple times in the analyzed text. For example, in the source text Text, the word "said" often appears. This positioning method can locate the target event according to its location, distinguish the event to be analyzed, avoid confusion, and provide the model with more accurate context information, which helps improve the classification effect. In the model output, you can set the hidden layer representation to be returned. Combined with the location information, you can get the hidden layer representation h of the event at a given location. 1 and h 2 Based on the event representation, the prediction layer obtains the prediction of the relative time of the event. Here, it is divided into two prediction layers, which predict the relative time of the start and end respectively. The prediction layer uses The activation function consists of a two-layer feedforward network that maps relative time to the range of [-1,1]. For each sample, the calculation method is as follows:

[0084] ;

[0085] in represents the representation of event i in the sample, represents the relative time prediction of the start time of event i in the sample, FFN m Represents the feedforward network layer. The calculation of the feedforward network layer here only includes:

[0086] , that is, W and b represent the weight matrix and bias vector respectively. These are the learnable parameters of the feedforward network layer and will be learned and updated during the training process.

[0087] Similarly, for the relative time prediction of the event end time:

[0088] ;

[0089] represents the relative time prediction of the start time of event i in the sample, FFN m Represents the feedforward network layer. The prediction layer here does not share parameters with the relative time prediction layer of the event start node. The two layers are trained and adjusted separately.

[0090] It should be noted that the values ​​obtained by relative time analysis are subtracted and combined with the event features to obtain an enhanced representation, which is then classified through the classification layer. Although there is no clear time information as a reference for determining the temporal order of events, the order represented by the labels can be used as incidental supervision for this task, such as for event 1 AFTERevent 2 According to the time relationship definition, event 1 comes after event 2, indicating the start time of event 1 At the end time of event 2 Afterwards, if it is an event 1 SIMULTANEOUS event 2 Similarly, modeling the relationship between different start times and end times based on relative time analysis can reflect the temporal relationship between events. Edge-based optimization methods are used to constrain the predicted relative event time, and different boundary conditions are used to optimize the task according to different temporal relationships.

[0091] In the specific implementation, the event relative time prediction task includes the relative time prediction of the event start time and the event end time, including: predicting the relative time of the start time and the end time of event 1 and event 2 respectively through the feedforward network layer, and the feedforward network layer uses the tanh activation function to map the relative time to the range of [-1,1]; according to the relative time difference between the start time and the end time of the event, the enhanced representation is obtained in combination with the event features, and the event relationship classification is realized through the classification layer.

[0092] It should be noted that the enhanced representation is obtained based on the relative time difference between the start time and the end time of the event, and the step of implementing event relationship classification through the classification layer includes: optimizing the relative time prediction task using different boundary conditions according to the relationship labels between the events; and calculating the loss of relative time prediction by the following formula:

[0093] ;

[0094] Among them, r represents the data label relationship between two events, represents the start time of event j, Indicates the start time of event i, Indicates the end time of event j, Indicates the end time of event i; Represents a conditional function. According to the labels in the training data, the relationship between events can be obtained. i ,event j The relationship between two events i and j, if the condition is met, then the value of the function is 1, otherwise it is 0. It represents the loss of relative time prediction, with the help of this loss to achieve training optimization of this task.

[0095] Step S40: Through a multi-task learning framework, jointly train the event relationship classification task and the event relative time prediction task to optimize the model parameters of the pre-trained language model ERNIE.

[0096] In the specific implementation, the multi-task learning framework is trained through a joint loss function, which is expressed as:

[0097] ;

[0098] in, represents the loss of the label prediction task, Represent the loss of the mask task for event 1 and event 2 respectively, Represents the overall loss of the multi-task learning, , and As a hyperparameter to adjust the weight of the auxiliary task, and satisfy conditions.

[0099] Step S50: outputting the time relationship between at least two events, where the time relationship includes at least one of a sequence of occurrence of the events, synchronous occurrence, inclusion relationship or uncertain relationship.

[0100] In a specific implementation, the method further includes: in the inference stage, generating time relationship labels of event pairs by beam search; selecting the complete sequence with the highest probability as the final output, and outputting the time relationship labels between the event pairs.

[0101] This embodiment obtains text data to be analyzed; inputs the text data into the pre-trained language model ERNIE, and enhances the pre-trained language model ERNIE's understanding of events and event context semantics by combining the Masked Language Modeling task; constructs a multi-task learning framework based on relative time prediction, the multi-task learning framework includes event relationship classification tasks and event relative time prediction tasks, and the event relative time prediction task includes the relative time prediction of event start time and event end time; through the multi-task learning framework, jointly trains the event relationship classification task and the event relative time prediction task, optimizes the model parameters of the pre-trained language model ERNIE; and outputs the time relationship between at least two events. Reduce the gap between the classification task and the pre-training task, thereby improving the model's effect on the classification of relationships between events.

[0102] In addition, an embodiment of the present application also proposes a computer-readable storage medium, on which is stored a program for extracting multi-task event relationships based on relative time analysis. When the program for extracting multi-task event relationships based on relative time analysis is executed by a processor, the steps of the method for extracting multi-task event relationships based on relative time analysis as described above are implemented.

[0103] Reference Figure 6 , Figure 6 This is a structural block diagram of the first embodiment of the multi-task event relationship extraction system based on relative time analysis of the present application.

[0104] like Figure 6 As shown, the multi-task event relationship extraction system based on relative time analysis proposed in the embodiment of the present application includes:

[0105] A data acquisition module 10, used to acquire text data to be analyzed, wherein the text data contains at least two events;

[0106] A data input module 20, used to input the text data into the pre-trained language model ERNIE, and enhance the pre-trained language model ERNIE's understanding of events and contextual semantics of the events by combining the Masked Language Modeling task;

[0107] A framework construction module 30 is used to construct a multi-task learning framework based on relative time prediction, wherein the multi-task learning framework includes an event relationship classification task and an event relative time prediction task, wherein the event relative time prediction task includes relative time prediction of an event start time and an event end time;

[0108] An optimization module 40, configured to optimize the model parameters of the pre-trained language model ERNIE by jointly training the event relationship classification task and the event relative time prediction task through the multi-task learning framework;

[0109] The output module 50 is used to output the time relationship between the at least two events, where the time relationship includes at least one of a sequence of occurrence of the events, synchronous occurrence, inclusion relationship or uncertain relationship.

[0110] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any limitation on this.

[0111] This embodiment obtains text data to be analyzed; inputs the text data into the pre-trained language model ERNIE, and enhances the pre-trained language model ERNIE's understanding of events and event context semantics by combining the Masked Language Modeling task; constructs a multi-task learning framework based on relative time prediction, the multi-task learning framework includes event relationship classification tasks and event relative time prediction tasks, and the event relative time prediction task includes the relative time prediction of event start time and event end time; through the multi-task learning framework, jointly trains the event relationship classification task and the event relative time prediction task, optimizes the model parameters of the pre-trained language model ERNIE; and outputs the time relationship between at least two events. Reduce the gap between the classification task and the pre-training task, thereby improving the model's effect on the classification of relationships between events.

[0112] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0113] In addition, for technical details not described in detail in this embodiment, please refer to the method for extracting multi-task event relations based on relative time analysis provided in any embodiment of the present application, which will not be repeated here.

[0114] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0115] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0116] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0117] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A multi-task event relationship extraction method based on relative time analysis, characterized in that: include: Acquire text data to be analyzed, wherein the text data contains at least two events; Input the text data into the pre-trained language model ERNIE, and enhance the pre-trained language model ERNIE's understanding of events and contextual semantics of the events by combining the Masked Language Modeling task; Constructing a multi-task learning framework based on relative time prediction, the multi-task learning framework includes an event relationship classification task and an event relative time prediction task, the event relative time prediction task includes relative time prediction of event start time and event end time; Through the multi-task learning framework, jointly train the event relationship classification task and the event relative time prediction task to optimize the model parameters of the pre-trained language model ERNIE; Outputting the time relationship between the at least two events, the time relationship including at least one of a sequence of occurrence of the events, synchronous occurrence, inclusion relationship, or indeterminate relationship; The step of combining the Masked Language Modeling task to enhance the pre-trained language model ERNIE's understanding of the event and the contextual semantics of the event includes: Randomly mask the specified event words or label words in the input text and require the model to predict the masked words; A template is constructed through the MLM task, which includes mask tasks for label words, event 1 and event 2. The specific form is: ； Among them, MLM label 、MLM event1 、MLM event2 They represent the mask task templates for the labels in the template and event 1 and event 2 respectively. Text, event1, [MASK], label, and event2 represent the analysis text where the event is located, event 1 to be analyzed, the mask symbol in the ERNIE model, the label word, and event 2 to be analyzed respectively. The MLM task enhances the model's ability to capture event trigger words and their contextual relationships, thereby improving the accuracy of event representation.

2. The multi-task event relationship extraction method based on relative time analysis according to claim 1 is characterized in that: The event relative time prediction task includes the relative time prediction of the event start time and the event end time, including: Predicting the relative time of the start time and the end time of event 1 and event 2 respectively through a feed-forward network layer, wherein the feed-forward network layer uses a tanh activation function to map the relative time to a range of [-1, 1]; According to the relative time difference between the start time and the end time of the event, the enhanced representation is obtained by combining the event features, and the event relationship classification is realized through the classification layer.

3. The multi-task event relationship extraction method based on relative time analysis according to claim 2 is characterized in that: The step of obtaining an enhanced representation based on the relative time difference between the start time and the end time of the event in combination with the event features and implementing event relationship classification through the classification layer includes: Optimize the relative time prediction task using different boundary conditions based on the relationship labels between events; The loss of relative time prediction is calculated by the following formula: ； in Represents a conditional function. According to the labels in the training data, the relationship between events can be obtained. i ,event j The relationship between two events i and j, if the condition is met, then the value of the function is 1, otherwise it is 0. It represents the loss of relative time prediction, with the help of this loss to achieve training optimization of this task.

4. The multi-task event relationship extraction method based on relative time analysis according to claim 1 is characterized in that: The multi-task learning framework is trained by a joint loss function, which is expressed as: ； in, represents the loss of the label prediction task, Represent the loss of the mask task for event 1 and event 2 respectively, Represents the overall loss of the multi-task learning, , and As a hyperparameter to adjust the weight of the auxiliary task, and satisfy conditions.

5. The multi-task event relationship extraction method based on relative time analysis according to claim 1 is characterized in that: The step of enhancing the pre-trained language model ERNIE's understanding of the event and the contextual semantics of the event includes: The MaskedLM class of the ERNIE model is used for mask task training. The MaskedLM class masks some words in the input text and requires the model to predict the masked words, thereby improving the model's ability to capture event trigger words and their contextual relationships. The cross entropy loss is used to calculate the difference between the prediction and the true label of the mask position and optimize the model parameters.

6. The multi-task event relationship extraction method based on relative time analysis according to claim 1 is characterized in that: The method further comprises: In the inference phase, the temporal relationship labels of event pairs are generated by beam search; The complete sequence with the highest probability is selected as the final output, and the temporal relationship labels between event pairs are output.

7. A multi-task event relationship extraction system based on relative time analysis, characterized in that: Executing the method according to claim 1, comprising: A data acquisition module, used to acquire text data to be analyzed, wherein the text data contains at least two events; A data input module, used to input the text data into the pre-trained language model ERNIE, and enhance the pre-trained language model ERNIE's understanding of events and contextual semantics of the events by combining the Masked Language Modeling task; A framework construction module, used to construct a multi-task learning framework based on relative time prediction, wherein the multi-task learning framework includes an event relationship classification task and an event relative time prediction task, wherein the event relative time prediction task includes a relative time prediction of an event start time and an event end time; An optimization module, used to optimize the model parameters of the pre-trained language model ERNIE by jointly training the event relationship classification task and the event relative time prediction task through the multi-task learning framework; The output module is used to output the time relationship between the at least two events, where the time relationship includes at least one of the sequence of event occurrence, synchronous occurrence, inclusion relationship or uncertain relationship.

8. A computer device, characterized in that: The device comprises: a memory and a processor, and when the processor runs the computer instructions stored in the memory, the processor executes the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 6.

Images