Method, apparatus and electronic device for event generation

Abstract

Embodiments of the present application disclose a method and device for event generation and electronic equipment. The embodiments of the present application acquire at least one target unstructured text; input the at least one target unstructured text into a pre-trained unified pre-training language model, and output at least one event sequence, wherein each event sequence comprises a delimiter, a trigger word and an argument; and deserialize the at least one event sequence to generate at least one target event, wherein the target event comprises the trigger word and the argument. Through the above method, the accuracy of event extraction results can be improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of data processing, in particular to an event generation method and device and electronic equipment. BACKGROUND

[0002] Events are an important form of knowledge, which are usually contained in a large amount of unstructured text and need to be extracted from the unstructured text.

[0003] In the prior art, the sequence labeling method is usually used for extraction. Specifically, trigger words representing the occurrence of an event are first found in unstructured text, and then each argument in the event is found in turn. The above method needs to define the schema of the event in advance, and then extract according to the defined schema. The method cannot extract events of types other than the schema. When facing a large amount of unstructured text, the user cannot define all the schemas of the events in advance. Therefore, when extracting events, the event extraction result is inaccurate.

[0004] To sum up, how to accurately extract events is a problem to be solved at present. SUMMARY

[0005] Therefore, the embodiments of the present application provide an event generation method, device and electronic equipment, which can improve the accuracy of event extraction results.

[0006] In a first aspect, the embodiments of the present application provide an event generation method, which comprises: acquiring at least one target unstructured text; inputting the at least one target unstructured text into a pre-trained unified pre-training language model to output at least one event sequence, wherein each event sequence comprises a separator, a trigger word and an argument; and performing deserialization processing on the at least one event sequence to generate at least one target event, wherein the target event comprises the trigger word and the argument.

[0007] Optionally, the pre-trained unified pre-training language model comprises a generation module and a discrimination module.

[0008] Optionally, the training process of the unified pre-training language model comprises:

[0009] acquiring a historical data set, wherein the historical data set comprises historical unstructured text and historical events;

[0010] alternately training the generation module and the discrimination module according to the historical data set to update the main parameters of the unified pre-training language model;

[0011] determining the unified pre-training language model according to the updated main parameters of the unified pre-training language model.

[0012] Optionally, the generation module is trained according to the historical data set, and main parameters of the unified pre-training language model are updated, specifically including:

[0013] The historical events are serialized to generate a historical event sequence, wherein the historical event sequence includes a delimiter, a trigger word and an argument.

[0014] A first input sequence is generated according to the historical unstructured text and the historical event sequence.

[0015] A generation loss of the generation module is calculated according to the first input sequence.

[0016] The main parameters of the unified pre-training language model are updated according to the generation loss.

[0017] Optionally, in response to the historical event sequence carrying a text category label and the first input sequence including a CLS semantic vector, the generation loss includes a generation task loss and a classification task loss.

[0018] Optionally, the generation task loss corresponds to a first weight, the classification task loss corresponds to a second weight, and the generation loss is determined according to numerical values of the first weight and the second weight.

[0019] Optionally, the discrimination module is trained according to the historical data set, and the main parameters of the unified pre-training language model are updated, specifically including:

[0020] The historical events are serialized to generate a historical event sequence, wherein the historical event sequence includes a delimiter, a trigger word and an argument.

[0021] An error historical event sequence is generated according to the historical events.

[0022] Second input sequences are respectively generated according to the historical unstructured text and the historical event sequence and the historical unstructured text and the error historical event sequence.

[0023] A discrimination loss of the discrimination module is calculated according to the second input sequence.

[0024] The main parameters of the unified pre-training language model are updated according to the discrimination loss.

[0025] In a second aspect, an event generation apparatus is provided, and the apparatus includes an acquisition unit configured to acquire at least one target unstructured text.

[0026] a processing unit, configured to input the at least one target unstructured text into a pre-trained unified pre-training language model, and output at least one event sequence, wherein each of the event sequences comprises a delimiter, a trigger word, and an argument;

[0027] a generating unit, configured to perform deserialization processing on the at least one event sequence, and generate at least one target event, wherein the target event comprises the trigger word and the argument.

[0028] Optionally, the pre-trained unified pre-training language model comprises a generating module and a discriminant module.

[0029] Optionally, in a training process of the unified pre-training language model:

[0030] The acquisition unit is further configured to acquire a historical data set, wherein the historical data set comprises historical unstructured text and historical events.

[0031] The apparatus further comprises an updating unit configured to alternately train the generating module and the discriminant module according to the historical data set, and update main parameters of the unified pre-training language model.

[0032] The apparatus further comprises a determination unit configured to determine the unified pre-training language model according to the updated main parameters of the unified pre-training language model.

[0033] Optionally, the updating unit is specifically configured to:

[0034] perform serialization processing on the historical events to generate a historical event sequence, wherein the historical event sequence comprises the delimiter, the trigger word, and the argument;

[0035] generate a first input sequence according to the historical unstructured text and the historical event sequence;

[0036] calculate a generation loss of the generating module according to the first input sequence;

[0037] update the main parameters of the unified pre-training language model according to the generation loss.

[0038] Optionally, in response to the historical event sequence carrying a text category label and the first input sequence comprising a CLS semantic vector, the generation loss comprises a generation task loss and a classification task loss.

[0039] Optionally, the generation task loss corresponds to a first weight, and the classification task loss corresponds to a second weight, and the generation loss is determined according to numerical values of the first weight and the second weight.

[0040] Optionally, the updating unit is specifically configured to:

[0041] serializing the historical events to generate a historical event sequence, wherein the historical event sequence comprises a delimiter, a trigger word and an argument;

[0042] generating an error historical event sequence according to the historical events;

[0043] generating a second input sequence according to the historical unstructured text and the historical event sequence, and the historical unstructured text and the error historical event sequence, respectively;

[0044] calculating a discriminant loss of the discriminant module according to the second input sequence;

[0045] updating the subject parameters of the unified pre-training language model according to the discriminant loss.

[0046] In a third aspect, an embodiment of the present application provides a computer readable storage medium, which stores computer program instructions, and the computer program instructions implement the method in the first aspect or any possible implementation of the first aspect when executed by a processor.

[0047] In a fourth aspect, an embodiment of the present application provides an electronic device, which comprises a memory and a processor, and the memory is configured to store one or more computer program instructions, and the one or more computer program instructions are configured to be executed by the processor to implement the method in the first aspect or any possible implementation of the first aspect.

[0048] In a fourth aspect, an embodiment of the present application provides an electronic device, which comprises a memory and a processor, and the memory is configured to store one or more computer program instructions, and the one or more computer program instructions are configured to be executed by the processor to implement the method in the first aspect or any possible implementation of the first aspect.

[0049] The embodiment of the present application acquires at least one target unstructured text; inputs the at least one target unstructured text into a pre-trained unified pre-training language model to output at least one event sequence, wherein each event sequence comprises a delimiter, a trigger word and an argument; and serializes at least one event sequence to generate at least one target event, wherein the target event comprises the trigger word and the argument. Through the above method, the accuracy of the event extraction result can be improved. BRIEF DESCRIPTION OF DRAWINGS

[0050] The above and other objects, features and advantages of the present application will become more apparent from the following description when taken in conjunction with the accompanying drawings, in which:

[0051] Figure 1 is a method flowchart of event generation according to an embodiment of the present application;

[0052] Figure 2 is a method flowchart of a training process of a unified pre-training language model according to an embodiment of the present application;

[0053] Figure 3 is a method flowchart for updating subject parameters according to an embodiment of the present application;

[0054] Figure 4 is a first input sequence structure schematic diagram according to an embodiment of the present application;

[0055] Figure 5 is a Mask matrix schematic diagram according to an embodiment of the present application;

[0056] Figure 6 is a method flowchart for updating subject parameters according to an embodiment of the present application;

[0057] Figure 7 is a second input sequence structure schematic diagram according to an embodiment of the present application;

[0058] Figure 8 is a device schematic diagram for event generation according to an embodiment of the present application;

[0059] Figure 9 is a schematic diagram of an electronic device according to an embodiment of the present application. DETAILED DESCRIPTION

[0060] The present disclosure is described in detail based on the embodiments. However, the present disclosure is not limited to only the embodiments. In the following detailed description of the present disclosure, some specific details are described in detail. The present disclosure can be fully understood without the description of these specific details by those skilled in the art. In order to avoid confusion of the essence of the present disclosure, the well-known methods, processes, flows, elements and circuits are not described in detail.

[0061] In addition, those skilled in the art should understand that the drawings provided herein are for illustrative purposes only, and the drawings are not necessarily drawn to scale.

[0062] Unless the context clearly requires otherwise, throughout the description and the claims of this application, the words "comprise", "comprising", and the like are to be construed in an inclusive sense as opposed to an exclusive or exhaustive sense; that is to say, in the sense of "including, but not limited to".

[0063] In the description of the present disclosure, it should be understood that the terms "first", "second" and the like are used only for the purpose of description and should not be construed as indicating or implying relative importance. In addition, in the description of the present disclosure, unless otherwise specified, the meaning of "multiple" is two or more.

[0064] In the prior art, events are usually extracted in a sequence labeling manner. Specifically, a trigger word representing an event occurrence is first found from unstructured text, and then each argument in the event is found in turn, where the argument is each type of word that constitutes the event. When the above extraction in the sequence labeling manner is used, the schema of the event needs to be defined in advance, and then the extraction is performed according to the defined schema, which cannot extract events of types other than the schema. For example, assuming that the schema is "verb + adjective + noun" (this is only an exemplary illustration), the event extraction is performed according to the schema in the unstructured text. However, in the face of a large amount of unstructured text, the user cannot define all the schemas of the events in advance, and therefore, when the event extraction is performed, the event extraction result is inaccurate. In summary, how to accurately extract events is a problem to be solved at present.

[0065] In the embodiments of the present application, Figure 1 is a method flowchart for event generation. As shown in Figure 1 , the method specifically includes the following steps:

[0066] Step S100, at least one target unstructured text is acquired.

[0067] In a possible implementation manner, it is assumed that the target unstructured text is "Xiaozhang fainted in the park", which is only an exemplary illustration. Specifically, the specific content of the target unstructured text is determined according to actual conditions, and the embodiments of the present application do not limit the same.

[0068] Step S101, the at least one target unstructured text is input into a pre-trained unified pre-training language model, and at least one event sequence is output, where each event sequence includes a delimiter, a trigger word and an argument.

[0069] Specifically, the pre-trained unified pre-training language model includes a generation module and a discrimination module.

[0070] In the embodiment of the application, the "Xiaozhang fainted in the park" is input into the pre-trained unified pre-training language (Unilm) model, and the output event sequence is "<extra_id_0> <extra_id_0> trigger word: fainted <extra_id_0> person: Xiaozhang <extra_id_1> <extra_id_0> place: park <extra_id_1> <extra_id_1> <extra_id_1>", wherein the <extra_id_0> is a start delimiter, the <extra_id_1> is an end delimiter, the trigger word is "fainted", and the "person: Xiaozhang" and "place: park" are arguments. This is only an exemplary illustration, and the specific event sequence is determined according to the actual situation, and the embodiment of the application does not limit it.

[0071] In step S102, at least one event sequence is deserialized to generate at least one target event, wherein the target event includes the trigger word and the argument.

[0072] Specifically, the event sequence "<extra_id_0> <extra_id_0> trigger word: fainted <extra_id_0> person: Xiaozhang <extra_id_1> <extra_id_0> place: park <extra_id_1> <extra_id_1> <extra_id_1>" is deserialized to generate the target event "trigger word: fainted person: Xiaozhang place: park", that is, only the trigger word and the argument are included, and the delimiter is not included.

[0073] According to the above method, the target event can be accurately obtained from the unstructured text without setting multiple schemas. Only the unstructured text needs to be input into the pre-set Unilm model, and then deserialization is performed, so that the processing efficiency is high and the accuracy is high.

[0074] In the embodiment of the application, the method flow chart of the training process of the unified pre-training language model is as shown in Figure 2 , and specifically includes:

[0075] In step S200, historical data sets are obtained, wherein the historical data sets include historical unstructured texts and historical events.

[0076] Specifically, the historical dataset can be downloaded from the Internet or obtained through other means. If the historical dataset is downloaded from the Internet, it can be Baidu's event extraction competition dataset. The event extraction competition dataset contains a large number of common event types, has a high degree of text freedom, and the text in the dataset has been manually annotated and can be used directly. The event types can be layoff events, sports events, economic events, etc., and this embodiment of the invention does not limit them.

[0077] In one possible implementation, the obtained historical dataset is as follows:

[0078] The text reads: "Company A lays off 4,000 employees: When the times abandon you, they won't even say goodbye!"

[0079] "id": "409389c96efe78d6"

[0080] “event_list”: [{“event_type”:“Organizational Relationship-Layoffs”,

[0081] "trigger": layoffs

[0082] "trigger_start_index": 2,

[0083] "arguments": [{ "argument_start_index": 0,

[0084] "role": "the party laying off employees"

[0085] “argument”: “Company A”

[0086] “alias”:[]},

[0087] {"argument_start_index": 4,

[0088] "role": "Number of employees laid off"

[0089] “argument”: “4000 people”

[0090] “alias”:[]}],

[0091] "class": "Organizational Relationship"

[0092] The historical dataset includes unstructured text “Company A lays off 4,000 people: When the times abandon you, they won’t even say goodbye!” and historical events “Layoffs: Company A lays off 4,000 people.”

[0093] Step S201, alternately training the generation module and the discrimination module according to the historical data set, updating the subject parameters of the unified pre-training language model.

[0094] In a possible implementation, when training the generation module and the discrimination module, each time one of the modules is selected to train with a certain probability, for example, 4 data are selected to train each time, which can be used to train the generation module or the discrimination module, the two modules share the subject parameters of the Unilm model, and the subject parameters of the Unilm model can also be updated when training the generation module and the discrimination module. Specifically, the discrimination module as an auxiliary module can strengthen the generation capability of the Unilm model by updating the subject parameters of the Unilm model.

[0095] Specifically, the alternately training the generation module and the discrimination module according to the historical data set, updating the subject parameters of the unified pre-training language model can be divided into two parts, specifically as follows:

[0096] The first part is training the generation module according to the historical data set, updating the subject parameters of the unified pre-training language model.

[0097] Specifically, Figure 3 is a method flowchart for updating subject parameters. As shown in Figure 3 , specifically includes the following steps:

[0098] Step S300, performing serialization processing on the historical events to generate a historical event sequence, wherein the historical event sequence includes a separator, a trigger word and an argument.

[0099] In a possible implementation, since the historical events labeled in the historical data set are all structured data, the historical events need to be serialized first, and then converted into a historical event sequence. Specifically, in the historical event sequence, <extra_id_0> and <extra_id_1> are used as the start delimiter and the end delimiter of each part in the event, that is, each argument, and the specific format is as follows: <extra_id_0> argument: {argument} <extra_id_1>; all the serialized arguments in the single event are spliced together, and the format is as follows: <extra_id_0> argument: {argument} <extra_id_1> <extra_id_0> argument: {argument} <extra_id_1>, which takes two arguments as an example, and the actual situation is determined; then the trigger word of the single event is spliced in to obtain the serialized text of the single event, and the specific format is: <extra_id_0> trigger word: {trigger word} {spliced argument} <extra_id_1>.

[0100] For example, first, each argument in a single event is serialized, including: <extra_id_0> layoff party: A company <extra_id_1>; <extra_id_0> number of layoffs: 4000 people <extra_id_1>; then all the serialized arguments in the single event are spliced together, as shown below: <extra_id_0> layoff party: A company <extra_id_1> <extra_id_0> number of layoffs: 4000 people <extra_id_1>; then the trigger word of the single event is spliced in to obtain the serialized text of the single event, and the specific format is: <extra_id_0> trigger word: layoffs <extra_id_0> layoff party: A company <extra_id_1>; <extra_id_0> number of layoffs: 4000 people <extra_id_1>

[0101] <extra_id_1>.

[0102] In a possible implementation, if the historical event includes multiple single events, assuming there are n single events, all the serialized single events are spliced together to obtain the final historical event sequence, and the specific format is as follows:

[0103] <extra_id_0> {serialized event 1}... {serialized event n} <extra_id_1>.

[0104] Step S301, generating a first input sequence according to the historical unstructured text and the historical event sequence.

[0105] Specifically, the first input sequence can be as follows: Figure 4 As shown, it includes CLS, x1, x2, x3, ..., xm, SEP, y1, y2, ..., yn, SEP, where x1, x2, x3, ..., xm are source tokens, and y1, y2, ..., yn are target tokens. The source token corresponds to the historical unstructured text, and the segment_id of the source token is 0. The target token corresponds to the historical event sequence, and the segment_id of the target token is 1.

[0106] Step S302: Calculate the generation loss of the generation module based on the first input sequence.

[0107] In one possible implementation, during the training of the Unilm model, characters in the source token sequence are mutually visible, i.e., x1, x2, x3, ..., xm are mutually visible; when calculating attention, x1, x2, x3, ..., xm participate in each other; on the target token, the current token character can only see the token characters before it and cannot see the token characters after it, to prevent information leakage during the generation process.

[0108] For example, assuming the source token is "What do you want to eat?" and the target token is "Barbecue", the source token and the target token are concatenated to generate the first input sequence: "[CLS]What do you want to eat?[SEP]Barbecue[SEP]", and the corresponding Mask matrix is ​​as follows. Figure 5 As shown, spaces indicate where tokens are visible in each row, and no spaces indicate where tokens are not visible. Cross-entropy is calculated for each position in the target token sequence, using the following formula:

[0109]

[0110] Wherein, yo,c is the true distribution, and po,c is the distribution predicted by the Unilm model.

[0111] In this embodiment of the invention, the cross-entropy, which is the generation loss of the generation module, can be denoted as Loss1.

[0112] Step S303, updating the main body parameters of the unified pre-training language model according to the generation loss.

[0113] Specifically, the gradient of the main body parameters of the unified pre-training language model is calculated according to the Loss1, and then the main body parameters of the unified pre-training language model are updated.

[0114] In a possible implementation, the generation loss can further include a classification task loss in addition to the generation task loss. Specifically, the classification task can be denoted as Loss2, and the semantic vector of the CLS position included in the first input sequence can be used for text classification in response to the historical event sequence carrying a text category label (tag). Different text categories usually contain different events, for example, financial news usually contains events such as listing and acquisition. The semantic vector of the CLS position is used for text classification. Figure 5 As shown in the Mask matrix, the semantic vector of the CLS position is fused with the semantic information of the entire source token sequence when attention is calculated. Therefore, the semantic vector of the CLS is fully connected to k dimensions, where k is the number of text category labels, and then Softmax is calculated for text classification. The Softmax calculation formula is as follows:

[0115]

[0116] wherein the represents the probability of being predicted as a certain label (Label), and the is the sum of all label probabilities. The cross-entropy loss is calculated according to the softmax result, that is, the Loss2.

[0117] In a possible implementation, the generation loss includes the generation task loss and the classification task loss. The generation task loss corresponds to a first weight, and the classification task loss corresponds to a second weight. The generation loss is determined according to the numerical values of the first weight and the second weight, and the specific formula is as follows:

[0118] Loss = λ1Loss1 + λ2Loss2

[0119] wherein the Loss is the generation loss, the Loss1 is the generation task loss, the Loss2 is the classification task loss, the λ1 is the weight of the Loss1, the λ2 is the weight of the Loss2, the λ1 and the λ2 are set by a user in advance, and specific data is determined according to actual conditions. The embodiments of the present application do not limit them.

[0120] In the embodiments of the present application, the classification task loss serves as an auxiliary to strengthen the understanding ability of the Unilm model.

[0121] The second part trains the discrimination module according to the historical data set, and updates the main body parameters of the unified pre-training language model.

[0122] Specifically, Figure 6 is a method flowchart for updating main body parameters according to an embodiment of the present application. As shown in the figure, the method specifically includes the following steps: Figure 6 The steps are as follows:

[0123] Step S600: serializing the historical events to generate a historical event sequence, wherein the historical event sequence includes a delimiter, a trigger word and an argument.

[0124] In a possible implementation, the historical events labeled in the historical data set are all structured data, for example, the historical event is “trigger word: layoffs, layoffs: A company, number of layoffs: 4000”; the historical event is serialized to generate a historical event sequence, specifically as follows: first, serialize each argument in a single event, including: <extra_id_0> layoffs: A company <extra_id_1>; <extra_id_0> number of layoffs: 4000 <extra_id_1>; then, concatenate all the serialized arguments in a single event, as follows: <extra_id_0> layoffs: A company <extra_id_1> <extra_id_0> number of layoffs: 4000 <extra_id_1>; then, concatenate the trigger word of the single event to obtain the serialized text of the single event, and the specific format is: <extra_id_0> trigger word: layoffs <extra_id_0> layoffs: A company <extra_id_1>; <extra_id_0> number of layoffs: 4000 <extra_id_1>

[0125] <extra_id_1>.

[0126] Step S601: generating an error historical event sequence according to the historical event.

[0127] Specifically, first, generate an error historical event according to the historical event, for example, the historical event is “trigger word: layoffs, layoffs: A company, number of layoffs: 4000”, and the corresponding unstructured text is “A company layoffs 4000: when the times discard you, even a greeting will not be given!”, and the error historical event can be “trigger word: layoffs, layoffs: times, number of layoffs: 4000”; wherein, the error historical event is generated by randomly replacing the words in the unstructured text from the historical event, and the error historical event can be generated in multiple.

[0128] In a possible implementation, the error history event sequence is generated according to the error history event, specifically as follows: first, serialize each argument in a single event, including: <extra_id_0> Layoff Party: Times <extra_id_1>; <extra_id_0> Layoff Number: 4000 people <extra_id_1>; then concatenate all the serialized arguments in a single event, as follows: <extra_id_0> Layoff Party: Times <extra_id_1> <extra_id_0> Layoff Number: 4000 people <extra_id_1>; then concatenate the trigger word of the single event to obtain the serialized text of the single event, and the specific format is: <extra_id_0> Trigger word: Layoff <extra_id_0> Layoff Party: Times <extra_id_1>; <extra_id_0> Layoff Number: 4000 people <extra_id_1> <extra_id_1>.

[0129] In step S602, a second input sequence is generated according to the history unstructured text, the history event sequence, the history unstructured text, and the error history event sequence respectively.

[0130] Specifically, in the Unilm model training process, each piece of history unstructured text and its corresponding history event sequence are concatenated as a positive sample, that is, a second input sequence, and the label is set to 1; each piece of history unstructured text and its corresponding error history event sequence are concatenated as a negative sample, that is, a second input sequence, and the label is set to 0.

[0131] In step S603, a discrimination loss of the discrimination module is calculated according to the second input sequence.

[0132] Specifically, the second input sequence is input into the Unilm model, segment_id is 0, it is guaranteed that the entire input sequence can be seen by each other, then the semantic vector at the CLS position is fully connected to two dimensions and the softmax is calculated, the binary classification is performed, and finally the cross-entropy loss is calculated according to the softmax result. That is, the discrimination loss.

[0133] For example, the structure of the second input sequence is as shown in Figure 7 , wherein the source is a history unstructured text, the target is the history event sequence, or the target is the error history event sequence, and the segment_id of the source and the target is 0. Figure 7In the embodiment, 0 represents a negative sample, 1 represents a positive sample, the second input sequence is input, and a semantic vector at a CLS position is fully connected to two dimensions and a softmax is calculated, a probability that it is a negative sample is 0.8, a probability that it is a positive sample is 0.2, and then a cross-entropy loss is calculated as a discrimination loss.

[0134] In step S604, the subject parameter of the unified pre-training language model is updated according to the discrimination loss.

[0135] In the embodiment, the subject parameter of the unified pre-training language model is updated by the first part and the second part.

[0136] In step S202, the unified pre-training language model is determined according to the updated subject parameter of the unified pre-training language model.

[0137] Figure 8 is a device schematic diagram of event generation in the embodiment of the application. As shown in Figure 8 The device of the embodiment includes an acquisition unit 801, a processing unit 802 and a generation unit 803.

[0138] The acquisition unit 801 is configured to acquire at least one target unstructured text; the processing unit 802 is configured to input the at least one target unstructured text into a pre-trained unified pre-training language model, and output at least one event sequence, wherein each event sequence includes a delimiter, a trigger word and an argument; and the generation unit 803 is configured to perform deserialization processing on the at least one event sequence, and generate at least one target event, wherein the target event includes the trigger word and the argument.

[0139] Further, the pre-trained unified pre-training language model includes a generation module and a discrimination module.

[0140] Further, in the training process of the unified pre-training language model:

[0141] The acquisition unit is further configured to acquire a historical data set, wherein the historical data set includes historical unstructured text and historical events.

[0142] The device further includes an update unit configured to alternately train the generation module and the discrimination module according to the historical data set, and update a subject parameter of the unified pre-training language model.

[0143] The device further includes a determination unit configured to determine the unified pre-training language model according to the updated subject parameter of the unified pre-training language model.

[0144] Further, the update unit is specifically configured to:

[0145] The historical events are serialized to generate a historical event sequence, wherein the historical event sequence includes delimiters, trigger words, and arguments;

[0146] A first input sequence is generated based on the historical unstructured text and the historical event sequence;

[0147] The generation loss of the generation module is calculated based on the first input sequence;

[0148] The principal parameters of the unified pre-trained language model are updated based on the generation loss.

[0149] Furthermore, in response to the historical event sequence carrying text category labels, and the first input sequence including a CLS semantic vector, the generation loss includes generation task loss and classification task loss.

[0150] Furthermore, the generation task loss corresponds to a first weight, and the classification task loss corresponds to a second weight. The generation loss is determined based on the values ​​of the first weight and the second weight.

[0151] Furthermore, the updating unit is specifically used for:

[0152] The historical events are serialized to generate a historical event sequence, wherein the historical event sequence includes delimiters, trigger words, and arguments;

[0153] Generate an error history event sequence based on the aforementioned historical events;

[0154] A second input sequence is generated based on the historical unstructured text and the historical event sequence, and the historical unstructured text and the erroneous historical event sequence, respectively.

[0155] The discrimination loss of the discrimination module is calculated based on the second input sequence;

[0156] The main parameters of the unified pre-trained language model are updated based on the discriminative loss.

[0157] Figure 9 This is a schematic diagram of an electronic device according to an embodiment of the present invention. (For example...) Figure 9 As shown, Figure 9The electronic device shown is an event generation apparatus, which includes a general computer hardware structure, at least including a processor 901 and a memory 902. The processor 901 and the memory 902 are connected through a bus 903. The memory 902 is suitable for storing instructions or programs executable by the processor 901. The processor 901 can be a stand-alone microprocessor, or a set of one or more microprocessors. Thus, the processor 901 performs the processing of data and the control of other apparatuses by executing the instructions stored in the memory 902, thereby implementing the method flow of the embodiments of the application as described above. The bus 903 connects the above-mentioned components together, while connecting the above-mentioned components to a display controller 904 and a display apparatus, and an input / output (I / O) apparatus 905. The input / output (I / O) apparatus 905 can be a mouse, a keyboard, a modem, a network interface, a touch input apparatus, a body sense input apparatus, a printer, and other apparatuses known in the art. Typically, the input / output apparatus 905 is connected to the system through an input / output (I / O) controller 906.

[0158] The instructions stored in the memory 902 are executed by the at least one processor 901 to implement: obtaining at least one target unstructured text; inputting the at least one target unstructured text into a pre-trained unified pre-training language model, and outputting at least one event sequence, wherein each of the event sequences includes a delimiter, a trigger word, and an argument; and deserializing the at least one event sequence to generate at least one target event, wherein the target event includes the trigger word and the argument.

[0159] Specifically, the electronic device includes one or more processors 901 and a memory 902, Figure 9 Taking one processor 901 as an example. The processor 901 and the memory 902 can be connected through a bus or other means, Figure 9 Taking the connection through the bus as an example. The memory 902 is a non-volatile computer readable storage medium, which can be used to store non-volatile software programs, non-volatile computer executable programs and modules. The processor 901 performs various functional applications and data processing of the device by running the non-volatile software programs, instructions and modules stored in the memory 902, that is, implements the above-mentioned event generation method.

[0160] The memory 902 can include a program storage area and a data storage area, where the program storage area can store an operating system, at least one application required by a function, and the data storage area can store an option list, etc. In addition, the memory 902 can include a high-speed random access memory, and can also include a nonvolatile memory, such as at least one magnetic disk storage device, a flash memory device, or other nonvolatile solid-state memory device. In some embodiments, the memory 902 can optionally include a memory disposed remotely with respect to the processor 901, which can be connected to an external device through a network. Examples of the above network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0161] One or more modules are stored in the memory 902, and when executed by the one or more processors 901, perform the method of event generation in any of the above method embodiments.

[0162] The above product can perform the method provided by the embodiments of the present application, has the corresponding function modules and beneficial effects of performing the method, and the technical details not described in detail in the embodiments can be referred to the method provided by the embodiments of the present application.

[0163] Embodiments of the present application relate to a non-volatile storage medium for storing a computer-readable program for a computer to execute part or all of the above method embodiments.

[0164] That is, those skilled in the art can understand that all or part of the steps of the above-mentioned embodiment methods can be completed by programs instructing related hardware, the programs are stored in a storage medium, and include a plurality of instructions for causing a device (which can be a single-chip microcomputer, a chip, etc.) or a processor to execute all or part of the steps of the method described in the embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0165] Those skilled in the art can understand that the above embodiments are specific embodiments for implementing the present application, and in actual applications, various changes can be made in form and details without departing from the spirit and scope of the present application.

Claims

1. A method for generating events, characterized in that, The method includes: Obtain at least one target unstructured text; The at least one target unstructured text is input into a pre-trained unified pre-trained language model, which outputs at least one event sequence, wherein each event sequence includes a delimiter, a trigger word, and an argument. The pre-trained unified pre-trained language model includes a generation module and a discrimination module. The training process of the unified pre-trained language model includes: acquiring a historical dataset, wherein the historical dataset includes historical unstructured text and historical events; alternately training the generation module and the discrimination module based on the historical dataset, and updating the main parameters of the unified pre-trained language model; and determining the unified pre-trained language model based on the updated main parameters of the unified pre-trained language model. At least one of the event sequences is deserialized to generate at least one target event, wherein the target event includes the trigger word and the argument.

2. The method as described in claim 1, characterized in that, The generation module is trained based on the historical dataset, and the main parameters of the unified pre-trained language model are updated, specifically including: The historical events are serialized to generate a historical event sequence, wherein the historical event sequence includes delimiters, trigger words, and arguments; A first input sequence is generated based on the historical unstructured text and the historical event sequence; The generation loss of the generation module is calculated based on the first input sequence; The principal parameters of the unified pre-trained language model are updated based on the generation loss.

3. The method as described in claim 2, characterized in that, In response to the historical event sequence carrying text category labels, and the first input sequence including a CLS semantic vector, the generation loss includes generation task loss and classification task loss.

4. The method as described in claim 3, characterized in that, The generation task loss corresponds to a first weight, and the classification task loss corresponds to a second weight. The generation loss is determined based on the values ​​of the first weight and the second weight.

5. The method as described in claim 1, characterized in that, The discrimination module is trained based on the historical dataset, and the main parameters of the unified pre-trained language model are updated, specifically including: The historical events are serialized to generate a historical event sequence, wherein the historical event sequence includes delimiters, trigger words, and arguments; Generate an error history event sequence based on the aforementioned historical events; A second input sequence is generated based on the historical unstructured text and the historical event sequence, and the historical unstructured text and the erroneous historical event sequence, respectively. The discrimination loss of the discrimination module is calculated based on the second input sequence; The main parameters of the unified pre-trained language model are updated based on the discriminative loss.

6. An event generation apparatus, characterized in that, The device includes: An acquisition unit is used to acquire at least one target unstructured text. A processing unit is configured to input the at least one target unstructured text into a pre-trained unified pre-trained language model and output at least one event sequence, wherein each event sequence includes a delimiter, a trigger word, and an argument. The pre-trained unified pre-trained language model includes a generation module and a discrimination module. The training process of the unified pre-trained language model includes: acquiring a historical dataset, wherein the historical dataset includes historical unstructured text and historical events; alternately training the generation module and the discrimination module according to the historical dataset to update the main parameters of the unified pre-trained language model; and determining the unified pre-trained language model according to the updated main parameters of the unified pre-trained language model. A generation unit is configured to deserialize at least one of the event sequences to generate at least one target event, wherein the target event includes the trigger word and the argument.

7. A computer-readable storage medium storing computer program instructions thereon, characterized in that, The computer program instructions, when executed by a processor, implement the method as described in any one of claims 1-5.

8. An electronic device comprising a memory and a processor, characterized in that, The memory is used to store one or more computer program instructions, wherein the one or more computer program instructions are executed by the processor to implement the method as described in any one of claims 1-5.

Images