Text extraction method and device, computer-readable storage medium, and electronic device

Abstract

The present disclosure relates to a text extraction method and device, a computer-readable storage medium, and an electronic device, and relates to the field of machine learning technology. The method comprises: obtaining data to be extracted, and using a target trigger word extraction model included in a target event extraction model to extract a current trigger word included in the data to be extracted (S110); generating a current question sentence based on the current trigger word (S120); extracting a current event element corresponding to the current trigger word based on the current question sentence and a target element extraction model included in the target event extraction model (S130); wherein the target trigger word extraction model and the target element extraction model have the same model structure and parameters, and are connected in a cascade manner. The present disclosure improves the correlation between the current trigger word and the current event element.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of machine learning technology, and in particular, to a text extraction method, a text extraction device, a computer-readable storage medium, and an electronic device. Background Art

[0002] In recent years, with the rise of media digitization and sustained economic growth, we have witnessed an explosive growth in digital media files. Massive amounts of media files exist on the Internet, such as a large amount of domestic and international news. Therefore, how to extract specific events from this massive amount of media files has become a problem that needs to be solved.

[0003] In traditional event definitions, an event consists of a trigger word and elements that describe the event structure. The trigger word signals the occurrence of the event, while the event element is the carrier that expresses the important information of the event. In some event extraction schemes, event extraction can be performed by first extracting the trigger word and then extracting the event elements.

[0004] However, since trigger words and event elements are extracted separately, there is a lack of information interaction between the trigger words and event elements, which makes the correlation between the trigger words and event elements weak. Summary of the Invention

[0005] According to one aspect of the present disclosure, a text extraction method is provided, comprising:

[0006] Acquire data to be extracted, and use the target trigger word extraction model included in the target event extraction model to extract the current trigger word included in the data to be extracted;

[0007] Generate a current question sentence according to the current trigger word;

[0008] Extracting a current event element corresponding to the current trigger word based on the current question sentence and the target element extraction model included in the target event extraction model; wherein the target trigger word extraction model and the target element extraction model have the same model structure and parameters and are connected in a cascade manner.

[0009] In an exemplary embodiment of the present disclosure, the target trigger word extraction model included in the target event extraction model is used to extract the current trigger word included in the data to be extracted, including:

[0010] Utilizing the first dynamic weight fusion Bert layer included in the target trigger word extraction model included in the target event model, the semantic text vectors of the data to be extracted and the importance of each semantic text vector to the text to be extracted are calculated, and a first current encoding vector is obtained based on the semantic text vectors and the importance.

[0011] Calculating the first current encoding vector using the first fully connected layer included in the target trigger word extraction model to obtain a confidence score that each character included in the to-be-extracted data belongs to the current trigger word;

[0012] Determining the starting position and the ending position of the current trigger word according to whether the confidence level is greater than a first preset threshold;

[0013] The starting position and the ending position of the current trigger word are marked with a first preset identifier, and characters corresponding to the positions marked with the first preset identifier are intercepted to generate the current trigger word.

[0014] In an exemplary embodiment of the present disclosure, the first dynamic weight fusion Bert layer includes multiple Transformer models;

[0015] Among them, using the first dynamic weight fusion Bert layer included in the target trigger word extraction model included in the target event model, calculating the semantic text vector of the data to be extracted and the importance of each semantic text vector to the text to be extracted, and obtaining the first current encoding vector according to the semantic text vector and the importance, including:

[0016] Performing word embedding on the data to be extracted to obtain a word embedding vector, a word embedding matrix, and a position embedding matrix of the data to be extracted;

[0017] Generate an embedding vector according to the word embedding vector, the word embedding matrix, and the position embedding matrix, and input the embedding vector into a first Transformer model to generate a first text semantic vector;

[0018] Inputting the first text semantic vector into a second Transformer model to generate a second text semantic vector, and repeating the steps of generating the second text semantic vector to obtain text semantic vectors corresponding to other Transformer models;

[0019] The importance of each Transformer model to the data to be extracted is calculated, and the first current encoding vector is obtained based on each importance, the embedding vector, and each text semantic vector.

[0020] In an exemplary embodiment of the present disclosure, obtaining the first current encoding vector according to each of the importance levels, the embedding vector, and each of the text semantic vectors includes:

[0021] Splicing the importance levels, and normalizing the spliced importance levels;

[0022] The first current encoding vector is obtained according to the normalized importance levels, the embedding vector, and the text semantic vectors.

[0023] In an exemplary embodiment of the present disclosure, extracting a current event element corresponding to the current trigger word according to the current question sentence and the target element extraction model included in the target event extraction model includes:

[0024] Generate a first sentence pair according to the data to be extracted and the current question sentence, and encode the first sentence pair using a second dynamic weight fusion Bert layer included in the target element extraction model included in the target event model to obtain a second current encoding vector;

[0025] Calculating boundaries of the current event element in the data to be extracted based on the second current encoding vector, and classifying the elements included in the boundaries to obtain probabilities that characters at all positions from a head position pointer to a tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element; wherein each boundary includes a head position pointer and a tail position pointer;

[0026] Determining the starting position and the ending position of the current event element according to whether the probability is greater than a second preset threshold, and marking the starting position and the ending position of the current event element with a second preset identifier;

[0027] The character corresponding to the position with the second preset identification mark is intercepted to generate the current event element.

[0028] In an exemplary embodiment of the present disclosure, the elements included in the boundary are classified to obtain the probability that the characters at all positions from the head position pointer to the tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element, including:

[0029] Classifying the roles of the elements included in the boundary based on multiple binary classification networks to obtain role labels of the elements;

[0030] Generate a head position matrix and a tail position matrix according to the head position pointers and the tail position pointers of all the role tags; wherein each row in the head position matrix and the tail position matrix represents a role type, and each column corresponds to each character of the data to be extracted;

[0031] The head position matrix and the tail position matrix are calculated using the second fully connected layer included in the target element extraction model to obtain the probability that the characters at all positions from the head position pointer to the tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element.

[0032] In an exemplary embodiment of the present disclosure, the text extraction method further includes:

[0033] The current event polarity and the current event tense of the data to be extracted are calculated according to the current trigger word and the target event attribute extraction model included in the target event extraction model.

[0034] In an exemplary embodiment of the present disclosure, calculating the current event polarity and the current event tense of the data to be extracted based on the current trigger word and the target event attribute extraction model included in the target event extraction model includes:

[0035] Generate a second sentence pair according to the current trigger word and the data to be extracted;

[0036] Encoding the second sentence pair using a third dynamic weight fusion Bert layer included in the target attribute extraction model to obtain a third current encoding vector;

[0037] Calculating the third current encoding vector using a third fully connected layer included in the target attribute extraction model to obtain a current event polarity of the data to be extracted;

[0038] The third current encoding vector is calculated using the fourth fully connected layer included in the target attribute extraction model to obtain the current event tense of the data to be extracted.

[0039] In an exemplary embodiment of the present disclosure, the text extraction method further includes:

[0040] Obtaining original text data, and calculating original trigger words included in the original text data according to the trigger word extraction model to be trained included in the event extraction model to be trained;

[0041] generating an original question sentence according to the original trigger word, and extracting original event elements included in the original text data according to the original question sentence and the element extraction model to be trained included in the event extraction model to be trained;

[0042] Constructing a first loss function based on the target trigger word and the original trigger word of the original text data, and constructing a second loss function based on the target event element and the original event element of the original news event;

[0043] The parameters included in the event extraction model to be trained are adjusted according to the first loss function and the second loss function to obtain a target event extraction model.

[0044] In an exemplary embodiment of the present disclosure, the parameters included in the event extraction model to be trained are adjusted according to the first loss function and the second loss function to obtain a target event extraction model, including:

[0045] Performing a sum operation on the first loss function and the second loss function to obtain a target loss function;

[0046] The parameters included in the event extraction model to be trained are adjusted according to the target loss function to obtain a target event extraction model.

[0047] In an exemplary embodiment of the present disclosure, the text extraction method further includes:

[0048] Calculating the original event polarity and the original event tense of the original text data according to the original trigger word and the event attribute extraction model to be trained;

[0049] Constructing a third loss function based on the target event polarity, target event tense, original event polarity, and original event tense of the original text data;

[0050] The parameters included in the attribute extraction model to be trained are adjusted according to the third loss function to obtain a target attribute extraction model.

[0051] According to one aspect of the present disclosure, there is provided a text extraction device, comprising:

[0052] A first extraction module is used to obtain data to be extracted and extract the current trigger word included in the data to be extracted using the target trigger word extraction model included in the target event extraction model;

[0053] A question sentence generating module, configured to generate a current question sentence based on the current trigger word;

[0054] The second extraction module is used to extract the current event element corresponding to the current trigger word based on the current question sentence and the target element extraction model included in the target event extraction model; wherein the target trigger word extraction model and the target element extraction model have the same model structure and parameters, and are connected in a cascade manner.

[0055] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the computer program implements any one of the above-mentioned text extraction methods.

[0056] According to one aspect of the present disclosure, there is provided an electronic device, including:

[0057] processor; and

[0058] a memory for storing executable instructions of the processor;

[0059] The processor is configured to execute any one of the above-mentioned text extraction methods by executing the executable instructions.

[0060] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0061] The accompanying drawings are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification, are used to explain the principles of the present disclosure. Obviously, the drawings described below are only some embodiments of the present disclosure, and those skilled in the art can derive other drawings based on these drawings without inventive effort. In the drawings:

[0062] Figure 1 The following schematically shows a flow chart of a text extraction method according to an exemplary embodiment of the present invention.

[0063] Figure 2 An example diagram of a target event extraction model according to an example embodiment of the present invention is schematically shown.

[0064] Figure 3 An example diagram schematically shows another target event extraction model according to an example embodiment of the present invention.

[0065] Figure 4 A flowchart of a method for training an event extraction model to be trained according to an exemplary embodiment of the present invention is schematically shown.

[0066] Figure 5 The following schematically shows an example diagram of an extraction process of original trigger words according to an exemplary embodiment of the present invention.

[0067] Figure 6 An example diagram of a process of extracting original event elements according to an example embodiment of the present invention is schematically shown.

[0068] Figure 7 A flowchart of a method for training an event attribute extraction model to be trained according to an exemplary embodiment of the present invention is schematically shown.

[0069] Figure 8An example diagram of a process for extracting original event attributes according to an example embodiment of the present invention is schematically shown.

[0070] Figure 9 A flowchart schematically illustrates a method for extracting a current trigger word included in the data to be extracted by utilizing a target trigger word extraction model included in a target event extraction model according to an exemplary embodiment of the present invention.

[0071] Figure 10 A flowchart schematically illustrates a method for extracting a current event element corresponding to the current trigger word based on the current question sentence and a target element extraction model included in a target event extraction model according to an exemplary embodiment of the present invention.

[0072] Figure 11 A flowchart schematically illustrates a method for calculating the current event polarity and the current event tense of the data to be extracted based on the current trigger word and the target event attribute extraction model included in the target event extraction model according to an example embodiment of the present invention.

[0073] Figure 12 An exemplary diagram schematically illustrates a process of extracting text events according to an exemplary embodiment of the present invention.

[0074] Figure 13 The following schematically shows a block diagram of a text extraction device according to an exemplary embodiment of the present invention.

[0075] Figure 14 An electronic device for implementing the above-mentioned text extraction method according to an exemplary embodiment of the present invention is schematically shown. DETAILED DESCRIPTION

[0076] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be embodied in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete and will fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0077] In addition, the accompanying drawings are merely schematic illustrations of the present disclosure and are not necessarily drawn to scale. Identical reference numerals in the figures denote identical or similar parts, and thus repetitive descriptions thereof will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically separate entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0078] This exemplary embodiment first provides a text extraction method, which can be run on a server, server cluster or cloud server, etc. Of course, those skilled in the art can also run the method of the present invention on other platforms as needed, and this exemplary embodiment does not specifically limit this. Figure 1 As shown, the text extraction method may include the following steps:

[0079] Step S110: Acquire the data to be extracted, and use the target trigger word extraction model included in the target event extraction model to extract the current trigger word included in the data to be extracted;

[0080] Step S120. Generate a current question sentence based on the current trigger word;

[0081] Step S130. Extract the current event element corresponding to the current trigger word according to the current question sentence and the target element extraction model included in the target event extraction model; wherein the target trigger word extraction model and the target element extraction model have the same model structure and parameters, and are connected in a cascade manner.

[0082] In the above text extraction method, on the one hand, by obtaining the data to be extracted and using the target trigger word extraction model included in the target event extraction model, the current trigger word included in the data to be extracted is extracted; then the current question sentence is generated according to the current trigger word; finally, the current event element corresponding to the current trigger word is extracted according to the current question sentence and the target element extraction model included in the target event extraction model; since the current event element is extracted based on the current question sentence generated by the current trigger word, the information interaction between the current trigger word and the current event element is fully considered in the process of extracting the current event element, which solves the problem in the prior art that the trigger word and the event element are extracted separately, and there is a lack of information interaction between the trigger word and the event element, thereby making the information interaction between the trigger word and the event element The problem of weak correlation is solved, and the correlation between the current trigger word and the current event element is improved; on the other hand, since the target trigger word extraction model and the target element extraction model are connected in a cascade manner, the current event element corresponding to the current trigger word is extracted in a reading comprehension manner, and the semantic representation layer of the cascade structure is shared, which enhances the information interaction between the current trigger word and the current event element; on the other hand, since the target trigger word extraction model and the target element extraction model have the same model structure and parameters, the target trigger word extraction model and the target element extraction model can be trained simultaneously during the model training process. As long as the parameters of one of the models are adjusted, the parameters of the other model no longer need to be adjusted, thereby improving the training speed of the model.

[0083] Hereinafter, the steps included in the text extraction method of the present disclosure will be explained and illustrated in detail with reference to the accompanying drawings.

[0084] First, the terms used in this disclosure are explained.

[0085] Event text extraction, which presents events from unstructured text in a structured form, plays a crucial role in areas such as public opinion monitoring, text summarization, automated question answering, and the automatic construction of causal graphs. Furthermore, in real news, event text extraction is a challenging task due to the complex sentence structure, active-passive transitions, and the sharing of subjects and objects in multiple events. This task can be broken down into four subtasks: trigger word identification, event type classification, event element (argument) identification, and role classification.

[0086] Domain-limited event extraction means that before extracting event text, the types of event texts to be extracted and the specific structure of each type (which specific event elements are included) are pre-defined. Usually, a certain amount of labeled data is given to extract events of specified event types and role types from unstructured text. For example, given the event schema shown in Table 1, two events, "competition behavior-win or lose" and "competition behavior-advancement", as well as two roles, "winner" and "advancement party", Raptors, "loser" 76ers, and "advancement event" Eastern Conference Finals, can be extracted from Table 2 (the trigger words here are not part of the event elements).

[0087] Table 1: Event schema

[0088] category Event Type Event Role Event Role Event Role Competition behavior Victory or defeat Winner loser time Competition behavior Advancement time Advancement qualifying tournament …… …… …… …… ……

[0089] Table 2: Leonard's buzzer-beater helped the Raptors eliminate the 76ers and advance to the Eastern Conference Finals

[0090]

[0091] Open-domain event extraction involves extracting event triggers, event arguments, and event attributes from news text. Traditionally, events are defined as consisting of a trigger and elements describing the event structure. The trigger signifies the occurrence of an event, while arguments, such as the subject, object, time, and location, convey important information about the event. The trigger, acting as the event predicate, is part of the event element, as shown in Examples 1 and 2 below. Events can also include attributes, including polarity and tense, which are crucial for determining whether an event actually occurred. Polarity categorizes events as positive, negative, or possible. Tense categorizes events as past, current, future, or other events whose tense is uncertain.

[0092] Example 1:

[0093] Text: At 7:15 p.m. Beijing time on March 27, British Prime Minister Boris Johnson was confirmed to have been infected with the new coronavirus.

[0094] trigger words main body object time Place Infect British Prime Minister Boris Johnson Coronavirus disease March 27, Beijing time none

[0095] Example 2:

[0096] Text: A spokesman for the transitional government forces said that NATO warplanes hit a building near Sirte on the 16th, killing a large number of Gaddafi forces.

[0097] trigger words main body object time Place hit NATO fighter jets A building 16th Near Sirte Killed by explosion NATO fighter jets Soldiers of Gaddafi's forces 16th Near Sirte

[0098] Example 3:

[0099] Text: The World Health Organization has declared "Public Health Emergencies of International Concern" five times in the past.

[0100] trigger words polarity tense Announce affim past

[0101] Example 4:

[0102] Text: The UK is likely to return to accepting World Trade Organization rules.

[0103] trigger words polarity tense Resume acceptance possible future

[0104] Next, the target event extraction model of the present disclosure is explained and illustrated. Figure 2As shown, the target event extraction model may include a target trigger word extraction model and a target element extraction model, and the target trigger word extraction model and the target element extraction model are connected in a cascade manner; further, the target event extraction model may also include a target attribute extraction model, and the target attribute extraction model is independent of the target trigger word extraction model and the target element extraction model, and it only takes the trigger word and the data to be extracted as input.

[0105] For further reference, Figure 3 As shown, the target trigger word extraction model may include a first dynamic weight fusion Bert layer (Dynamic Weight Fusion Layer) 301 and a first fully connected layer 302, the first fully connected layer is composed of Dense+Sigmoid; the first dynamic weight fusion Bert layer is connected to the first fully connected layer. Among them, the first dynamic weight fusion Bert layer 301 includes multiple Transformer models (Transformer Encoder 1, Transformer Encoder2,..., Transformer Encoder L) 3011, and each Transformer model corresponds to a fully connected unit (Dense Unit). Among them, the first dynamic weight fusion Bert layer is used to calculate the first current encoding vector of the data to be extracted, the first fully connected layer 302 is used to extract the current trigger word, and the fully connected unit (Dense Unit) is used to calculate the importance of each Transformer model. The first dynamic weight fusion BERT layer 301 also includes a dynamic weight fusion layer 3012, which is used to weightedly fuse the outputs h1, h2...hL of multiple Transformer models and the weights μ1, μ2...μL of multiple Transformer models to obtain the output of the first dynamic weight fusion BERT layer 301.

[0106] Continue to refer Figure 3 As shown, the target element extraction model includes a second dynamic weight fusion Bert layer 303 and a second fully connected layer 304. Its functions and connection methods are similar to those of the above-mentioned target trigger word extraction models, and will not be repeated here.

[0107] Continue to refer Figure 3As shown, the target attribute extraction model may include a third dynamic weight fusion BERT layer 305, a third fully connected layer 306, and a fourth fully connected layer 307, wherein the third dynamic weight fusion BERT layer 305, the third fully connected layer 306, and the fourth fully connected layer 307 are connected in sequence. The third dynamic weight fusion BERT layer is used to calculate the third current encoding vector of the second sentence pair generated according to the current trigger word and the data to be extracted, the third fully connected layer is used to calculate the current event polarity of the data to be extracted, and the fourth fully connected layer is used to calculate the current event tense of the data to be extracted.

[0108] It should be noted that the first fully connected layer, the second fully connected layer, the third fully connected layer and the fourth fully connected layer can each include two Dense layers and one sigmoid function layer.

[0109] The following is an explanation of the training process of the target event extraction model. Figure 4 As shown, the training process of the target event extraction model may include steps S410 to S440.

[0110] In step S410, original text data is acquired, and original trigger words included in the original text data are calculated based on the trigger word extraction model to be trained included in the event extraction model to be trained.

[0111] In step S420, an original question sentence is generated according to the original trigger word, and the original event elements included in the original text data are extracted according to the original question sentence and the element extraction model to be trained included in the event extraction model to be trained;

[0112] In step S430, a first loss function is constructed based on the target trigger word and the original trigger word of the original text data, and a second loss function is constructed based on the target event element and the original event element of the original news event;

[0113] In step S440, the parameters included in the event extraction model to be trained are adjusted according to the first loss function and the second loss function to obtain a target event extraction model.

[0114] In this example embodiment, first, the first loss function and the second loss function are summed to obtain a target loss function; secondly, the parameters included in the event extraction model to be trained are adjusted according to the target loss function to obtain a target event extraction model.

[0115] Hereinafter, step S410 to step S440 will be explained and illustrated.

[0116] First, the encoding principle of the dynamic weight fusion BERT layer is explained and illustrated. Specifically, the present invention uses BERT as an encoder to extract the semantic features of the text. Specifically: at the data input end, the input sentence X is first encoded into a word embedding matrix W t and the position embedding matrix W p ; Where X=(x1,x2,...,x n Then, the word embedding matrix and the position embedding matrix are added together as the total input embedding representation h0, and the input vector representation h0 is passed through an N-layer Transformer network to obtain the text semantic representation vector h l , which can be specifically expressed as the following formula (1) and formula (2):

[0117] h0=XW t +W p ; Formula (1)

[0118] h l =Transformer(h l-1 ),l∈[1,N]; Formula (2)

[0119] Among them, h l is the hidden layer vector, i.e. the output of the l-th layer Transformer network.

[0120] At the same time, in order to effectively utilize the information of each layer in Bert, first, a fully connected unit (DenseUnit) can be used to learn the importance of each Transformer μ, and then the results of all Transformer layers are weighted and superimposed to obtain the final semantic representation, which can be shown in the following formula (3):

[0121] μ l =σ(Dense unit＝1 (h l )); Formula (3)

[0122] Among them, σ is the ReLu activation function; secondly, the weights of each layer are spliced and normalized with the softmax function to obtain a 1*L weight vector μ L , which can be specifically expressed as the following formula (4):

[0123] μ L =softmax(concatenate([μ1,μ2,...,μ l ])); Formula (4)

[0124] Then, all the Transformer results in BERT are concatenated to obtain a fused representation, which can be specifically expressed as follows:

[0125] h L =concatenate([h1,h2,...,h l ]); Formula (5)

[0126] Finally, the weighted fusion obtained by formula (4) can be used to obtain the final fusion semantic representation h DWT , which is the final encoding vector, can be specifically expressed as follows:

[0127] h DWF =matmul([μ L ,h L ]); Formula (6)

[0128] It should be further explained here that since the weight μ is automatically updated according to the importance of each layer during network training, the Bert layer can be called adaptive weight fusion BERT.

[0129] Secondly, step S410 to step S440 are explained and illustrated in combination with the encoding principle of the dynamic weight fusion Bert layer. Specifically, first, the original text data is input into the underlying DWF-BERT model in a single input mode, that is: the sentence is encoded into [CLS: first vector, text vector of the sentence] On the evening of March 27, Beijing time, British Prime Minister Boris Johnson was diagnosed with new coronary pneumonia [SEP, two input separators], which is input into the underlying DWF-BERT model (the first dynamic weight fusion Bert layer) to obtain the encoding vector output by BERT, and then the encoding vector output by BERT is passed through the first fully connected layer, and the value at each position of the final output sequence is the confidence of the start and end positions of the entity. For details, please refer to Figure 5 Here, we take the positions with confidence greater than 0.45 as the start and end positions of the entity, and intercept the corresponding positions of the original text data to get the original trigger word.

[0130] The specific calculation process of the confidence can be shown in the following formula (7) and formula (8):

[0131]

[0132]

[0133] Among them, x i =h DWF [i], that is, the encoding vector of the i-th character of the original text data after BERT encoding, σ is the sigmoid activation function, W start and W end is the preset trainable weight (parameter), bs and b e is the corresponding bias term (parameter), as well as The probability that the i-th character in the original text data is the starting position and the ending position of the trigger word, respectively. When the probability is greater than the preset threshold (0.45), the corresponding position will be marked as 1, otherwise it will be marked as 0 (for details, please refer to Figure 5 shown).

[0134] Secondly, after obtaining the original trigger word, it is necessary to extract the original event element corresponding to the original trigger word. Specifically, this disclosure treats the extraction of original event elements as a sequence labeling task. Unlike traditional sequence labeling solutions based on BiLSTM-CRF, this disclosure uses a reading comprehension approach combined with a multi-layer label pointer network to label event elements.

[0135] Specifically, first, a question sentence query can be constructed using the original trigger words obtained by the trigger word extraction model to be trained. For example, in the example sentence "On the evening of March 27th, Beijing time, British Prime Minister Boris Johnson was diagnosed with COVID-19," the original trigger word extracted is "infection," and the constructed question sentence query is: "Find the subject, object, time, and place related to #infection#." Secondly, after obtaining the question sentence, the query and the original text data can be combined into a sentence pair. This sentence pair can, for example, be: "[CLS]Find the subject, object, time, and place related to #infection#[SEP]On the evening of March 27th, Beijing time, British Prime Minister Boris Johnson was diagnosed with COVID-19[SEP]." This sentence pair is then fed into the DWF-BERT model as a dual input. It should be noted that to highlight the role of the original trigger word, "#" symbols can be added to the left and right of the trigger word "infection" in the sentence pair query. Of course, other emphasis symbols can also be used, and this example does not impose any special restrictions on this. Of course, unlike traditional reading comprehension methods, the query and the answer do not correspond one-to-one. Therefore, each event element type is still marked using annotations. The purpose of constructing the query is to enhance the semantic information of the trigger word, so as to better obtain the event element corresponding to the current trigger word.

[0136] Furthermore, the upper layer DWF-BERT model (the second dynamic weight fusion BERT layer) is used to encode the sentence pairs composed of the question sentence and the original text data. The specific encoding process is similar to the encoding process of the original text data described above and will not be repeated here. At the same time, after obtaining the encoding of the sentence pair, the encoding of the sentence pair is input into the second fully connected layer to obtain the original event element. The specific processing process can be shown as follows:

[0137] To extract multiple event elements simultaneously, we first need to use two binary classification networks to generate two 0 / 1 sequences to determine the boundary spans of the event elements in the sequence (each span is determined by a head position pointer start and a tail position pointer end); then, we use multiple binary classification networks to classify the element spans into roles; among them, each character in the input sequence can represent the start and end positions of a certain element, and the span composed of text between any two characters can be represented as any event role.

[0138] Figure 6 The following gives the labeling examples of the input samples of sentence pairs obtained from the question sentence and the original text data. Each role corresponds to a set of pointer vectors (start, end). Combining the start and end pointer vectors of all labels together can obtain two two-dimensional matrices, which are denoted as S. s and S e , where S s and S e Each row in the text represents a character type, and each column corresponds to a character in the text. The text uses multiple sets of binary classification networks to predict the probability that all positions in the start and end pointer vectors corresponding to each character in the input sequence are 0 / 1 to determine the start and end positions of the element and the character type. The entire task can be regarded as a multi-label classification of each character in the input sequence, and the probability value of the i-th character being predicted as the start and end position of the element of character r is as well as They can be expressed as the following formulas (9) and (10):

[0139]

[0140]

[0141] in, is the starting position probability, is the probability of the terminal position, x i =h DWF [i], that is, the encoding vector of the i-th character in the sentence pair after BERT encoding, σ is the sigmoid activation function, as well as are preset trainable weights (parameters), as well as For the corresponding bias item (parameter), when the probability is greater than the preset threshold (0.45), the corresponding position will be marked as 1, otherwise it will be marked as 0 (for details, please refer to Figure 6 As shown), the original event element can be obtained by intercepting the corresponding position of the original text data.

[0142] Furthermore, after obtaining the original trigger word and the original event element, a first loss function can be constructed based on the original trigger word and the corresponding target trigger word, and a second loss function can be constructed based on the original event element and the corresponding target event element. Both the first loss function and the second loss function can adopt a binary cross entropy loss function, which can be specifically shown in the following formulas (10) and (11):

[0143]

[0144]

[0145] Among them, loss trigger is the first loss function, loss argument is the second loss function, as well as is the known correct classification label corresponding to the target trigger word, as well as is the probability of output, n is the number of characters in the original text data (the length of the input sequence); as well as is the known correct classification label corresponding to the target event element, as well as is the output probability, m is the overall length of the input sequence, and k is the length of the question sentence. At the same time, when calculating the second loss function, the loss of the question sentence is not considered.

[0146] Finally, after obtaining the first loss function and the second loss function, the first loss function and the second loss function can be summed to obtain the target loss function loss, which can be specifically shown in the following formula (12):

[0147] loss=loss trigger +loss argument ; Formula (12)

[0148] Finally, the parameters included in the event extraction model to be trained are adjusted according to the target loss function to obtain the target event extraction model.

[0149] Furthermore, in order to further improve the accuracy of the extracted events, an event attribute extraction model can be added. Therefore, in order to extract event attributes based on the event extraction model, the event attribute extraction model to be trained needs to be trained. Figure 7 As shown, the specific training process may include steps S710 to S730.

[0150] In step S710, the original event polarity and the original event tense of the original text data are calculated based on the original trigger word and the event attribute extraction model to be trained;

[0151] In step S720, a third loss function is constructed based on the target event polarity, target event tense, original event polarity, and original event tense of the original text data;

[0152] In step S730, the parameters included in the attribute extraction model to be trained are adjusted according to the third loss function to obtain a target attribute extraction model.

[0153] Below, steps S710 to S730 will be explained and illustrated. Specifically, event attributes include event polarity and event tense, among which event polarity and event tense are important bases for measuring whether an event actually occurred; through event polarity, events can be divided into positive events, negative events and possible events; through event tense, events can be divided into events that occurred in the past, events that are happening now, events that will happen, and other events whose tense cannot be determined. Based on this, first, the original trigger words and the original text data can be spliced into sentence pairs, and then input into the DWF-Bert (third dynamic weight fusion Bert layer) of the event attribute extraction model to be trained to obtain the encoding corresponding to the sentence pair, and then the encoding corresponding to the sentence pair is respectively input into the third fully connected layer and the fourth fully connected layer, thereby obtaining the original event polarity and the original event tense. That is, the [CLS] flag vector is used to connect two fully connected layers to classify tense and polarity respectively. The two tasks share the DWF-BERT network and are optimized at the same time. For details, please refer to Figure 8 As shown. Among them, the event polarity classification result output by the event attribute extraction model to be trained is And the event temporal classification results They can be expressed as the following formulas (13) and (14):

[0154]

[0155]

[0156] Among them, W p and W t is the preset trainable parameter matrix (parameter), W p ∈R K×H , W t ∈R J×H , K is the number of label types of event polarity, J is the number of label types of event temporality, H is the dimension of the network hidden layer, b p and bt is the bias term (parameter), h DWF-CLS The encoding vector output by DWF-Bert is obtained by concatenating the original trigger words and original text data into sentence pairs.

[0157] Furthermore, after obtaining the original event polarity and original event tense, it is necessary to construct a third loss function and then train the event attribute extraction model to be trained based on the third loss function. The third loss function can be specifically shown in the following formula (15):

[0158]

[0159] Among them, L is the third loss function, as well as is the classification result of the original event polarity and the classification result of the original event tense, as well as is the correct classification label corresponding to the target event polarity and target event tense.

[0160] The following will be combined Figure 2-Figure 8 , steps S110 to S130 are explained and illustrated.

[0161] In step S110 , data to be extracted is acquired, and the target trigger word extraction model included in the target event extraction model is used to extract the current trigger word included in the data to be extracted.

[0162] Specifically, the data to be extracted can be obtained from a database or platform. The data to be extracted can include foreign news events, and of course can also include other text data. This example does not impose any special restrictions on this. After obtaining the data to be extracted, the target trigger word extraction model included in the target event extraction model can be used to extract the current trigger word included in the data to be extracted. Specifically, refer to Figure 9 As shown, using the target trigger word extraction model included in the target event extraction model, extracting the current trigger word included in the data to be extracted may include steps S910 to S940.

[0163] In step S910, the first dynamic weight fusion Bert layer included in the target trigger word extraction model included in the target event model is used to calculate the semantic text vector of the data to be extracted and the importance of each semantic text vector to the text to be extracted, and obtain the first current encoding vector based on the semantic text vector and the importance; wherein, the first dynamic weight fusion Bert layer includes multiple Transformer models.

[0164] In this example embodiment, first, word embedding is performed on the data to be extracted to obtain the character embedding vector, character embedding matrix and position embedding matrix of the data to be extracted; secondly, an embedding vector is generated based on the character embedding vector, character embedding matrix and position embedding matrix, and the embedding vector is input into the first text semantic vector of the first Transformer model; then, the first text semantic vector is input into other Transformer models to obtain text semantic vectors corresponding to the other Transformer models; wherein, in the other Transformer models, the output of the previous Transformer model is the input of the next Transformer model corresponding thereto; finally, the importance of each Transformer model to the data to be extracted is calculated, and the first current encoding vector is obtained based on each importance and the embedding vector and the text semantic vector.

[0165] Among them, obtaining the first current encoding vector based on each of the importance levels, the embedding vector, and the text semantic vector can specifically include: first, splicing each of the importance levels and normalizing the spliced importance levels; second, obtaining the first current encoding vector based on the normalized importance levels, the embedding vector, and the text semantic vector.

[0166] In step S920, the first current encoding vector is calculated using the first fully connected layer included in the target trigger word extraction model to obtain the confidence that each character included in the data to be extracted belongs to the current trigger word;

[0167] In step S930, the starting position and the ending position of the current trigger word are determined according to whether the confidence level is greater than a first preset threshold;

[0168] In step S940, the starting position and the ending position of the current trigger word are marked with a first preset identifier, and characters corresponding to the positions marked with the first preset identifier are intercepted to generate the current trigger word.

[0169] The following explains and illustrates steps S910 to S940. First, the sentence X of the text to be extracted is encoded into a word embedding matrix W using the first Transformer model. t and the position embedding matrix W p ; Where X=(x1,x2,...,x n); Then, the word embedding matrix and the position embedding matrix are added together as the total embedding vector h0, and then the embedding vector h0 is passed through other Transformer models to obtain multiple text semantic representation vectors h l (e.g., the first text semantic vector, the second text semantic vector, etc.), which can be specifically shown as follows:

[0170] h0=XW t +W p ; Formula (1)

[0171] h l =Transformer(h l-1 ),l∈[1,N]; Formula (2)

[0172] Among them, h l is the text semantic vector of each layer, that is, the output of the l-th layer Transformer network.

[0173] At the same time, in order to effectively utilize the information of each layer in Bert, first, a fully connected unit (DenseUnit) can be used to learn the importance of each Transformer μ, and then the results of all Transformer layers are weighted and superimposed to obtain the final semantic representation, which can be shown in the following formula (3):

[0174] μ l =σ(Dense unit＝1 (h l )); Formula (3)

[0175] Among them, σ is the ReLu activation function; secondly, the weights of each layer are spliced and normalized with the softmax function to obtain a 1*L weight vector μ L , which can be specifically expressed as the following formula (4):

[0176] μ L =softmax(concatenate([μ1,μ2,...,μ l ])); Formula (4)

[0177] Then, all the Transformer results in BERT are concatenated to obtain a fused representation, which can be specifically expressed as follows:

[0178] h L =concatenate([h1,h2,...,h l ]); Formula (5)

[0179] Finally, the weighted fusion obtained by formula (4) can be used to obtain the final fusion semantic representation h DWT , that is, the first current encoding vector, which can be specifically expressed as follows:

[0180] h DWF =matmul([μ L ,h L ]); Formula (6)

[0181] Next, the first current encoding vector is passed through two Dense layers (the first fully connected layer) using the Sigmoid activation function. The final output sequence value at each position is the confidence level of the entity's start and end positions. Here, the positions with a confidence level greater than 0.45 are taken as the entity's start and end positions. The corresponding positions of the data to be extracted are intercepted to obtain the current trigger word.

[0182] The specific calculation process of the confidence can be shown in the following formula (7) and formula (8):

[0183]

[0184]

[0185] Among them, x i =h DWF [i], that is, the i-th vector in the first current encoding vector, σ is the sigmoid activation function, W start and W end The weight of the target trigger word extraction model, b s and b e is the corresponding bias term, as well as They are the confidence levels of the starting and ending positions of the i-th character of the data to be extracted as the trigger word. When the confidence level is greater than the preset threshold (0.45), the corresponding position will be marked as 1, otherwise it will be marked as 0.

[0186] In step S120, a current question sentence is generated according to the current trigger word.

[0187] For example, in the example sentence (data to be extracted) "On the evening of March 27th, Beijing time, British Prime Minister Boris Johnson was confirmed to be infected with the new coronavirus", the current trigger word extracted is "infection", so the current question sentence query constructed is: "Find the subject, object, time and place related to #infection#";

[0188] In step S130, the current event element corresponding to the current trigger word is extracted according to the current question sentence and the target element extraction model included in the target event extraction model; wherein the target trigger word extraction model and the target element extraction model have the same model structure and weight, and are connected in a cascade manner.

[0189] In this example embodiment, reference Figure 10 As shown, based on the current question sentence and the target element extraction model included in the target event extraction model, extracting the current event element corresponding to the current trigger word may include steps S1010 to S1040.

[0190] In step S1010, a first sentence pair is generated based on the data to be extracted and the current question sentence, and the first sentence pair is encoded using the second dynamic weight fusion Bert layer included in the target element extraction model included in the target event model to obtain a second current encoding vector.

[0191] In step S1020, the boundary of the current event element in the data to be extracted is calculated based on the second current encoding vector, and the elements included in the boundary are classified to obtain the probability that the characters at all positions from the head position pointer to the tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element; wherein, each boundary includes a head position pointer and a tail position pointer.

[0192] Among them, the elements included in the boundary are classified to obtain the probability that the characters at all positions from the head position pointer to the tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element. Specifically, it can include: first, based on multiple binary classification networks, the roles to which the elements included in the boundary belong are classified to obtain the role labels of the elements; secondly, a head position matrix and a tail position matrix are generated according to the head position pointer and the tail position pointer of all role labels; wherein each row in the head position matrix and the tail position matrix represents a role type, and each column corresponds to each character of the data to be extracted; finally, the head position matrix and the tail position matrix are calculated using the second fully connected layer included in the target element extraction model to obtain the probability that the characters at all positions from the head position pointer to the tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element.

[0193] In step S1030, the starting position and the ending position of the current event element are determined according to whether the probability is greater than a second preset threshold, and the starting position and the ending position of the current event element are marked with a second preset identifier.

[0194] In step S1040, the character corresponding to the position with the second preset identification mark is intercepted to generate the current event element.

[0195] Below, steps S1010 to S1040 will be explained and illustrated. Specifically, after obtaining the question sentence, the query and the original text data can be combined into a first sentence pair; wherein, the sentence pair can be, for example: "[CLS] Find the subject, object, time and place related to #infection# [SEP] On the evening of March 27, Beijing time, British Prime Minister Boris Johnson was diagnosed with new coronary pneumonia [SEP]"; then, the first sentence pair is sent to the second dynamic weight fusion Bert layer in a dual input manner to obtain the second current encoding vector. The specific generation process of the second current encoding vector is similar to that of the first current encoding vector, and will not be repeated here.

[0196] At the same time, after obtaining the second current encoding vector, in order to extract multiple event elements at the same time, it is first necessary to generate two 0 / 1 sequences through two binary classification networks to determine the boundary span of the event element in the sequence (each span is determined by a head position pointer start and a tail position pointer end); secondly, multiple binary classification networks are used to classify the element span into roles; wherein, each character in the input sequence can be represented as the start and end position of a certain element, and the span composed of text between any two characters can be represented as any event role; then, the start and end pointer vectors of all labels are combined together to obtain the head position matrix S s And the tail position matrix S e , where S s and S e Each row in the text represents a character type, and each column corresponds to a character in the text. The text uses multiple binary classification networks to predict the probability that all positions of the start and end pointer vectors corresponding to each character in the input sequence are 0 / 1 to determine the start and end positions of the element and the character type. The entire task can be regarded as multi-label classification for each character in the input sequence, and the probability value of the i-th character being predicted as the start and end position of the element of character r is as well as They can be expressed as the following formulas (9) and (10):

[0197]

[0198]

[0199] in, is the starting position probability, is the probability of the terminal position, x i=h DWF [i], that is, the encoding vector of the i-th character in the sentence pair after BERT encoding, σ is the sigmoid activation function, as well as are preset trainable weights (parameters), as well as is the corresponding bias item (parameter). When the probability is greater than the preset threshold (0.45), the corresponding position will be marked as 1, otherwise it will be marked as 0. The target event element can be obtained by intercepting the corresponding position of the data to be extracted.

[0200] It should be further explained here that the weights in the second dynamic weight fusion Bert layer change as the weights in the first dynamic weight fusion Bert layer change, thereby further improving the correlation between the current trigger word and the current event element, thereby improving the accuracy of the event text.

[0201] Furthermore, in order to further improve the accuracy of event text, the text extraction method may also include: calculating the current event polarity and current event tense of the data to be extracted based on the current trigger word and the target event attribute extraction model included in the target event extraction model.

[0202] Specifically, refer to Figure 11 As shown, calculating the current event polarity and current event tense of the data to be extracted based on the current trigger word and the target event attribute extraction model included in the target event extraction model may include steps S1110 to S1140.

[0203] In step S1110, a second sentence pair is generated according to the current trigger word and the data to be extracted;

[0204] In step S1120, the second sentence pair is encoded using the third dynamic weight fusion Bert layer included in the target attribute extraction model to obtain a third current encoding vector;

[0205] In step S1130, the third current encoding vector is calculated using the third fully connected layer included in the target attribute extraction model to obtain the current event polarity of the data to be extracted;

[0206] In step S1140, the third current encoding vector is calculated using the fourth fully connected layer included in the target attribute extraction model to obtain the current event tense of the data to be extracted.

[0207] The following will explain and illustrate steps S1110 to S1140. Specifically, first, the target trigger word and the data to be extracted can be spliced into the second sentence pair, and then the second sentence pair is input into the third dynamic weight fusion Bert layer. The [CLS] flag vector is used to connect two fully connected layers to classify the tense and polarity respectively. The two tasks share the DWF-BERT network and are optimized at the same time. Among them, the classification result of the current event polarity is And the classification results of the current event tense They can be expressed as the following formulas (13) and (14):

[0208]

[0209]

[0210] Among them, W p and W t is the parameter matrix of the target event polarity extraction model, W p ∈R K×H , W t ∈R J×H , K is the number of event polarity types, J is the number of event temporal types, H is the dimension of the network hidden layer, b p and b t is the bias term (parameter), h DWF-CLS is the third current encoding vector.

[0211] Furthermore, after obtaining the current event polarity and the current event tense, the event text included in the data to be extracted can be generated based on the current trigger word, the current event element, the current event polarity and the current event tense; of course, other applications can also be performed based on the current trigger word, the current event element, the current event polarity and the current event tense, and this example does not impose any special restrictions on this.

[0212] The following, combined Figure 12 The text extraction method disclosed in the present invention is further explained and illustrated.

[0213] First, input the text sequence: CLS Beijing time, March 27th evening, British Prime Minister Boris Johnson was diagnosed with new coronary pneumonia SEP, through the first dynamic weight fusion Bert layer 301 and the first fully connected layer 302, get the trigger word "infection", then, based on "infection", generate the question sentence "find the subject, object, time and place related to #infection#", and form the first sentence pair: [CLS] Find the subject, object, time and place related to #infection# On the evening of March 27th Beijing time, British Prime Minister Boris Johnson was diagnosed with new coronary pneumonia [SEP], through the second dynamic weight fusion Bert layer 303 and the second fully connected layer 304, get the event element corresponding to the trigger word: subject: Prime Minister Boris Johnson, object: new coronary pneumonia; time: March 27th Beijing time.

[0214] The text extraction method provided in the present disclosure can simultaneously extract event trigger words and event elements through a cascade-structured reading comprehension event extraction model; at the same time, adaptive weights are used to weightedly fuse the results of multiple Transformer layers of the pre-trained language model BERT, effectively improving the model's effectiveness.

[0215] In addition, in response to problems such as active-passive conversion and subject-object sharing of multiple events in event extraction, this paper proposes a text extraction method based on a reading comprehension event extraction model with a cascade structure. This scheme divides the entire model into a two-level structure. At the first level, a dual-pointer network is used to identify trigger words in the text. At the second level, based on the obtained trigger words, a query is constructed, and multiple binary classification networks are used to annotate the answers, i.e., event elements. The two-level structures share a BERT weight and are optimized at the same time, which further improves the accuracy of the event text.

[0216] The present disclosure also provides a text extraction device. Figure 13 As shown, the text extraction device may include a first extraction module 1310, a second extraction module 1320, and an event generation module 1330.

[0217] The first extraction module 1310 may be used to obtain data to be extracted and extract the current trigger word included in the data to be extracted using the target trigger word extraction model included in the target event extraction model;

[0218] The question sentence generating module 1320 may be configured to generate a current question sentence based on the current trigger word;

[0219] The second extraction module 1330 can be used to extract the current event element corresponding to the current trigger word based on the current question sentence and the target element extraction model included in the target event extraction model; wherein the target trigger word extraction model and the target element extraction model have the same model structure and weight, and are connected in a cascade manner.

[0220] In an exemplary embodiment of the present disclosure, the target trigger word extraction model included in the target event extraction model is used to extract the current trigger word included in the data to be extracted, including:

[0221] Utilizing the first dynamic weight fusion Bert layer included in the target trigger word extraction model included in the target event model, the semantic text vectors of the data to be extracted and the importance of each semantic text vector to the text to be extracted are calculated, and a first current encoding vector is obtained based on the semantic text vectors and the importance.

[0222] Calculating the first current encoding vector using the first fully connected layer included in the target trigger word extraction model to obtain a confidence score that each character included in the data to be extracted belongs to the current trigger word;

[0223] Determining the starting position and the ending position of the current trigger word according to whether the confidence level is greater than a first preset threshold;

[0224] The starting position and the ending position of the current trigger word are marked with a first preset identifier, and characters corresponding to the positions marked with the first preset identifier are intercepted to generate the current trigger word.

[0225] In an exemplary embodiment of the present disclosure, the first dynamic weight fusion Bert layer includes multiple Transformer models;

[0226] Among them, using the first dynamic weight fusion Bert layer included in the target trigger word extraction model included in the target event model, calculating the semantic text vector of the data to be extracted and the importance of each semantic text vector to the text to be extracted, and obtaining the first current encoding vector according to the semantic text vector and the importance, including:

[0227] Performing word embedding on the data to be extracted to obtain a word embedding vector, a word embedding matrix, and a position embedding matrix of the data to be extracted;

[0228] Generate an embedding vector according to the word embedding vector, the word embedding matrix, and the position embedding matrix, and input the embedding vector into a first Transformer model to generate a first text semantic vector;

[0229] Inputting the first text semantic vector into another Transformer model to obtain a text semantic vector corresponding to the other Transformer model; wherein, in the other Transformer model, the output of the previous Transformer model is the input of the next corresponding Transformer model;

[0230] The importance of each Transformer model to the data to be extracted is calculated, and the first current encoding vector is obtained based on each importance, the embedding vector, and each text semantic vector.

[0231] In an exemplary embodiment of the present disclosure, obtaining the first current encoding vector according to each of the importance levels, the embedding vector, and each of the text semantic vectors includes:

[0232] Splicing the importance levels, and normalizing the spliced importance levels;

[0233] The first current encoding vector is obtained according to the normalized importance levels, the embedding vector, and the text semantic vectors.

[0234] In an exemplary embodiment of the present disclosure, extracting a current event element corresponding to the current trigger word according to the current question sentence and the target element extraction model included in the target event extraction model includes:

[0235] Generate a first sentence pair according to the data to be extracted and the current question sentence, and encode the first sentence pair using a second dynamic weight fusion Bert layer included in the target element extraction model included in the target event model to obtain a second current encoding vector;

[0236] Calculating boundaries of the current event element in the data to be extracted based on the second current encoding vector, and classifying the elements included in the boundaries to obtain probabilities that characters at all positions from a head position pointer to a tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element; wherein each boundary includes a head position pointer and a tail position pointer;

[0237] Determining the starting position and the ending position of the current event element according to whether the probability is greater than a second preset threshold, and marking the starting position and the ending position of the current event element with a second preset identifier;

[0238] The character corresponding to the position with the second preset identification mark is intercepted to generate the current event element.

[0239] In an exemplary embodiment of the present disclosure, the elements included in the boundary are classified to obtain the probability that the characters at all positions from the head position pointer to the tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element, including:

[0240] Classifying the roles of the elements included in the boundary based on multiple binary classification networks to obtain role labels of the elements;

[0241] Generate a head position matrix and a tail position matrix according to the head position pointers and the tail position pointers of all the role tags; wherein each row in the head position matrix and the tail position matrix represents a role type, and each column corresponds to each character of the data to be extracted;

[0242] The head position matrix and the tail position matrix are calculated using the second fully connected layer included in the target element extraction model to obtain the probability that the characters at all positions from the head position pointer to the tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element.

[0243] In an exemplary embodiment of the present disclosure, the text extraction device may further include:

[0244] The third extraction module can be used to calculate the current event polarity and the current event tense of the data to be extracted based on the current trigger word and the target event attribute extraction model included in the target event extraction type.

[0245] In an exemplary embodiment of the present disclosure, calculating the current event polarity and the current event tense of the data to be extracted based on the current trigger word and the target event attribute extraction model included in the target event extraction model includes:

[0246] Generate a second sentence pair according to the current trigger word and the data to be extracted;

[0247] Encoding the second sentence pair using a third dynamic weight fusion Bert layer included in the target attribute extraction model to obtain a third current encoding vector;

[0248] Calculating the third current encoding vector using a third fully connected layer included in the target attribute extraction model to obtain a current event polarity of the data to be extracted;

[0249] The third current encoding vector is calculated using the fourth fully connected layer included in the target attribute extraction model to obtain the current event tense of the data to be extracted.

[0250] In an exemplary embodiment of the present disclosure, the text extraction device further includes:

[0251] The first calculation module can be used to obtain original text data and calculate the original trigger words included in the original text data according to the trigger word extraction model to be trained included in the event extraction model to be trained;

[0252] a fourth extraction module, which may be configured to generate an original question sentence based on the original trigger word, and extract original event elements included in the original text data based on the original question sentence and the element extraction model to be trained included in the event extraction model to be trained;

[0253] A first loss function construction module can be used to construct a first loss function based on the target trigger word and the original trigger word of the original text data, and to construct a second loss function based on the target event element and the original event element of the original news event;

[0254] The first parameter adjustment module can be used to adjust the parameters included in the event extraction model to be trained according to the first loss function and the second loss function to obtain a target event extraction model.

[0255] In an exemplary embodiment of the present disclosure, the parameters included in the event extraction model to be trained are adjusted according to the first loss function and the second loss function to obtain a target event extraction model, including:

[0256] Performing a sum operation on the first loss function and the second loss function to obtain a target loss function;

[0257] The parameters included in the event extraction model to be trained are adjusted according to the target loss function to obtain a target event extraction model.

[0258] In an exemplary embodiment of the present disclosure, the text extraction device further includes:

[0259] A second calculation module may be used to calculate the original event polarity and the original event tense of the original text data based on the original trigger word and the event attribute extraction model to be trained;

[0260] The second loss function calculation module can be used to construct a third loss function according to the target event polarity, target event tense and original event polarity and original event tense of the original text data;

[0261] The second parameter training and adjustment block can be used to adjust the parameters included in the attribute extraction model to be trained according to the third loss function to obtain a target attribute extraction model.

[0262] The specific details of each module in the above text extraction device have been described in detail in the corresponding text extraction method, so they will not be repeated here.

[0263] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to an embodiment of the present invention, the features and functions of two or more modules or units described above can be concretized in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided into multiple modules or units to be concretized.

[0264] Furthermore, although the steps of the method of the present invention are described in a particular order in the accompanying drawings, this does not require or imply that the steps must be performed in this particular order, or that all steps must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps.

[0265] In an exemplary embodiment of the present invention, an electronic device capable of implementing the above method is also provided.

[0266] Those skilled in the art will appreciate that various aspects of the present invention may be implemented as systems, methods, or program products. Therefore, various aspects of the present invention may be implemented in the following forms: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which may be collectively referred to herein as "circuits," "modules," or "systems."

[0267] Refer to the following Figure 14 An electronic device 1400 according to this embodiment of the present invention will be described. Figure 14 The electronic device 1400 shown is merely an example and should not limit the functions and scope of use of the embodiments of the present invention.

[0268] like Figure 14 As shown, electronic device 1400 is implemented as a general-purpose computing device. Components of electronic device 1400 may include, but are not limited to, the aforementioned at least one processing unit 1410, the aforementioned at least one storage unit 1420, a bus 1430 connecting various system components (including storage unit 1420 and processing unit 1410), and a display unit 1440.

[0269] The storage unit stores program codes, which can be executed by the processing unit 1410, so that the processing unit 1410 performs the steps according to various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 1410 can perform the following steps: Figure 1 Step S110 shown in: obtaining the data to be extracted, and using the target trigger word extraction model included in the target event extraction model to extract the current trigger word included in the data to be extracted; step S120: generating a current question sentence according to the current trigger word; step S130: extracting the current event element corresponding to the current trigger word according to the current question sentence and the target element extraction model included in the target event extraction model; wherein the target trigger word extraction model and the target element extraction model have the same model structure and parameters, and are connected in a cascade manner.

[0270] The storage unit 1420 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 14201 and / or a cache memory unit 14202 , and may further include a read-only memory unit (ROM) 14203 .

[0271] The storage unit 1420 may also include a program / utility 14204 having a set (at least one) of program modules 14205, such program modules 14205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0272] The bus 1430 may represent one or more of several types of bus structures, including a memory bus or memory controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0273] Electronic device 1400 can also communicate with one or more external devices 1500 (e.g., a keyboard, pointing device, Bluetooth device, etc.), one or more devices that enable a user to interact with electronic device 1400, and / or any device that enables electronic device 1400 to communicate with one or more other computing devices (e.g., a router, modem, etc.). Such communication can occur via input / output (I / O) interface 1450. Furthermore, electronic device 1400 can communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network such as the Internet) via network adapter 1460. As shown, network adapter 1460 communicates with other modules of electronic device 1400 via bus 1430. It should be understood that, although not shown, other hardware and / or software modules can be used in conjunction with electronic device 1400, including but not limited to microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0274] Through the description of the above embodiments, it is easy for those skilled in the art to understand that the example embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present invention can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the embodiments of the present invention.

[0275] In exemplary embodiments of the present invention, a computer-readable storage medium is also provided, storing a program product capable of implementing the methods described above. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product comprising program code. When the program product is executed on a terminal device, the program code is configured to cause the terminal device to perform the steps according to various exemplary embodiments of the present invention described in the "Exemplary Methods" section above.

[0276] According to an embodiment of the present invention, a program product for implementing the above-mentioned method can be a portable compact disc read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, and the program can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0277] The program product may be implemented in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or component, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof.

[0278] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0279] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wireline, optical fiber cable, RF, etc., or any suitable combination of the foregoing.

[0280] The program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, and the like, as well as conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a stand-alone software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0281] Furthermore, the above-described figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention and are not intended to be limiting. It is readily understood that the processes illustrated in the above-described figures do not indicate or limit the temporal order of these processes. Furthermore, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0282] Other embodiments of the present invention will readily occur to those skilled in the art after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of the invention that follow from the general principles of the invention and include common knowledge or customary techniques in the art not invented herein. The specification and examples are to be considered as exemplary only, with the true scope and spirit of the invention being indicated by the claims.

Claims

1. A text extraction method, characterized in that: include: Acquire the data to be extracted, and use the target trigger word extraction model included in the target event extraction model to extract the current trigger word included in the data to be extracted; wherein, the current trigger word is obtained in the following manner: using the first dynamic weight fusion Bert layer included in the target trigger word extraction model included in the target event model, calculate the semantic text vector of the data to be extracted and the importance of each semantic text vector to the data to be extracted, and obtain a first current encoding vector based on the semantic text vector and the importance; use the first fully connected layer included in the target trigger word extraction model to calculate the first current encoding vector to obtain the confidence that each character included in the data to be extracted belongs to the current trigger word; determine the starting position and ending position of the current trigger word based on whether the confidence is greater than a first preset threshold; use a first preset identifier to mark the starting position and ending position of the current trigger word, and intercept the characters corresponding to the positions marked with the first preset identifier to generate the current trigger word; Generate a current question sentence according to the current trigger word; According to the current question sentence and the target element extraction model included in the target event extraction model, the current event element corresponding to the current trigger word is extracted; wherein the target trigger word extraction model and the target element extraction model have the same model structure and parameters, and are connected in a cascade manner.

2. The text extraction method according to claim 1, characterized in that The first dynamic weight fusion Bert layer includes multiple Transformer models; Among them, using the first dynamic weight fusion Bert layer included in the target trigger word extraction model included in the target event model, calculating the semantic text vector of the data to be extracted and the importance of each semantic text vector to the data to be extracted, and obtaining the first current encoding vector according to the semantic text vector and the importance, including: Performing word embedding on the data to be extracted to obtain a word embedding vector, a word embedding matrix, and a position embedding matrix of the data to be extracted; Generate an embedding vector according to the word embedding vector, the word embedding matrix, and the position embedding matrix, and input the embedding vector into a first Transformer model to generate a first text semantic vector; Inputting the first text semantic vector into other Transformer models to obtain text semantic vectors corresponding to the other Transformer models; wherein, in the other Transformer models, the output of the previous Transformer model is the input of the next corresponding Transformer model; The importance of each Transformer model to the data to be extracted is calculated, and the first current encoding vector is obtained based on each importance, the embedding vector, and each text semantic vector.

3. The text extraction method according to claim 2, characterized in that Obtaining the first current encoding vector according to each of the importance levels, the embedding vector, and each of the text semantic vectors includes: Splicing the importance levels, and normalizing the spliced importance levels; The first current encoding vector is obtained according to the normalized importance levels, the embedding vector, and the text semantic vectors.

4. The text extraction method according to claim 1, wherein: Extracting a current event element corresponding to the current trigger word according to the current question sentence and the target element extraction model included in the target event extraction model includes: Generate a first sentence pair according to the data to be extracted and the current question sentence, and encode the first sentence pair using a second dynamic weight fusion Bert layer included in the target element extraction model included in the target event model to obtain a second current encoding vector; Calculating boundaries of the current event element in the data to be extracted based on the second current encoding vector, and classifying the elements included in the boundaries to obtain probabilities that characters at all positions from a head position pointer to a tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element; wherein each boundary includes a head position pointer and a tail position pointer; Determining the starting position and the ending position of the current event element according to whether the probability is greater than a second preset threshold, and marking the starting position and the ending position of the current event element with a second preset identifier; The character corresponding to the position with the second preset identification mark is intercepted to generate a current event element.

5. The text extraction method according to claim 4, characterized in that: Classifying the elements included in the boundary, obtaining the probability that the characters at all positions from the head position pointer to the tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element, including: Classifying the roles of the elements included in the boundary based on multiple binary classification networks to obtain role labels of the elements; Generate a head position matrix and a tail position matrix according to the head position pointers and the tail position pointers of all the role tags; wherein each row in the head position matrix and the tail position matrix represents a role type, and each column corresponds to each character of the data to be extracted; The head position matrix and the tail position matrix are calculated using the second fully connected layer included in the target element extraction model to obtain the probability that the characters at all positions from the head position pointer to the tail position pointer of the elements included in each boundary belong to the start and end positions of the current event element.

6. The text extraction method according to claim 1, characterized in that The text extraction method further includes: The current event polarity and the current event tense of the data to be extracted are calculated according to the current trigger word and the target event attribute extraction model included in the target event extraction type.

7. The text extraction method according to claim 6, characterized in that: Calculating the current event polarity and the current event tense of the data to be extracted based on the current trigger word and the target event attribute extraction model included in the target event extraction model, including: Generate a second sentence pair according to the current trigger word and the data to be extracted; Encoding the second sentence pair using a third dynamic weight fusion Bert layer included in the target attribute extraction model to obtain a third current encoding vector; Calculating the third current encoding vector using a third fully connected layer included in the target attribute extraction model to obtain a current event polarity of the data to be extracted; The third current encoding vector is calculated using the fourth fully connected layer included in the target attribute extraction model to obtain the current event tense of the data to be extracted.

8. The text extraction method according to claim 1, characterized in that: The text extraction method further includes: Obtaining original text data, and calculating original trigger words included in the original text data according to the trigger word extraction model to be trained included in the event extraction model to be trained; generating an original question sentence according to the original trigger word, and extracting original event elements included in the original text data according to the original question sentence and the element extraction model to be trained included in the event extraction model to be trained; Constructing a first loss function based on the target trigger word and the original trigger word of the original text data, and constructing a second loss function based on the target event element and the original event element of the original text data; The parameters included in the event extraction model to be trained are adjusted according to the first loss function and the second loss function to obtain a target event extraction model.

9. The text extraction method according to claim 8, characterized in that: Adjusting the parameters included in the event extraction model to be trained according to the first loss function and the second loss function to obtain a target event extraction model, including: Perform a sum operation on the first loss function and the second loss function to obtain the target loss function; The parameters included in the event extraction model to be trained are adjusted according to the target loss function to obtain a target event extraction model.

10. The text extraction method according to claim 8, characterized in that: The text extraction method further includes: Calculating the original event polarity and the original event tense of the original text data according to the original trigger word and the event attribute extraction model to be trained; Constructing a third loss function based on the target event polarity, target event tense, original event polarity, and original event tense of the original text data; The parameters included in the attribute extraction model to be trained are adjusted according to the third loss function to obtain a target attribute extraction model.

11. A text extraction device, characterized in that: include: A first extraction module is used to obtain data to be extracted, and use the target trigger word extraction model included in the target event extraction model to extract the current trigger word included in the data to be extracted; wherein, the current trigger word is obtained in the following manner: using the first dynamic weight fusion Bert layer included in the target trigger word extraction model included in the target event model to calculate the semantic text vector of the data to be extracted and the importance of each semantic text vector to the data to be extracted, and obtain a first current encoding vector based on the semantic text vector and the importance; using the first fully connected layer included in the target trigger word extraction model to calculate the first current encoding vector, and obtain the confidence that each character included in the data to be extracted belongs to the current trigger word; judging the starting position and ending position of the current trigger word based on whether the confidence is greater than a first preset threshold; marking the starting position and ending position of the current trigger word with a first preset identifier, and intercepting the characters corresponding to the positions marked with the first preset identifier to generate the current trigger word; A question sentence generating module, configured to generate a current question sentence based on the current trigger word; The second extraction module is used to extract the current event element corresponding to the current trigger word based on the current question sentence and the target element extraction model included in the target event extraction model; wherein the target trigger word extraction model and the target element extraction model have the same model structure and weight, and are connected in a cascade manner.

12. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the text extraction method according to any one of claims 1 to 10 is implemented.

13. An electronic device, characterized in that: include: processor; as well as a memory for storing executable instructions of the processor; The processor is configured to execute the text extraction method according to any one of claims 1 to 10 by executing the executable instructions.

Images