A key event based multi-document summarization generation method

Abstract

The application discloses a kind of multi-document abstract generation methods based on key event, first by internet collection under the same theme multiple media articles, and in document set basis, according to artificial rule, standard abstract is generated to construct sample dataset;Then the dataset is preprocessed, and the input data of training model is generated;Then construct the sequence-to-sequence multi-document abstract generation model based on key event fusion;Finally, based on the model after training, the output model is constructed, and the document set to be summarized is automatically summarized using the output model.The application uses event extraction technology to extract key events containing dynamic and static information from multiple documents to mine the relationship of multiple documents, which can guide the abstract generation process at multiple levels, thereby improving the information coverage and factual consistency of the abstract result, highlighting the event information in the original text, and enhancing the logicality of the abstract.

Description

Technical Field

[0001] This invention relates to a method for generating multi-document summaries based on key events, belonging to the fields of Internet and artificial intelligence technology. Background Technology

[0002] With the rapid development of internet technology, online media has become an important platform for people to quickly obtain and publish information, leading to an explosive growth in various types of text, including news. Therefore, comprehensively analyzing multiple documents on the same topic, extracting, refining, and generating important information containing key events, and aggregating it into concise and clear summaries for readers can effectively help readers quickly and conveniently understand the main content of media reports, improving their information acquisition efficiency. The goal of multi-document summarization is to summarize multiple documents on the same topic. However, multiple documents contain a large amount of redundant information, making it very time-consuming and difficult to find key information and generate logically coherent and concise summaries. Therefore, in multi-document summarization, extracting multiple key events, including both dynamic and static information, from the documents to guide summary generation can effectively model the relationship between documents and events, thereby generating summaries containing important event content.

[0003] Sequence-to-sequence (seq2seq) generative summarization models are currently the mainstream models in the field of text summarization. These models consist of an encoder and a decoder. The encoder encodes the input text sequence into a hidden state vector, and the decoder decodes this hidden state vector into a summary for output. However, traditional seq2seq models use attention mechanisms to focus on important encoded information. But in summarization tasks, there is often a significant difference in length between the original text and the target summary. Attention weights are easily scattered across a large amount of redundant information, resulting in generated summaries with problems such as missing important information, inconsistencies with the context and topic, and inconsistencies with the facts.

[0004] To this end, this invention introduces event extraction technology on the basis of the seq2seq model. It uses the OmniEvent tool to extract events from the text, identifies key event information containing key event trigger words and entity parameters in multiple documents, encodes and constructs graphs, and uses a graph attention mechanism to integrate event information and document information. Furthermore, it designs a hierarchical attention mechanism to guide the decoding process with key events, effectively guiding the generation of summaries. Summary of the Invention

[0005] To address the problems and shortcomings of existing technologies, this invention provides a multi-document summarization method based on key events. This method can extract trigger words and entities of key events and improve the lack of effective control in the summarization process by integrating key events, thereby enhancing the topic consistency, information coverage, and factual matching of the summarization results.

[0006] To achieve the above-mentioned objectives, the present invention provides a multi-document summarization method based on key events, the model overview diagram of which is shown below. Figure 1 As shown, this method first extracts events from the text using the OmniEvent tool, identifies key events that occur frequently in the document through statistical analysis, and constructs a graph of the document and events. Next, it builds a hierarchical Transformer multi-document encoder and a Transformer event encoder, using multiple documents and key event information as input to obtain document representations and event representations, respectively. Then, a graph attention layer interacts the key event information with the document representation to obtain an event-aware document representation. Finally, the key event representation is integrated into the Transformer decoding layer to guide the generation of multi-document summaries. This method mainly includes four steps, as follows:

[0007] Step 1: Collect multiple documents from the Internet to accumulate a sample dataset; a sample in the dataset includes multiple documents on the same topic and standard abstracts;

[0008] Step 2: Extract event information for each sample in the dataset. Use the OmniEvent toolkit to extract event trigger words and related entities from each sentence, and statistically analyze the events that are mentioned most frequently.

[0009] Step 3: Train a multi-document summarization generation model based on key events. First, obtain document representations from multiple documents using a hierarchical Transformer encoder, and obtain event representations from key events using another Transformer encoder. Then, obtain event-aware document representations using a graph attention layer. Next, use a decoder with a hierarchical decoding layer to let key events guide summary generation. Finally, train the model using cross-entropy loss.

[0010] Step 4: Generate summaries for multiple documents on the same topic to be summarized. For multiple documents to be summarized, firstly, use the OmniEvent toolkit to extract key events. Input the original text of multiple documents and key events into the multi-document summarization generation model trained in step (3) to generate text summaries. This approach can extract key text information, including event trigger words and entities, from multiple dimensions, overcoming the problems of inconsistent topics, incomplete information, and inconsistencies with facts in traditional text summarization methods. It can be applied to the accurate extraction of key events in multiple documents, improving the effect of multi-document summarization.

[0011] Compared with the prior art, the advantages of the present invention are as follows:

[0012] (1) The event-aware multi-document encoding module used in this invention can extract and select key events from multiple documents, and perform multi-level semantic context representation of key dynamic and static information in the text, highlighting key event information in multiple documents and effectively improving the factual consistency and information coverage of the summary results.

[0013] (2) The present invention adopts an attention mechanism that integrates key event information, which can effectively integrate key events at multiple levels and guide the summary generation process from multiple angles, reduce the interference of irrelevant information, effectively improve the factual accuracy and factual logic of the summary results, and highlight the key events and their order, which is especially effective in event-oriented texts such as news. Attached Figure Description

[0014] Figure 1 This is an overview diagram of the model of the present invention.

[0015] Figure 2 This is a flowchart illustrating the processing of an embodiment of the present invention.

[0016] Figure 3 A flowchart for training a multi-document summarization generation model based on key events. Detailed Implementation

[0017] To enhance understanding of the present invention, the invention will be further explained below with reference to specific embodiments.

[0018] Example 1: See Figure 2 , Figure 3 A method for generating summaries by integrating key information, the specific implementation steps of which are as follows:

[0019] Step 1: Accumulate the sample dataset. In this embodiment, multiple documents categorized by topic are first collected from the Internet, and standard summaries are generated according to manually prescribed rules. These documents together constitute the sample dataset D. The rules for generating the standard summaries are as follows: for 30% of the document set, the standard summary is generated by concatenating the titles of each document; for 30% of the document set, the standard summary is generated by concatenating the first two sentences of each document; and for 40% of the media text, a manually written summary is used as the standard summary.

[0020] Step 2, Data Preprocessing: In this embodiment, the document sets under each topic in dataset D are first concatenated. Event extraction is performed using the OmniEvent toolkit. The M most frequently occurring event trigger words are selected as the final key event trigger words of the document set, serving as the dynamic features of the events. Entity mentions related to these trigger words are selected as static features, and they are organized into trigger word sequences and entity sequences respectively. In this embodiment, M is set to 10. For each document, stop words are removed, and it is truncated into a sequence of length 800.

[0021] Step 3: Train the multi-document summarization generation model based on key event information using the dataset D processed in Step 2. This step can be divided into the following sub-steps:

[0022] Sub-step 3-1: Construct the input layer. The input layer receives key event sequences and multi-document sequences as input, and transforms each word sequence into a word vector sequence. The event vector is obtained by fusing and concatenating event trigger words and event entities, resulting in the mapped key event vector sequence E. v , and multiple document vector sequences E d .

[0023] Sub-step 3-2 involves constructing a text encoding layer. In this embodiment, two encoders are used to encode events and documents separately. The event encoder employs a traditional Transformer encoder to encode the key event vector sequence E. v Semantic encoding extraction is performed to obtain the hidden state vector h of the key event vector sequence. v The document encoder employs a hierarchical Transformer encoder, considering both attention calculations within a single document and relationship modeling among multiple documents. It encodes multiple documents according to document and word levels using formulas (1) and (2), inputting the text sequence of each level into x. l Residual connection calculation is performed, and then multi-head pooling is used to obtain the hidden state vector h of the multi-document vector sequence with a unified dimension through formula (3). d Then, as shown in formula (4), the relationship between multiple documents is modeled through an interactive attention mechanism, incorporating information from other documents into the document itself, to obtain a new document representation c. i The final text sequence representation x is then calculated again using a residual network. ij .

[0024] h = LayerNorm(x) l-1 +MHAttn(x l-1 )) (1)

[0025] x l=LayerNorm(h+FFN(h)) (2)

[0026] h d =MHPool(h w1 ,h w2 ,…h wl (3)

[0027] c i =InterAttn(h d1 ,h d2 ,…,h dn (4)

[0028] Sub-step 3-3: Construct an event-aware graph attention layer. In this embodiment, a single graph attention layer is used, treating document and event representations as nodes n of the graph. i Does the event exist in the document as an edge e? ij Attention is calculated by obtaining events and documents of the same length through multi-head attention pooling. A linear transformation using LeakyReLU as the activation function combines the nodes and edges of the graph to obtain z. ij Then, the graph attention coefficient α is obtained by calculating it using formula (5). ij Finally, using formula (6) and a linear transformation with sigmoid as the activation function, the fused node and edge information is calculated to obtain an event-aware document representation u incorporating key event information. i .

[0029]

[0030]

[0031] Sub-steps 3-4 involve constructing the digest decoding layer. This embodiment employs a hierarchical Transformer decoder for digest decoding. First, after entering the Transformer decoding layer, the hidden state vector of the key event sequence is used as the query vector. Attention is then calculated between the query vector and the decoded digest from the decoder to obtain the event pointer. Then, the obtained event pointer is used through formula (7). Provide guidance, and interpret the text representation H to be decoded. w

[0032] Then, attention calculation is performed to obtain r. i Finally, through a linear transformation, the final output representation of the decoded product is obtained using the LeakyReLU activation function according to formula (8). i The formula is as follows:

[0033]

[0034]

[0035] Sub-steps 3-5 involve constructing a probabilistic summarization layer. A linear mapping function and a softmax activation function are used to transform the hidden state vector of the decoding layer into the predicted probability distribution P of the summary words. The calculation formula (9) is shown below:

[0036]

[0037] Among them, W g and b g These are the parameters that the model needs to be trained on.

[0038] Sub-steps 3-6 involve training the model. In this embodiment, all trainable parameters are initialized using random initialization. During training, the Adam optimizer is used for gradient backpropagation to update the model parameters. The initial learning rate is set to 0.001, and the beam search is set to 5. Model training ends when the training loss no longer decreases or the number of training epochs exceeds 100.

[0039] Step 4: Utilize the trained model to build an output model to generate summaries. The output model requires pre-extracting event information as auxiliary input, using the text set to be summarized as input, and then using the summarization layer's input words at each time step as the summary words generated in the previous time step. The initial summary word is a special start marker "". <start>"The summary word at each time step is the word with the highest probability output by the summary probabilistic layer. The output ends when the marker "" is displayed. <end>When the condition is met, stop generating the summary and output the generated summary terms as the predicted summary of the input media text.

[0040] Based on the same inventive concept, embodiments of the present invention also provide a summary generation apparatus based on key events, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the computer program is loaded onto the processor, it implements the above-described multi-document summary generation method based on key events.

[0041] It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. After reading this invention, any modifications of the invention in various equivalent forms by those skilled in the art will fall within the scope defined by the appended claims.< / end> < / start>

Claims

1. A method for generating multi-document summaries based on key events, characterized in that, The method includes the following steps: Step 1: Accumulate sample datasets, Step 2: Data preprocessing, Step 3: Train a multi-document summarization generation model based on key events. Step 4: Use the trained model to build an output model to generate a summary; Step 3 involves training the multi-document summarization generation model based on key event information using the dataset D processed in step 2. This step consists of the following sub-steps: Sub-step 3-1, constructing an input layer, the input layer receives a key event sequence and a multi-document sequence as input, converts each word sequence into a word vector sequence, wherein the event vector is fused and spliced after the event trigger word and the event entity are converted, and a mapped key event vector sequence is obtained , and a multi-document vector sequence , Sub-step 3-2 involves constructing a text encoding layer, using two encoders to encode events and documents respectively. The event encoder employs a traditional Transformer encoder to encode key event vector sequences. Semantic encoding extraction is performed to obtain the hidden state vector of the key event vector sequence. The document encoder employs a hierarchical Transformer encoder to process the encoding of multiple documents, taking into account both the attention calculation within a single document and the modeling of relationships between multiple documents. It encodes multiple documents according to document level and word level using formulas (1) and (2), inputting the text sequence of each level... Residual connection calculations are performed, and then multi-head pooling is used to obtain the hidden state vector of the document vector sequence with a unified dimension through formula (3). Then, as shown in formula (4), the relationship between multiple documents is modeled through the interactive attention mechanism, and the information of other documents is integrated into the document itself to obtain a new document representation. The final text sequence representation is then obtained again through residual network calculation. , Sub-step 3-3: Construct an event-aware graph attention layer using a single-layer graph attention network, treating document and event representations as nodes in the graph. Does the event exist in the document as an edge? Attention is calculated by obtaining events and documents of the same length through multi-head attention pooling, and then combining the nodes and edges of the graph through a linear transformation using LeakyReLU as the activation function. Then, the graph attention coefficient is calculated using formula (5). Finally, by applying formula (6) and performing a linear transformation with sigmoid as the activation function, the fused node and edge information is calculated to obtain an event-aware document representation that incorporates key event information. , Sub-steps 3-4 involve constructing a digest decoding layer. A hierarchical Transformer decoder is used for digest decoding. After entering the Transformer decoding layer, the hidden state vector of the key event sequence is used as the query vector Query. Attention is calculated between Query and the decoded digest of the decoder to obtain the event pointer. Then, the obtained event pointer is used through formula (7). Provide guidance, and represent the text to be decoded. Then perform attention calculation to obtain Finally, through a linear transformation, the final output representation of the decoded data is obtained using the LeakyReLU activation function according to formula (8). The formula is as follows: Sub-steps 3-5 involve constructing a probabilistic summarization layer. A linear mapping function and a softmax activation function are used to transform the hidden state vector of the decoding layer into the predicted probability distribution P of the summary words. The calculation formula (9) is shown below: wherein, and are parameters of the model to be trained, Sub-steps 3-6: Train the model. Initialize all training parameters using random initialization. During training, use the Adam optimizer for gradient backpropagation to update the model parameters. Set the initial learning rate to 0.001 and the beam search to 5. The model training ends when the training loss no longer decreases or the number of training rounds exceeds 100. 2.The key event based multi-document summarization generation method of claim 1, wherein, Step 1, accumulate sample data set, as follows: first, collect a plurality of documents classified by topics from the Internet, and generate standard abstracts according to artificial rules, which together constitute a sample data set The artificial generation rule of the standard abstract is: 30% of the document set splices the title of each document as the standard abstract, 30% of the document set splices the first two sentences of each document as the standard abstract, and 40% of the media text uses an artificially written abstract as the standard abstract. 3.The key event based multi-document summarization generation method of claim 1, wherein, Step 2, data preprocessing, as follows: first, the document set under each topic of the data set is spliced, event extraction is performed using the OmniEvent toolkit, the M most frequently occurring event trigger words are selected as the final key event trigger words of the document set as the dynamic features of the event, and entities related to these trigger words are selected as static features, and are respectively organized into trigger word sequences and entity sequences. 4.The key event based multi-document summarization generation method of claim 1, wherein, Step 4: Use the trained model to build an output model to generate a summary, as follows: The output model needs to extract event information as auxiliary input in advance using a tool, take a text set to be summarized as input, and then take the input word at each time of the summary decoding layer as the summary word generated at the last time, and the initial summary word as a special start mark <start>" at each time instance, the summary word is the word with the highest probability output by the summarization probability layer, and the output ends with the end token <end> When the condition is met, stop generating the summary and output the generated summary terms as the predicted summary of the input media text.< / end> < / start>

Images