An event-oriented multi-document summary generation system and method

Abstract

The present invention provides an event-oriented multi-document summary generation system and method, which relates to the field of natural language processing. A document set is generated in target multi-document information through a Spacy module and an encoder representation is generated; a simple recurrent unit generates a simple recurrent unit output according to the encoder representation; a multi-layer perceptron obtains the simple recurrent unit output and generates a probability distribution result; a graph-based encoder obtains the simple recurrent unit output and the probability distribution result as a node feature to generate a final event representation; a timeline analysis module generates a local value and uses the final event representation as a global value to construct a time-event key-value memory network; and the timeline analysis module determines the output text according to the time-event key-value memory network. A more accurate, clear summary containing rich timeline information is generated.

Description

Technical Field

[0001] The present application relates to the field of natural language processing, and in particular to an event-oriented multi-document summary generation system and method. Background Art

[0002] Text summarization technology refers to the natural language understanding and generation technology that generates a relatively short summary that can summarize the main content of the source text after reading the source text. The generated summary can compress the text content but retain the core information in the source text. Text summarization technology mainly includes extractive summary generation and generative summary generation. The extractive technology is mainly based on statistical ideas, extracting keywords and sentences recognized by statistical models from existing news texts to form a relatively short summary, while the generative technology generates summaries based on the model's learning and understanding of the source text.

[0003] Although current multi-document summary generation technologies have achieved certain successes by relying on pre-trained language models and have demonstrated powerful summary generation capabilities, they generally face some limitations. Most of these technologies enhance events as fine-grained information to produce more faithful summaries. However, the problem of excessive dispersion of event information has not been effectively solved, and there is a lack of comprehensive summary generation methods that can integrate key event analysis and timeline analysis technologies on the basis of multiple documents. This is particularly evident in that existing technologies often generate general summary summaries rather than precise summaries that focus on key events and their timeline development. Although timeline analysis technology focuses on temporal relationship analysis and extraction of key events, there is relatively little research on holistic summary generation, which is particularly evident in multi-document summary generation tasks that process long texts. Therefore, there are still obvious obstacles to directly applying existing technologies to actual situations, especially when clear and coherent timeline summaries need to be generated.

[0004] Therefore, how to generate a more accurate, clear summary that contains rich timeline information to improve users' understanding and grasp of key events and their development context described in multiple documents has become a technical problem that needs to be solved urgently. Summary of the invention

[0005] In order to generate a more accurate, clear summary that contains rich timeline information, the present application provides an event-oriented multi-document summary generation system and method.

[0006] In a first aspect, the present application provides an event-oriented multi-document summary generation system, comprising: a multi-document summary generation module, a key event analysis module, and a timeline analysis module;

[0007] The multi-document summary generation module includes a base model and corresponding decoders and encoders;

[0008] The key event analysis module includes: a Spacy module, a simple recurrent unit, a multi-layer perceptron, and a graph-based encoder;

[0009] The Spacy module is used to generate a document set in the target multi-document information and input the document set to the multi-document summary generation module to generate an encoder representation;

[0010] The simple recurrent unit is configured to determine a sentence representation matrix based on the encoder representation to generate a simple recurrent unit output;

[0011] The multilayer perceptron is used to obtain the output of the simple recurrent unit and generate a probability distribution result;

[0012] The graph-based encoder is used to obtain the simple cycle unit output and the probability distribution result as node features to generate a final event representation;

[0013] The timeline analysis module is used to encode the key as time information, concatenate the probability distribution result and the simple cycle unit output to generate a local value, and construct a time-event key value memory network based on the final event representation as a global value;

[0014] The timeline analysis module is used to determine the output text according to the time-event key-value memory network in combination with the summary generation module.

[0015] Optionally, the multi-document summary generation module includes a base model, and the base model is composed of a BART model, a PEGASUS model and a T5 model;

[0016] The base model is initialized by the long-Former to replace the encoder and decoder parts of the long-Former in the base model.

[0017] Optionally, the Spacy module is also used to obtain events in the target multi-document information. , define the acquisition function :

[0018]

[0019] Where s represents a sentence, D represents a document in a document collection, and time-related entities represent time-related entities.

[0020] Optionally, the key event analysis module is used to collect the document Input to the multi-document summary generation module encoder , get the serialized representation of the document, event sentence Via BART encoder After processing, the encoder representation is generated;

[0021]

[0022] in, Representation Document The encoder representation of

[0023] Determine where a sentence starts in a document and ;

[0024] The encoding representation of the query sentence based on the starting position is calculated by averaging the vectors:

[0025]

[0026] in It is a document The vector representation of position t in the last layer;

[0027] Integrate event sentence vectors to represent all event sentences extracted and vectorized from documents Put it together to generate the sentence representation matrix V:

[0028]

[0029] Where m is the total number of event sentences and V is the representation matrix consisting of all event representations.

[0030] Optionally, the multilayer perceptron is used to perform probability calculation through a hidden layer, a RELU activation function and a softmax output layer:

[0031]

[0032] The simple cycle unit , is the probability distribution corresponding to different events.

[0033] Optionally, the timeline analysis module is used to use a preset time text query context representation in the encoder as time information ;

[0034] in, is the vector representation of the document at position t in the last layer;

[0035] Obtain the position embedding position embedding encoding information of the tokenizer module in the multi-document summary generation module as the position information

[0036] ;

[0037] The starting point information is and the endpoint information is ;

[0038] Determine the keys in the key-value memory network by splicing :

[0039] .

[0040] Optionally, the timeline analysis module is further used to input the keys, local values ​​and global values ​​in the time-event key-value memory network into the decoder in the multi-document summary generation module so that the decoder generates output text.

[0041] In a second aspect, the present application provides an event-oriented multi-document summary generation method, the event-oriented multi-document summary generation method comprising:

[0042] The Spacy module generates a document set in the target multi-document information and inputs the document set into the multi-document summary generation module to generate an encoder representation;

[0043] A simple recurrent unit determines a sentence representation matrix based on the encoder representation to generate a simple recurrent unit output;

[0044] The multilayer perceptron obtains the output of the simple recurrent unit and generates a probability distribution result;

[0045] The graph-based encoder obtains the simple cycle unit output and the probability distribution result as node features to generate a final event representation;

[0046] The timeline analysis module encodes the key as time information, concatenates the probability distribution result and the simple recurrent unit output to generate a local value, and constructs a time-event key value memory network based on the final event representation as a global value;

[0047] The timeline analysis module determines the output text according to the time-event key-value memory network in combination with the summary generation module.

[0048] In a third aspect, the present application provides a computer device, comprising: a memory and a processor, wherein the processor executes the method described above when running computer instructions stored in the memory.

[0049] In a fourth aspect, the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method described above.

[0050] In summary, the present application includes the following beneficial technical effects:

[0051] This application generates a document set in the target multi-document information through the Spacy module and inputs the document set into the multi-document summary generation module to generate an encoder representation; the simple recurrent unit determines the sentence representation matrix according to the encoder representation to generate the simple recurrent unit output; the multi-layer perceptron obtains the simple recurrent unit output and generates a probability distribution result; the graph-based encoder obtains the simple recurrent unit output and the probability distribution result as node features to generate the final event representation; the timeline analysis module encodes the key as time information, splices the probability distribution result and the simple recurrent unit output to generate a local value and constructs a time-event key-value memory network according to the final event representation as a global value; the timeline analysis module determines the output text according to the time-event key-value memory network combined with the summary generation module. A more accurate, clear summary containing rich timeline information is generated. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 It is a schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application;

[0053] Figure 2 It is a structural block diagram of the first embodiment of the event-oriented multi-document summary generation system of the present application;

[0054] Figure 3 This is the event-oriented multi-document summary generation method based on the application, which is an event-oriented precise multi-document summary generation framework diagram based on timeline analysis;

[0055] Figure 4 This is an event analysis module diagram of the event-oriented multi-document summary generation method of the present application;

[0056] Figure 5 This is a timeline analysis module diagram of the event-oriented multi-document summary generation method of this application;

[0057] Figure 6 It is a flowchart of the first embodiment of the event-oriented multi-document summary generation method of the present application. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below through the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0059] Reference Figure 1 , Figure 1A schematic diagram of the computer device structure of the hardware operating environment involved in the embodiment of the present application.

[0060] like Figure 1 As shown, the computer device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. Among them, the communication bus 1002 is used to realize the connection and communication between these components. The user interface 1003 may include a display screen (Display), an input unit such as a keyboard (Keyboard), and the optional user interface 1003 may also include a standard wired interface and a wireless interface. The network interface 1004 may optionally include a standard wired interface and a wireless interface (such as a wireless fidelity (Wireless-Fidelity, Wi-Fi) interface). The memory 1005 may be a high-speed random access memory (Random Access Memory, RAM), or a stable non-volatile memory (Non-Volatile Memory, NVM), such as a disk storage. The memory 1005 may also be a storage device independent of the aforementioned processor 1001.

[0061] Those skilled in the art will understand that Figure 1 The structure shown in the figure does not constitute a limitation on the computer device, and may include more or less components than shown in the figure, or combine some components, or arrange the components differently.

[0062] like Figure 1 As shown, the memory 1005 as a storage medium may include an operating system, a network communication module, a user interface module, and an event-oriented multi-document summary generation program.

[0063] exist Figure 1 In the computer device shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the present application can be set in the computer device, and the computer device calls the event-oriented multi-document summary generation program stored in the memory 1005 through the processor 1001, and executes the event-oriented multi-document summary generation method provided in the embodiment of the present application.

[0064] Reference Figure 2 , Figure 2 This is a structural block diagram of the first embodiment of the event-oriented multi-document summary generation system of the present application.

[0065] like Figure 2As shown, the event-oriented multi-document summary generation system proposed in the embodiment of the present application includes: a multi-document summary generation module 10, a key event analysis module 20 and a timeline analysis module 30.

[0066] It should be noted that event-oriented summarization refers to the generation of contextualized summaries for specific events given documents and extracted event structures. This is a technology that combines traditional summarization with document-level event extraction. For multi-document summarization technology, it can achieve key information acquisition and faithful summary generation. Existing research mainly uses events as a means of knowledge enhancement for summary generation, and uses extracted events to guide the generation of summaries covering multiple events to improve the fidelity and fluency of summaries. In combination with controllable summary generation, it can also focus on single, user-specified events as key events, based on which event-related summaries are generated, that is, short and targeted summaries of specific events based on documents and extracted event structures. The technical methods used include identifying events based on BERT, KeyBERT, etc. and combining summary generation models to achieve event-oriented multi-document summary generation.

[0067] The goal of timeline analysis is to concisely summarize the evolution of events along the timeline. Unlike traditional document summarization, timeline analysis needs to model the time series information of input events and summarize important events in chronological order. Current technologies mainly include sentence clustering and sorting, date selection, searching for sentence combinations from the entire document collection to build a timeline, and a two-stage method consisting of date selection and date summarization to build a timeline. These traditional methods can be called extractive methods. Generative methods are less studied, including RNN-based and graph-based methods.

[0068] In the specific implementation, Figure 3 The event-oriented accurate multi-document summary generation framework based on timeline analysis proposed in this embodiment is combined with the event analysis module and the timeline analysis module for the end-to-end multi-document summary generation model, and the key event information and timeline information are integrated into the summary generation. A unified framework is used to analyze and integrate the events in multiple documents and the timelines between events. The entire framework is divided into two different stages: training and reasoning. Since the event and timeline information are obtained from multiple reference documents, or more precisely, analyzed from the events and time information obtained therein, the processing of each module is similar in the training and generation stages. In order to improve the effect of event analysis, timeline analysis and information integration, this embodiment constructs different analysis modules for different events and timeline information. The framework can be roughly divided into three parts:

[0069] Multi-document summary generation module, key event analysis module, timeline analysis module, wherein the multi-document summary generation module is based on the BART-large model to meet the needs of multi-document summaries, that is, the long text problem in the actual application scenario. The pre-trained weights are initialized using LongFormer. The input of longformer will be longer than that of BART, and the number of tokens can reach 8,000, which can meet the needs of long text to a certain extent. The key event analysis module is to first use the HeidelTime module to extract events from multiple documents as the event content for subsequent analysis, that is, the event corresponding to a single time information. After that, this embodiment will use the representation in the encoder as a learner for dividing key events, and then construct an MLP for screening, and then represent the part of the text to construct eventembedding. The timeline analysis module is a time-event memory network. This part uses the key-value memory module to store the corresponding relationship between time and event information. The generated summary realized by this technology should capture time series information to guide the chronological generation process. Therefore, the key-value memory module, in which the keys together form a timeline, and the time series information is used to guide the generation process. The key in this memory is the time position encoding. This embodiment will use this key as a time guide to extract information from the value part in memory. The value part stores local event information in local values ​​and global event information in global values. The local value only stores event representations, which means that it only captures information about the current event. On the other hand, the global value is responsible for learning event features from a global perspective, not only based on itself but also based on its relationship with other events, where global information comes from the graph encoder. In this embodiment, transformers are used to encode the relationship between events and model the relationship between events, that is, to obtain global information. The event and time fusion information is then passed to the decoder part of the summary generation module, where the fusion calculation is performed in the cross-attention part of the first layer of the decoder to achieve the integration of time-event information, thereby guiding the generation of model summaries.

[0070] The multi-document summary generation module 10 includes a base model 101 and corresponding decoders 102 and encoders 103 .

[0071] It should be noted that the multi-document summary generation module includes a base model, which is composed of a BART model, a PEGASUS model and a T5 model; the base model is initialized by a long-Former to replace the encoder and decoder parts in the long-Former in the base model.

[0072] In the specific implementation, the multi-document summary generation module, this embodiment uses long-former to initialize the weights of the summary generation pre-trained language model. For English, the existing BART model with good performance is used, but because BART currently has an English version of the open source model, the BART model can be used for English text summary generation, as well as PEGASUS and T5, which have shown good performance in summary generation tasks. The most important thing is that these models are pre-training methods designed for text generation tasks, and have good adaptability to generation tasks. Therefore, this embodiment selects these models as base models, but considering the processing problem of longer texts composed of multiple documents in multiple documents, this embodiment uses long-Former to initialize the model weights. Long-Former is a model variant of Transformer, designed to process long documents. Longformer has achieved good performance in various long document tasks, such as long document classification, question and answer, etc. It can process sequences up to 4096 tokens, and after initialization, it can process token contents that are much longer than the maximum input length of base models such as BART / PEGASUS / T5. It has good adaptability to multi-document summary tasks, so the multi-document summary module in this embodiment chooses to use Long-Former to initialize the model weights to meet the needs of multi-document summary processing of long texts. The initialization of the model weights is to replace the encoder-decoder part of BART with the encoder and decoder part of Long-Former, and then use consistent structures, parameters, and weights to complete the replacement. The converted BART can process longer sequences to meet the needs of multi-document summarization. Compared with the graph method to encode multiple documents, that is, to use graph neural networks to link information or layers between documents, build an intermediate representation of a single document, and then aggregate cross-document information. These models require additional domain-specific information, such as abstract meaning representation, which either requires a discourse graph or uses a customized architecture for a specific domain dataset, which makes it difficult to use pre-trained language models. However, pre-trained language models have shown strong performance in text generation, including large language models, so this embodiment uses pre-trained language models as base models to achieve summary generation of multiple documents.

[0073] The key event analysis module 20 includes: a Spacy module 201, a simple cycle unit 202, a multi-layer perceptron 203 and a graph-based encoder 204.

[0074] It should be noted that in the key event analysis module, this embodiment combines event analysis, and first uses Spacy to filter event information for the original text, where the event can set a time condition, and filters out events corresponding to different times as the goal of the entire event analysis. Subsequently, SRU and MLP are used to implement the analysis of key events, that is, after filtering the original multi-document text, the events in the document are obtained. On this basis, the encoded data of the encoder part of the summary model is found in combination with the index, and the analysis of key events is implemented based on the representation of the encoder and SRU and MLP. The parameters of SRU and MLP are trained and optimized together with the overall model. The goal of training is to obtain key events. Figure 4 It is an event analysis module that can filter out key events, especially events that are more critical to timeline summary generation. This can be divided into two stages. In the training stage, the parameters of the module, mainly the parameters of the MLP position, will be trained and optimized. In the inference stage, the module will infer the event distribution, which will be combined with the subsequent time-event module to jointly guide the subsequent summary generation.

[0075] The Spacy module 201 is used to generate a document set in the target multi-document information and input the document set to the multi-document summary generation module to generate an encoder representation.

[0076] It is understandable that the Spacy module is also used to obtain events in the target multi-document information. , define the acquisition function :

[0077]

[0078] Where s represents a sentence and D represents a document in a document collection. This module can directly obtain related entities and use them to obtain a sentence representing an event.

[0079] Key event analysis module, used to collect documents Input to the encoder of the multi-document summary generation module , get the serialized representation of the document, event sentence Via BART encoder After processing, the encoder representation is generated;

[0080]

[0081] in, Representation Document The encoder representation of

[0082] Determine where a sentence starts in a document and ;

[0083] The encoding representation of the query sentence based on the starting position is calculated by averaging the vectors:

[0084]

[0085] in It is a document The vector representation of position t in the last layer;

[0086] Integrate event sentence vectors to represent all event sentences extracted and vectorized from documents Put it together to generate the sentence representation matrix V:

[0087]

[0088] Where m is the total number of event sentences and V is the representation matrix consisting of all event representations.

[0089] It should be noted that the use of SRU can capture and utilize the dynamic characteristics in event sequences. Because a lot of text data, especially descriptions of related events, usually contain time order information, which is crucial to understanding the overall context and making accurate classifications. In this way, MLP not only processes static sentence features, but also incorporates the temporal dynamics extracted by SRU, making the final classification or prediction more accurate and temporally interpretable. The representation matrix V is input into the SRU unit. SRU processes the entire sequence through its recursive structure, updating its hidden state at each time step. This can be expressed by the following formula:

[0090]

[0091] where U is the output of SRU and has the same shape as V.

[0092] A simple recurrent unit 202 is used to determine a sentence representation matrix according to the encoder representation to generate a simple recurrent unit output.

[0093] The multi-layer perceptron 203 is used to obtain the simple cycle unit output and generate a probability distribution result.

[0094] It should be noted that the multi-layer perceptron is used to calculate the probability through the hidden layer, RELU activation function and softmax output layer:

[0095]

[0096] Simple recurrent unit , is the probability distribution corresponding to different events.

[0097] The graph-based encoder 204 is used to obtain the simple recurrent unit output and the probability distribution results as node features to generate the final event representation.

[0098] In specific implementation, event analysis not only needs to analyze a single event, but also needs to analyze the relationship between events for summaries. In particular, the relationship between events will affect the attention paid to different events in summary generation. That is, whether an event is a key event depends not only on the event itself but also on the influence of other events. This embodiment uses the event probability distribution output by MLP and the event representation output by SRU as node features of the graph-based encoder:

[0099]

[0100] in Represents the concatenated feature vector as the feature of node i. In order to embed relational information, relational edges are set in the document modeling graph. The relational edges in the graph are first initialized by event representation: where we need to find the representation of the nth event and the representation of the kth event, which are and :

[0101]

[0102] in Representing the relationship edge, we first use MLP to process it, representing the relationship edge between two events n and k. Next, in the relationship-aware encoding process, the relationship edge is converted into Merged into the final event representation,

[0103]

[0104] In the formula Indicates 1 to All indexes between . This module is based on Transformer.

[0105] The calculation method of RE is similar to that of Transformer. It is modified on the original basis. Specifically, the edge information is propagated to the sub-layer output. The calculation method is:

[0106]

[0107] in Represents each weight coefficient. In this way, each event is represented more comprehensively, consisting of its relationship dependency information with other events. At the same time, when determining the weight of each edge, the information of the relationship edge is also considered, because a close relationship will have a great impact on the weight of the edge. It can be further specified as:

[0108]

[0109] The intuition of the Transformer architecture is that each input is not isolated, and its representation also depends on other inputs. In this part of the enhanced Transformer (i.e., graph-based encoder), the modified event representation , expands the dependencies between input documents. Relationships depend not only on their corresponding content, but also on other inputs, and the relationships between them.

[0110] A timeline analysis module 30, for encoding the key as time information, concatenating the probability distribution result and the simple recurrent unit output to generate a local value and constructing a time-event key value memory network according to the final event representation as a global value;

[0111] The timeline analysis module 30 is used to determine the output text according to the time-event key-value memory network combined with the summary generation module.

[0112] A timeline analysis module for querying the context representation in the encoder using preset time text as temporal information ;

[0113] in, is the vector representation of the document at position t in the last layer;

[0114] The calculation method here is similar to the previous calculation, but the target is different. It can be regarded as the same formula, but because the target is different, different target symbols are used to represent it.

[0115] Obtain the position embedding position embedding encoding information of the tokenizer module in the multi-document summary generation module as the position information

[0116] ;

[0117] The starting point information is and the endpoint information is ;

[0118] Determine the keys in the key-value memory network by splicing :

[0119] .

[0120] It should be noted that if Figure 5The timeline analysis module shown in the figure is a time-event key-value memory network. The key represents the encoding of time information. There are two types of time information here. One is to directly use time-related text to query the context representation in the encoder output. The other is to obtain the position embedding of the event in the tokenizer. The two represent the time and position information of the event in the context respectively.

[0121] It should be noted that the timeline analysis module is also used to input the keys, local values ​​and global values ​​in the time-event key-value memory network into the decoder in the multi-document summary generation module so that the decoder generates output text.

[0122] In a specific implementation, for the multi-document summarization task, this embodiment can regard the source text composed of multiple documents as a long sequence composed of Tokens, and regard the target time event summary as a sequence composed of shorter Tokens. , which is used as the representation of the source text, and the target summary text is represented as ,in and Indicates the start and end of a sequence marker, and Respectively represent the length of the sequence, that is, the number of Tokens. Then the method adopted in this embodiment is to use the Long-former model to initialize the weight of the summary model (BART) for processing, which mainly includes two parts: the encoder and the decoder. Specifically, the encoder Mapped into a series of hidden states, represented as , that is, after a series of calculations, the final hidden vector output by the encoder is obtained ,in Represents the calculated vector, the dimension of which is equal to the number of Tokens in the original sequence. , the processing process can be expressed as follows:

[0123]

[0124] Assume that the previous Tokens have been generated and are being generated , that is, Token, the decoder hides the state through the self-attention encoder With the previously generated A Token is To calculate the current vector :

[0125]

[0126] Indicates that the decoder is used for processing, and can be obtained in parallel during training , which is the vector ,Depend on Composition, vector dimension size and Dimension size Equal, so the calculation can be expressed as the following formula:

[0127]

[0128] Finally, after Softmax, we can calculate The conditional probability of The probability under the confirmed condition is expressed as follows:

[0129]

[0130]

[0131] in Represents the parameter matrix, that is, the parameters that can be optimized during training, using the Adam optimizer. Represents the negative log-likelihood loss function, which is used for classification problems, that is, calculating the probability of the current Token corresponding to the character or symbol in the vocabulary.

[0132] In the decoder stage, that is, the Cross-Attention, in general, the information from the encoder is accepted as K and V. In information fusion, the event information is fused into the decoder, and the information in the key-value memory network is used as K and V to fuse the corresponding information, that is, the output of the time-event key-value memory network is integrated into the Cross-Attention layer of the decoder as an additional input. In each time step of the decoder, not only the hidden state from the encoder is used as the input of the attention mechanism, but also the key and value from the key-value memory network. The calculation is as follows:

[0133] For the state of the decoder at time t , the output of the encoder is , the key in the key-value memory network is K, the local value is X, and the global value is Y. First, calculate Query, Key and Value, that is , ,

[0134] ,in , , It is a learnable weight matrix, after which the attention can be calculated and the decoder state can be updated according to the normal processing logic, and the generated text can be output.

[0135] Through this fusion method, the decoder can comprehensively utilize the context information obtained from the encoder and the time-event information obtained from the key-value memory network when generating summaries. This allows the generated timeline summary to not only accurately reflect the content of the event, but also to logically organize the information in chronological order, improving the quality of the summary and the temporal accuracy of the information.

[0136] This embodiment generates a document set in the target multi-document information through the Spacy module and inputs the document set into the multi-document summary generation module to generate an encoder representation; the simple recurrent unit determines the sentence representation matrix according to the encoder representation to generate a simple recurrent unit output; the multi-layer perceptron obtains the simple recurrent unit output and generates a probability distribution result; the graph-based encoder obtains the simple recurrent unit output and the probability distribution result as node features to generate the final event representation; the timeline analysis module encodes the key as time information, splices the probability distribution result and the simple recurrent unit output to generate a local value and constructs a time-event key-value memory network according to the final event representation as a global value; the timeline analysis module determines the output text according to the time-event key-value memory network combined with the summary generation module. A more accurate, clear summary containing rich timeline information is generated.

[0137] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any limitation on this.

[0138] The present application embodiment provides an event-oriented multi-document summary generation method, referring to Figure 6 , Figure 6 This is a flowchart of the first embodiment of the event-oriented multi-document summary generation method of the present application.

[0139] In this embodiment, the event-oriented multi-document summary generation method includes the following steps:

[0140] Step S10: The Spacy module generates a document set in the target multi-document information and inputs the document set to the multi-document summary generation module to generate an encoder representation.

[0141] Step S20: The simple recurrent unit determines a sentence representation matrix according to the encoder representation to generate a simple recurrent unit output.

[0142] Step S30: The multilayer perceptron obtains the simple cycle unit output and generates a probability distribution result.

[0143] Step S40: The graph-based encoder obtains the simple cycle unit output and the probability distribution result as node features to generate the final event representation.

[0144] Step S50: The timeline analysis module encodes the key as time information, concatenates the probability distribution result and the simple recurrent unit output to generate a local value, and uses the final event representation as a global value to construct a time-event key-value memory network.

[0145] Step S60: The timeline analysis module determines the output text according to the time-event key-value memory network combined with the summary generation module.

[0146] This application generates a document set in the target multi-document information through the Spacy module and inputs the document set into the multi-document summary generation module to generate an encoder representation; the simple recurrent unit determines the sentence representation matrix according to the encoder representation to generate the simple recurrent unit output; the multi-layer perceptron obtains the simple recurrent unit output and generates a probability distribution result; the graph-based encoder obtains the simple recurrent unit output and the probability distribution result as node features to generate the final event representation; the timeline analysis module encodes the key as time information, splices the probability distribution result and the simple recurrent unit output to generate a local value and constructs a time-event key-value memory network according to the final event representation as a global value; the timeline analysis module determines the output text according to the time-event key-value memory network combined with the summary generation module. A more accurate, clear summary containing rich timeline information is generated.

[0147] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0148] In addition, for technical details not fully described in this embodiment, please refer to the event-oriented multi-document summary generation method provided in any embodiment of the present application, which will not be repeated here.

[0149] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0150] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as a read-only memory (ROM) / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods of each embodiment of the present application.

[0152] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. An event-oriented multi-document summary generation system, characterized in that: include: Multi-document summary generation module, key event analysis module, and timeline analysis module; The multi-document summary generation module includes a base model and corresponding decoders and encoders; The key event analysis module includes: a Spacy module, a simple recurrent unit, a multi-layer perceptron, and a graph-based encoder; The Spacy module is used to generate a document set in the target multi-document information and input the document set to the multi-document summary generation module to generate an encoder representation; The simple recurrent unit is configured to determine a sentence representation matrix based on the encoder representation to generate a simple recurrent unit output; The multilayer perceptron is used to obtain the output of the simple recurrent unit and generate a probability distribution result; The graph-based encoder is used to obtain the simple cycle unit output and the probability distribution result as node features to generate a final event representation; The timeline analysis module is used to encode the key as time information, concatenate the probability distribution result and the simple cycle unit output to generate a local value, and construct a time-event key value memory network based on the final event representation as a global value; The timeline analysis module is used to determine the output text according to the time-event key-value memory network in combination with the summary generation module; The Spacy module is also used to obtain events in the target multi-document information. , define the acquisition function : Where s represents a sentence, D represents a document in a document collection, and time-related entities represent time-related entities; The key event analysis module is used to collect the document Input to the multi-document summary generation module encoder , get the serialized representation of the document, event sentence Via BART encoder After processing, the encoder representation is generated; in, Representation Document The encoder representation of Determine where a sentence starts in a document and ; The encoding representation of the query sentence based on the starting position is calculated by averaging the vectors: in It is a document The vector representation of position t in the last layer; Integrate event sentence vectors to represent all event sentences extracted and vectorized from documents Put it together to generate the sentence representation matrix V: Where m is the total number of event sentences and V is the representation matrix consisting of all event representations.

2. The event-oriented multi-document summary generation system according to claim 1, characterized in that: The multi-document summary generation module includes a base model, which is composed of a BART model, a PEGASUS model and a T5 model; The base model is initialized by the long-Former to replace the encoder and decoder parts of the long-Former in the base model.

3. The event-oriented multi-document summary generation system according to claim 1, characterized in that: The multi-layer perceptron is used to perform probability calculation through a hidden layer, a RELU activation function and a softmax output layer: The simple cycle unit , is the probability distribution corresponding to different events.

4. The event-oriented multi-document summary generation system according to claim 1, characterized in that: The timeline analysis module is used to query the context representation in the encoder using a preset time text as time information ; in, is the vector representation of the document at position t in the last layer; Obtain the position embedding position embedding encoding information of the tokenizer module in the multi-document summary generation module as the position information ； The starting point information is and the endpoint information is ; Determine the keys in the key-value memory network by splicing : 。 5. The event-oriented multi-document summary generation system according to any one of claims 1 to 4, characterized in that: The timeline analysis module is also used to input the keys, local values ​​and global values ​​in the time-event key-value memory network into the decoder in the multi-document summary generation module so that the decoder generates output text.

6. An event-oriented multi-document summary generation method, characterized in that: The event-oriented multi-document summary generation method comprises: The Spacy module generates a document set in the target multi-document information and inputs the document set into the multi-document summary generation module to generate an encoder representation; A simple recurrent unit determines a sentence representation matrix based on the encoder representation to generate a simple recurrent unit output; The multilayer perceptron obtains the output of the simple recurrent unit and generates a probability distribution result; The graph-based encoder obtains the simple cycle unit output and the probability distribution result as node features to generate a final event representation; The timeline analysis module encodes the key as time information, concatenates the probability distribution result and the simple recurrent unit output to generate a local value, and constructs a time-event key value memory network based on the final event representation as a global value; The timeline analysis module determines the output text according to the time-event key-value memory network in combination with the summary generation module; The Spacy module obtains events in the target multi-document information. For the document collection , define the acquisition function : Where s represents a sentence, D represents a document in a document collection, and time-related entities represent time-related entities; The key event analysis module collects the documents Input to the multi-document summary generation module encoder , get the serialized representation of the document, event sentence Via BART encoder After processing, the encoder representation is generated; in, Representation Document The encoder representation of Determine where a sentence starts in a document and ; The encoding representation of the query sentence based on the starting position is calculated by averaging the vectors: in It is a document The vector representation of position t in the last layer; Integrate event sentence vectors to represent all event sentences extracted and vectorized from documents Put it together to generate the sentence representation matrix V: Where m is the total number of event sentences and V is the representation matrix consisting of all event representations.

7. A computer device, characterized in that: The device comprises: a memory and a processor, and when the processor runs the computer instructions stored in the memory, the method as claimed in claim 6 is executed.

8. A computer-readable storage medium, characterized in that: The method comprises instructions which, when executed on a computer, cause the computer to execute the method as claimed in claim 6.

Images