A multi-document summarization generation method, system and device

Abstract

This invention provides a method, system, and device for generating multi-document summaries, relating to the field of computer technology. The method includes: employing an improved Siamese neural network to evaluate the relevance scores between document paragraphs and headings in a multi-document context; combining a hierarchical clustering deduplication method based on relevance scores to extract a subset of paragraphs from the document paragraph set whose importance is greater than a set importance level and whose redundancy is less than a set redundancy level; and processing the paragraph subset using a multi-level Transformer-based summarization device to generate multi-document summary text. The multi-level Transformer-based summarization device has cross-document learning capabilities. This invention can generate concise, high-quality multi-document summary text.

Description

Technical Field

[0001] This invention relates to the field of computer technology, and in particular to a method, system and device for generating multi-document summaries. Background Technology

[0002] Automatic document summarization is an important subtask in natural language processing. Automatic summarization technology can obtain compressed versions of massive amounts of data, facilitating the processing of ever-expanding text data. Text summarization is divided into single-document summarization and multi-document summarization. Although multi-document summarization is based on single-document summarization, it is more comprehensive and accurate because it can encompass information from a wider range of time and space, generating a summary from multiple documents written by different people at different times. Therefore, research on multi-document summarization has become a hot topic in the field of automatic document summarization, yielding many research results and being successfully applied in many application areas, such as breaking news, scientific publications, email, intelligent writing, Wikipedia article generation, and medical documents.

[0003] Multi-document summarization has a wide range of applications, but the complex relationships between documents, such as complementarity, overlap, and conflict, and the typically large input volume of documents, make multi-document summarization more complex and difficult to solve than single-document summarization. Furthermore, a major obstacle to multi-document summarization applications is the sheer number of source documents, making it practically impossible to embed all source documents into a vector given current hardware computing and storage limitations. Summary of the Invention

[0004] The purpose of this invention is to provide a method, system, and device for generating multi-document summaries to solve the problem of low accuracy in multi-document summarization.

[0005] To achieve the above objectives, the present invention provides the following solution:

[0006] A method for generating multi-document summaries, comprising:

[0007] An improved Siamese neural network was used to evaluate the correlation score between document paragraphs and headings in a multi-document scenario;

[0008] Combining a hierarchical clustering deduplication method based on relevance, a subset of paragraphs in the document paragraph set with an importance greater than a set importance and a redundancy lower than a set redundancy is extracted according to the relevance score;

[0009] The multi-level Transformer-based summarization generation device processes the paragraph subset to generate multi-document summary text; the multi-level Transformer-based summarization generation device has cross-document learning capabilities.

[0010] Optionally, the improved structure in the improved Siamese neural network specifically includes: a coding layer structure, an attention layer structure, and an output layer structure;

[0011] The coding layer structure is as follows:

[0012]

[0013]

[0014] in, The encoding result for the nth word of the title. This is the encoding result for the m-th word of the paragraph; after encoding, it will be... Input into the Bi-LSTM layer for encoding. The result after encoding the title. This represents the result after paragraph encoding; n is the number of title tokens; m is the number of paragraph tokens;

[0015] The attention layer structure is as follows:

[0016]

[0017]

[0018]

[0019] in, To be The vector representation is input into the attention layer and computed through the self-attention mechanism. To be The vector representation is input into the attention layer and computed through the self-attention mechanism. To be and The vector representation obtained by connecting the input to the fully connected layer;

[0020] The output layer structure is as follows:

[0021]

[0022] Among them, W s W is the first weight parameter of the activation function. t b is the second weight parameter of the activation function. t b is the first offset parameter of the activation function. s is the second offset parameter of the activation function; r is the association score between the heading and paragraph sets.

[0023] Optionally, an improved Siamese neural network can be used to evaluate the correlation between document paragraphs and headings in a multi-document scenario, specifically including:

[0024] Obtain a document cluster comprising multiple documents; each document includes a title and a body text;

[0025] The main text of multiple documents in each document cluster is split into multiple document paragraph sets according to line breaks;

[0026] The set of document paragraphs is input into the improved Siamese neural network to generate the importance of each document paragraph and its corresponding title to the final generated summary text; the importance is the correlation score between the document paragraph and the title.

[0027] Optionally, the set of document paragraphs is input into the improved Siamese neural network to generate the importance of each document paragraph and its corresponding title to the final generated summary text, and then the process further includes:

[0028] The document paragraph set is reordered according to the relevance score to generate a sorted document paragraph set.

[0029] Select the first K paragraphs from the sorted document paragraph set as a data subset containing key information; K is a positive integer greater than or equal to 1.

[0030] Optionally, in conjunction with a hierarchical clustering deduplication method based on relevance, a subset of paragraphs in the document paragraph set whose importance is greater than a set importance level and whose redundancy is lower than a set redundancy level is extracted according to the relevance score. Specifically, this includes:

[0031] Obtain the set of relevance scores, and establish a relevance adjacency matrix based on the set of relevance scores;

[0032] Using the aforementioned adjacency matrix as input, a clustering result set is generated by combining a hierarchical clustering deduplication method based on correlation.

[0033] The probability distribution of all clustering results in the clustering result set is calculated using Gini impurity, and the clustering result with the most uniform distribution is selected.

[0034] Each cluster center in the most uniformly distributed clustering results is taken as a subset of the clustering data for the association score;

[0035] The paragraph subset corresponding to the clustered data subset containing the key information is determined based on the data subset containing the correlation score; the paragraph subset is the paragraph subset in the document paragraph set whose importance is greater than a set importance level and whose redundancy is lower than a set redundancy level.

[0036] Optionally, a multi-level Transformer-based summarization device processes the subset of paragraphs to generate multi-document summary text, specifically including:

[0037] The paragraph subset is used as input to the multi-level Transformer's summary generation device, and the encoder of the multi-level Transformer's summary generation device generates a fusion of global and local encoding vectors for the paragraph subset.

[0038] The decoder of the multi-level Transformer summarization generation device decodes the encoded vector into natural language; the natural language is multi-document summary text.

[0039] Optionally, the paragraph subset is used as input to the multi-level Transformer summarization generation device, and the encoder of the multi-level Transformer summarization generation device generates a fusion global and local encoding vector of the paragraph subset. Specifically, the encoder includes a 6-layer local network structure and a 2-layer global network structure.

[0040] The local network structure is used to encode each paragraph in the paragraph set to generate local features for each paragraph;

[0041] A global network structure is used to learn cross-document relationships between paragraphs in the paragraph set, generating global features for each paragraph.

[0042] The local features and the global features are fused to generate the segment subset encoding vector that combines global and local features.

[0043] A multi-document summarization system, comprising:

[0044] The relevance score evaluation module is used to evaluate the relevance scores between document paragraphs and headings in a multi-document scenario using an improved Siamese neural network.

[0045] The paragraph subset extraction module is used to combine a hierarchical clustering deduplication method based on relevance score to extract a subset of paragraphs from the document paragraph set whose importance is greater than a set importance score and whose redundancy is lower than a set redundancy score.

[0046] The summary text generation module is used to process the paragraph subset based on a multi-level Transformer summary generation device to generate multi-document summary text; the multi-level Transformer summary generation device has cross-document learning capabilities.

[0047] An electronic device includes a memory and a processor, the memory storing a computer program, and the processor running the computer program to cause the electronic device to perform the above-described multi-document digest generation method.

[0048] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described multi-document digest generation method.

[0049] According to specific embodiments provided by the present invention, the following technical effects are disclosed: The present invention provides a multi-document summarization generation method, system, and device. It employs an improved Siamese neural network to evaluate the relevance scores between document paragraphs and headings in a multi-document context. By ranking paragraphs based on the relevance scores, a subset of paragraphs with importance greater than a set importance level and redundancy lower than a set redundancy level is extracted from the document paragraph set as input to a multi-level Transformers summarization generation device, thus solving the problem of document input limitations in multi-document summarization tasks. Simultaneously, a hierarchical clustering method based on relevance reduces redundant information in the input of the multi-level Transformers summarization generation device, thereby reducing hardware computation and improving summarization quality to some extent. Furthermore, using a multi-level Transformers summarization generation device to encode documents has the ability to extract dependencies within and between paragraphs, solving the cross-document learning problem in multi-document summarization, and ultimately generating concise, high-quality multi-document summary text. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a flowchart of the multi-document digest generation method provided in Embodiment 1 of the present invention;

[0052] Figure 2 This is a flowchart of the method for calculating the correlation score between a set of multiple document paragraphs and the corresponding title based on an improved Siamese neural network structure, as provided in Embodiment 2 of the present invention.

[0053] Figure 3 This is a screenshot diagram of a data sample provided in Embodiment 2 of the present invention;

[0054] Figure 4 This is a schematic diagram of a data sample when K is 40, provided in Embodiment 2 of the present invention;

[0055] Figure 5 This is a schematic diagram of a clustered paragraph set data example provided in Embodiment 2 of the present invention;

[0056] Figure 6A flowchart of the multi-level Transformer summary generation apparatus provided in Embodiment 2 of the present invention;

[0057] Figure 7 This is a framework diagram of the Transformer-based multi-document summarization generation method provided in Embodiment 2 of the present invention;

[0058] Figure 8 This is a schematic diagram illustrating a data example of obtaining the final multi-document summary text provided in Embodiment 2 of the present invention. Detailed Implementation

[0059] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0060] The purpose of this invention is to provide a method, system, and device for generating multi-document summaries, which can generate concise, high-quality multi-document summary text.

[0061] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0062] Example 1

[0063] like Figure 1 As shown, the present invention provides a method for generating multi-document summaries, including:

[0064] Step 101: Use an improved Siamese neural network to evaluate the relevance scores of document paragraphs and headings in a multi-document context.

[0065] In practical applications, step 1.1 involves taking a document cluster D consisting of multiple documents as input. Each document d in D consists of a title and body text. After obtaining each document cluster, the body text of the multiple documents is split into a paragraph set P according to the line breaks, which serves as the input for the next stage.

[0066] Step 1.2: Input the obtained paragraph set P into the improved Siamese neural network structure (SN) that has been fully trained. The neural network structure includes an encoding layer, a self-attention layer and an output layer. The training objective of SN is to calculate the importance (relevance score r) between each paragraph and the title for the final generated summary. Input the paragraph set P into the Siamese network SN to obtain the relevance score Rn of all paragraphs, where n is the number of paragraphs in the paragraph set.

[0067] The improved structure in the improved Siamese neural network specifically includes: a coding layer structure, an attention layer structure, and an output layer structure;

[0068] The coding layer structure is as follows:

[0069]

[0070]

[0071] in, For the model in the Embedding layer Encode and generate word vectors. The encoding result for the nth word of the title. This is the encoding result for the m-th word of the paragraph; after encoding, it will be... Input into the Bi-LSTM layer for encoding. The result after encoding the title. This represents the result after paragraph encoding; n is the number of title tokens; m is the number of paragraph tokens.

[0072] The attention layer structure is as follows:

[0073]

[0074]

[0075]

[0076] in, To be The vector representation is input into the attention layer and computed through the self-attention mechanism. To be The vector representation is input into the attention layer and computed through the self-attention mechanism. To be and The vector representation obtained by connecting the input to the fully connected layer.

[0077] The output layer structure is as follows:

[0078]

[0079] Among them, W s W is the first weight parameter of the activation function. t b is the second weight parameter of the activation function. t b is the first offset parameter of the activation function. sis the second offset parameter of the activation function; the model maps the association between titles and paragraphs to an association score r with a value range of [0,1] through the logistic regression layer, where r is the association score between the set of titles and paragraphs.

[0080] Step 1.3: Reorder P according to the paragraph relevance score r, and take the top-K paragraphs of the sorted set as the data subset P' containing key information.

[0081] Step 102: Combining the hierarchical clustering deduplication method based on relevance, extract a subset of paragraphs from the document paragraph set whose importance is greater than a set importance level and whose redundancy is lower than a set redundancy level according to the relevance score.

[0082] In practical applications, step 2.1: Based on the relevance score set R obtained in step 1.2... n The adjacency matrix is ​​constructed by initializing an n*n adjacency matrix with n nodes. The formula for calculating the sample distance between nodes can be expressed as:

[0083] Dis ij =|r i -r j |(i,j∈{2,……,L})

[0084] This yields the adjacency matrix M, which represents the degree of association.

[0085] Step 2.2: Using the adjacency matrix M as input, the hierarchical clustering method is used to obtain the clustering result set C, where the i-th cluster is c. i .

[0086] Step 2.3: To obtain reasonable classification results, the probability distribution p(c) of all clustering results is calculated using Gini impurity. i ),Right now:

[0087] The impurity of the Gini is Where p(c i ) represents class c i The probability that any element belongs to class i is used to obtain the Gini impurity distribution. t Then, take argmin{Gini1,……,Gini t The corresponding t-classification at time} is taken as the final clustering result. Each cluster center in the most uniformly distributed clustering result is taken as the clustering data subset CR of the correlation score. According to the data subset P' of the key information obtained in step three of claim 1, the paragraph set P” corresponding to CR can be obtained as the processing object of the next stage.

[0088] Step 103: The multi-level Transformer-based summarization generation device processes the paragraph subset to generate multi-document summary text; the multi-level Transformer-based summarization generation device has cross-document learning capabilities.

[0089] In practical applications, step 3.1: Using P” as the input of the summary generation device HT, the encoder of HT can obtain the key feature vector V of P” which combines global and local features. The encoder consists of a 6-layer first processing unit (local network structure LT) and a 2-layer second processing unit (global network structure GT).

[0090] First processing unit: Local network structure LT used to collect text dependencies within a document.

[0091] The first processing unit is specifically used to encode each paragraph of P” to obtain the local features of each paragraph. The calculation formula for the encoding result after inputting into the LT layer is as follows:

[0092] h = LN(x) l-1 +MHAtt(x l-1 ))

[0093] x l =LN(h+FFN(h))

[0094] Here, MHPool is the multi-head pooling operation, LN refers to the neuron layer normalization operation, and the output is the input x of the next layer. l FFN represents a two-layer feedforward network with ReLU as the activation function.

[0095] The second processing unit is a global network structure GT used to collect textual dependencies between documents.

[0096] The second processing unit is specifically used to perform cross-document learning among the paragraphs in P”, obtaining the global encoding vector for each paragraph. The formula for calculating the encoding result after inputting it into the GT layer is as follows:

[0097] h' = x l-1 +Inter-att(MHPool(x l-1 ))

[0098] x l =LN(x l-1 +FFN(h'))

[0099] Inter-att refers to the attention mechanism that collects information across all paragraphs.

[0100] Step 3.2: Take V as input and decode the key feature vector into natural language through the decoder of HT, that is, output the generative summary Sum of document cluster D. The decoder consists of a 1-layer third processing unit.

[0101] The third processing unit: a Transformer-based decoder structure.

[0102] The third processing unit is specifically used to decode the local features and global encoding vectors of each paragraph in P”, and to decode the hidden vectors in the neural network structure into natural language. The structure of the third processing unit is completely consistent with that of the first processing unit.

[0103] This invention employs an improved Siamese neural network to evaluate the correlation between document paragraphs and titles in a multi-document context. It combines a correlation-based ranking algorithm and hierarchical clustering to extract a subset of paragraphs with high importance and low redundancy. This subset is then processed using a multi-level Transformer summarization device with cross-document learning capabilities. This solution addresses the challenges of extracting important information, high redundancy, and the inability to learn across documents in multi-document summarization, providing insights for the design of multi-document summarization devices. It produces fluent and complete summary texts and improves the overall quality of multi-document summarization.

[0104] Example 2

[0105] Embodiment 2 of this invention provides a method for calculating the relevance score between a set of multiple document paragraphs and their corresponding titles based on an improved Siamese neural network structure. See [link to documentation]. Figure 2 The method includes:

[0106] 1. Calculate the relevance score:

[0107] Step 1: Calculate the relevance score between the paragraph set of the input document and the corresponding title in the case of multiple documents using an improved Siamese neural network structure.

[0108] In this embodiment of the invention, the sample is a feature vector obtained after various data conversions. The data includes multi-document text data, etc. The data can be various network data obtained from the network, or various data stored locally.

[0109] Step 2: After obtaining the correlation degree r between any paragraph and its corresponding title in the multi-document paragraph set, re-sort the paragraph set according to the correlation degree r, and take the top-K paragraphs of the sorted set as the data subset containing key information.

[0110] This invention constructs the multi-document dataset WikiExtend based on entries from the WikiSum dataset. The WikiExtend dataset contains 1,717,905 instances, composed of relevant information from English Wikipedia entries, including titles, text, URLs, and related articles. The dataset is stored in XML format, with each entry corresponding to a separate file, using the title... <t>Store the title, using Store paragraphs of the main text using <g>Store reference summary information.

[0111] Example:

[0112] <data>

[0113] <t> title< / t>

[0114] < / data> < / g>

[0115] Paragraphs in the main text...

[0116]

[0117] <g>

[0118] Summary information.

[0119] < / g>

[0120]

[0121] Data sample screenshots are as follows Figure 3 As shown.

[0122] This invention provides a detailed description of a method for calculating the relevance score between a set of multiple document paragraphs and their corresponding titles based on an improved Siamese neural network structure. In this embodiment, the aforementioned set of multiple document paragraphs and their corresponding titles are input into the improved Siamese neural network structure. The implementation phase is divided into two stages: training and validation. In the training stage, the model mentioned above is trained using the method of minimizing cross-entropy. The ROUGE-2 recall score y between the paragraph text and the reference summary is used as the relevance score. After training, the model will obtain the relevance score r between the input paragraph text and the corresponding title, ultimately making the probability distribution of the relevance score r consistent with that of the recall y. In this embodiment, firstly, the relevance score between paragraphs and titles is calculated; then, the paragraph set is reordered according to the relevance score. Through an iterative update process, a paragraph set ordered by importance can be obtained; next, the Top-K algorithm is used to obtain a subset of paragraphs with key information. Finally, the Top-K paragraphs are selected as the input for the next stage. In this example, the value of K is determined by specific task experiments; in this example, K is 40. Data samples are as follows: Figure 4 As shown.

[0123] 2. Deduplication methods based on hierarchical clustering of association degree, including:

[0124] Step 1: Construct an adjacency matrix based on the relevance score set.

[0125] Based on the sample distance formula mentioned above, construct an adjacency matrix based on correlation degree.

[0126] Step 2: Calculate the data distribution of all clustering results using Gini impurity. Take each cluster center in the most evenly distributed clustering results as a subset of data for subsequent processing. This subset of data contains comprehensive and important information and has low redundancy.

[0127] After paragraph rating, the paragraph with the highest relevance score, p1, is extracted from the subset of paragraphs. The relevance scores of the remaining paragraphs are used as input for clustering. Let the difference distance between relevance scores be defined as the absolute value of the difference between the relevance scores of two paragraphs, denoted as Dis. Construct the relevance difference distance matrix RD. After n clustering operations, the number of categories K in the nth layer of clustering is obtained. Let K... p The set of relevance R of the corresponding paragraphs in the class p K q The set of relevance degrees for the corresponding paragraphs in the class is R. q Define class K p and class K q The distance between the correlations is min{|R p -R q Cluster analysis is performed using the correlation difference distance. In each clustering operation, the two paragraphs with the smallest correlation difference distance are grouped into one cluster. After N clustering calculations using the above algorithm, all paragraphs are finally grouped into one cluster, resulting in the cluster tree C. t , t∈{1,……,N-1} (remove extreme clustering results where all clusters merge into one class), C t The clustering result shows that nodes G are divided into k classes, denoted as c1, ..., c2. k The Gini impurity of each layer of the clustering tree G is calculated. A smaller Gini impurity value indicates a more gradual distribution of association differences among the categories, and a more reasonable clustering result. After clustering, the text corresponding to the center of each cluster is obtained. The sorting order is preserved, and the resulting data is rearranged to obtain P'. A sample of the clustered paragraph set data is shown below. Figure 5 As shown.

[0128] 3. Figure 6 This is a flowchart of the multi-level Transformer summary generation apparatus provided in Embodiment 2 of the present invention. Figure 7 This is a framework diagram of the Transformer-based multi-document digest generation method provided in Embodiment 2 of the present invention, as shown below. Figures 6-7 As shown, the summary generation method based on a multi-level Transformer summary generation device includes:

[0129] Step 1: After completing the above processing, input the sorted and clustered paragraph set into the summary generation device. Using the first processing unit, namely the LT network structure, the paragraph text is encoded with local features to obtain the local encoding vector of each paragraph.

[0130] Step 2: Input all the encoding vectors in the paragraph set, and use the second processing unit, namely the GT network structure, to obtain the global encoding vector of each paragraph.

[0131] Step 3: Input the encoded vector of each paragraph, which combines local and global encodings. Using the third processing unit, namely the decoder of HT, the encoded vectors are output as natural language, i.e., the final multi-document summary text. Data sample: Figure 8 As shown.

[0132] 4. Summary generation results of this invention.

[0133] Through the above process and steps, the performance of the method was tested on the self-built dataset WikiExtend. WikiExtend is an expansion, optimization, and reorganization of the public dataset WikiSum, ultimately establishing the WikiExtend dataset. The WikiExtend dataset includes 1,717,905 instances, each containing a title and a corresponding set of multiple document paragraphs, split in a 50:1:1 ratio. 50% of the split is used as the training set, and the remaining portion is used as the test and validation sets.

[0134] To verify the effectiveness of the proposed method for calculating the relevance score between a set of multiple document paragraphs and their corresponding titles based on an improved Siamese neural network structure, WikiExtend was used as the test data. Four ranking methods—NatureSeq, Tf-IDF, HTMS, and RF—were compared in the experiment.

[0135] NatureSeq: It extracts the first K paragraphs and links them together according to the natural order of the dataset, that is, according to the title, the reference links in the article body (in order of appearance), and the search results (sorted by results).

[0136] Tf-Idf: Calculates the Tf-Idf cosine distance between each paragraph and the title as a similarity ranking method.

[0137] HTMS: A ranking method based on training the correlation between headings and paragraphs using max pooling operations.

[0138] RF: A method for ranking paragraphs based on the correlation degree calculated using an improved Siamese network model.

[0139] This invention uses relevance to represent the dependency between the input paragraph and the target summary. The importance of the paragraphs is ranked using the dependency score. The higher the relevance score, the more important the input paragraph is and the more information it contributes to the final generated summary text.

[0140] Data was input into four methods, each sorted according to a different comparative system. The title and the top K paragraphs were concatenated as the input text sequence. ROUGE-L Recall scores were calculated for the four methods and the target summary. ROUGE-L Recall is an automatic summarization evaluation method that compares scores obtained from automatic and human summarization. It calculates the recall of the summary results by using the longest common subsequence between the output sentence and the reference sentence. A higher ROUGE-L Recall score indicates that the sorting method better ensures the completeness of paragraph information. The calculation results are shown in Table 1.

[0141] Table 1. ROUGE-L Score Table for Four Sorting Methods

[0142]

[0143] The data in Table 1 clearly demonstrates the ranking advantage of RF. When the same K value is used, RF has a higher ROUGE-LRecall score, which means that after the paragraphs are ranked, the first K paragraphs in the model of this invention contain more important information contained in the target summary, thus proving the effectiveness of the ranking method of this invention.

[0144] To verify the effectiveness of the hierarchical clustering method based on correlation in this invention, the sorted paragraph set was used as input to verify the clustering effect. The specific clustering results are shown in Table 2.

[0145] Table 2. Comparison of informativeness and compression rate results of clustering methods.

[0146]

[0147] The number of paragraphs decreased significantly before and after clustering, while ROUGE-L Recall did not change significantly, indicating that the clustering effect has the ability to remove duplicates and does not lose paragraphs containing important information.

[0148] As shown in Table 2, ROUGE-L Recall did not change significantly after clustering, but the number of paragraphs decreased significantly. Moreover, the deduplication effect of hierarchical clustering was very significant as the number of input paragraphs increased. When K=40, the clustering compression rate C... k It can reach 21.93%, achieving the effect of removing redundancy.

[0149] To verify the effectiveness of the hierarchical Transformers-based summarization device of this invention, two evaluation methods, ROUGE-N (N=1,2) and ROUGE-L, were used to evaluate the method's effectiveness, and their harmonic mean F1 score was used as the basis for evaluation. Furthermore, the ROUGE evaluation system cannot determine the readability of the text. To determine the readability of the generated summary, a manual scoring method was adopted. The manual judgment criteria included three aspects: completeness (whether it contains the main semantic components of the summary), continuity (whether it is correctly written human-readable text), and conciseness (whether it meets the summary's generalization requirements). A score of 1 was assigned if the criteria were met, and 0 otherwise.

[0150] The following four algorithms were used to generate summaries in the experiment, and the summaries were compared with those generated by humans:

[0151] Gold: Manually written target summary text, which in this experiment is the first paragraph of the main text of the data.

[0152] Lead: A simple approach that uses the data headings and the first K sorted paragraphs as a summary; in this experiment, K was set to 3.

[0153] HTMS: A hierarchical Transformers model that uses a linear retrospective model to train and rank the correlation between headings and paragraphs.

[0154] RHT: Only the improved Siamese network model is used to calculate the relevance for paragraph ranking, without using the hierarchical Transformers model for relevance clustering deduplication.

[0155] RCHT: The method proposed in this invention.

[0156] The automatic scoring results of the abstract generation are shown in Table 3, and the manual scoring results of the abstract generation are shown in Table 4.

[0157] Table 3. Automatic scoring results of the abstract generation.

[0158]

[0159] Table 4. Human scoring results of the abstract generation.

[0160]

[0161] As can be seen from the results in Tables 3 and 4, the method of the present invention achieves good results in multi-document automatic summarization tasks regardless of whether automatic or manual evaluation is used.

[0162] Example 3

[0163] In order to implement the method corresponding to Embodiment 1 above and achieve the corresponding functions and technical effects, a multi-document summary generation system is provided below.

[0164] A multi-document summarization system, comprising:

[0165] The relevance score evaluation module is used to evaluate the relevance scores of document paragraphs and headings in a multi-document scenario using an improved Siamese neural network.

[0166] The paragraph subset extraction module is used to combine a hierarchical clustering deduplication method based on relevance to extract a subset of paragraphs from the document paragraph set that have an importance greater than a set importance level and a redundancy level lower than a set redundancy level, based on the relevance score.

[0167] The summary text generation module is used to process the paragraph subset based on a multi-level Transformer summary generation device to generate multi-document summary text; the multi-level Transformer summary generation device has cross-document learning capabilities.

[0168] Example 4

[0169] This invention provides an electronic device including a memory and a processor. The memory stores a computer program, and the processor runs the computer program to enable the electronic device to perform the multi-document digest generation method provided in Embodiment 1.

[0170] In practical applications, the aforementioned electronic devices can be servers.

[0171] In practical applications, electronic devices include: at least one processor, memory, bus, and communication interface.

[0172] The processor, communication interface, and memory communicate with each other via a communication bus.

[0173] A communication interface is used to communicate with other devices.

[0174] The processor is used to execute programs, specifically the methods described in the above embodiments.

[0175] Specifically, the program may include program code, which includes computer operation instructions.

[0176] The processor may be a central processing unit (CPU), an application-specific integrated circuit (ASIC), or one or more integrated circuits configured to implement embodiments of the present invention. The electronic device may include one or more processors of the same type, such as one or more CPUs; or it may include processors of different types, such as one or more CPUs and one or more ASICs.

[0177] Memory is used to store programs. Memory may include high-speed RAM, and may also include non-volatile memory, such as at least one disk drive.

[0178] Based on the description of the above embodiments, this application provides a storage medium storing computer program instructions thereon, which can be executed by a processor to implement the methods described in any embodiment.

[0179] The multi-document digest generation system provided in this application exists in various forms, including but not limited to:

[0180] (1) Mobile communication devices: These devices are characterized by their mobile communication capabilities and primarily aim to provide voice and data communication. These terminals include: smartphones (e.g., iPhones), multimedia phones, feature phones, and low-end phones, etc.

[0181] (2) Ultra-mobile personal computer devices: These devices fall under the category of personal computers, possessing computing and processing capabilities, and generally also have mobile internet access capabilities. These terminals include PDAs, MIDs, and UMPCs, such as the iPad.

[0182] (3) Portable entertainment devices: These devices can display and play multimedia content. This category includes: audio and video players (such as iPods), handheld game consoles, e-books, as well as smart toys and portable car navigation devices.

[0183] (4) Other electronic devices with data interaction functions.

[0184] Specific embodiments of the subject matter have now been described. Other embodiments are within the scope of the appended claims. In some cases, the actions described in the claims can be performed in a different order and still achieve the desired result. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing can be advantageous.

[0185] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0186] For ease of description, the above apparatus is described by dividing it into various functional units. Of course, in implementing this application, the functions of each unit can be implemented in one or more software and / or hardware components. Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0187] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0188] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0189] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0190] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0191] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0192] Computer-readable media, including both permanent and non-permanent, removable and non-removable media, can store information using any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, and CD-ROM.

[0193] Digital multifunction optical disc (DVD) or other optical storage, magnetic cassette tape, magnetic tape, disk storage or other magnetic storage devices

[0194] Or any other non-transmission medium that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include transient media, such as modulated data signals and carrier waves.

[0195] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0196] This application can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform specific transactions or implement specific abstract data types. This application can also be practiced in distributed computing environments where transactions are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0197] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the descriptions are relatively simple; relevant parts can be referred to the method section.

[0198] This document uses specific examples to illustrate the principles and implementation methods of the present invention. The descriptions of the above embodiments are only for the purpose of helping to understand the method and core ideas of the present invention. Furthermore, those skilled in the art will recognize that, based on the ideas of the present invention, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of the present invention.< / t>

Claims

1. A method for multi-document summarization, the method comprising: The application relates to a method for generating a multi-document summary text, comprising the following steps: The method comprises the following steps: A document cluster comprising multiple documents is acquired; each document comprises a title and a body; The bodies of the multiple documents in each document cluster are split into multiple document paragraph sets according to line breaks; The document paragraph sets are input into the improved twin neural network to generate the importance of each document paragraph and the corresponding title for the finally generated summary text; The document paragraph sets are reordered according to the correlation scores to generate reordered document paragraph sets; The top K paragraphs in the reordered document paragraph sets are selected as a data subset containing key information; K is a positive integer greater than or equal to 1; the importance is the correlation score of the document paragraph and the title; the improved structure of the improved twin neural network comprises an encoding layer structure, an attention layer structure and an output layer structure; The encoding layer structure is as follows: { ,…, }=Bi-LSTMt({ ,…, }) { ,…, }=Bi-LSTMp({ ,…, }) wherein, the encoding result of the nth word of the title, the encoding result of the mth word of the paragraph; after encoding, , input into the Bi-LSTM layer for encoding, the result of the encoding of the title, the result of the encoding of the paragraph; n is the number of title tokens; m is the number of paragraph tokens; The attention layer structure is as follows: { ,…, }=self-attt({ ,…, }) { ,…, }=self-attp({ ,…, }) wherein, to obtain the vector representation of the input text sequence; to obtain the vector representation of the input text sequence; to obtain the vector representation of the input text sequence; to obtain the vector representation of the input text sequence; to obtain the vector representation of the input text sequence; and to obtain the vector representation of the input text sequence; The output layer structure is as follows: r = sigmoid(W s (tanh(W t + b t ))+b s ) where W s is a first weight parameter for the activation function; W t is a second weight parameter for the activation function, b t is a first offset parameter for the activation function; b s is a second offset parameter for the activation function; r is an association score between the title and the paragraph collection; In combination with a correlation-based hierarchical clustering deduplication method, a paragraph subset with an importance greater than a set importance and a redundancy lower than a set redundancy is extracted from the document paragraph set according to the correlation score, and the method comprises the following steps: A correlation score set is acquired, and a correlation adjacency matrix is established according to the correlation score set; The correlation adjacency matrix is taken as input, and a clustering result set is generated by combining the correlation-based hierarchical clustering deduplication method; The probability distribution of all clustering results in the clustering result set is calculated by using Gini impurity, and a clustering result with the most uniform distribution is selected; Each clustering center in the clustering result with the most uniform distribution is taken as a correlation score clustering data subset; A paragraph subset corresponding to the correlation score clustering data subset is determined according to the data subset containing key information; the paragraph subset is a paragraph subset with an importance greater than a set importance and a redundancy lower than a set redundancy in the document paragraph set; A multi-document summary text is generated by processing the paragraph subset by using a multi-level Transformer-based summary generation device; the multi-level Transformer-based summary generation device has cross-document learning capability.

2. The multi-document summarization generation method of claim 1, wherein, The multi-level Transformer-based summary generation device processes the paragraph subset to generate a multi-document summary text, and the method comprises the following steps: The paragraph subset is taken as input of the multi-level Transformer-based summary generation device, and the paragraph subset fusion global and local encoding vectors are generated by using an encoder of the multi-level Transformer-based summary generation device; The encoding vectors are decoded into natural language by using a decoder of the multi-level Transformer-based summary generation device; the natural language is a multi-document summary text.

3. The multi-document summarization generation method of claim 2, wherein, The paragraph subset is taken as an input of the multi-level Transformer summary generation device, and an encoder of the multi-level Transformer summary generation device generates the paragraph subset fusion global and local encoding vector, specifically including that the encoder includes 6 layers of local network structure and 2 layers of global network structure; The local network structure is used to encode each paragraph in the paragraph set to generate the local feature of each paragraph; The global network structure is used to perform cross-document learning between each paragraph in the paragraph set to generate the global feature of each paragraph; The local feature and the global feature are fused to generate the paragraph subset fusion global and local encoding vector.

4. A multi-document summarization generation system, characterized by, The multi-document summary generation system performs the multi-document summary generation method in any one of claims 1-3, and the multi-document summary generation system includes: A correlation score evaluation module is configured to evaluate the correlation score of the document paragraph and the title in the multi-document case by using an improved twin neural network; A paragraph subset extraction module is configured to extract a paragraph subset with greater importance and lower redundancy from the paragraph set according to a correlation-based hierarchical clustering deduplication method, wherein the importance is greater than a set importance and the redundancy is lower than a set redundancy; An abstract text generation module is configured to process the paragraph subset by using a multi-level Transformer summary generation device to generate a multi-document summary text, wherein the multi-level Transformer summary generation device has a cross-document learning capability.

5. An electronic device, comprising: The electronic device includes a memory and a processor, the memory is configured to store a computer program, and the processor is configured to run the computer program to enable the electronic device to perform the multi-document summary generation method in any one of claims 1-3.

6. A computer-readable storage medium, characterized in that, The computer program is stored in the memory and is executed by the processor to implement the multi-document summary generation method in any one of claims 1-3.

Images