A method and device for automatically generating extractive summaries

Abstract

The present invention provides a method and device for automatically generating an extractive summary. The method comprises: obtaining the summaries added by m at the current moment i i The document set D consists of i , get the document collection stream DL i ; D i Each sentence of each document is represented as a vector of equal dimension, and D is obtained by weighted summation of the sentence vectors. i The vector representation of each document of D i and DL i Based on the vector representation of D i The importance of each sentence is scored, and the document collection flow DL at the previous moment is analyzed based on the score i‑1 The summary is updated to generate DL i When a new document is added, the present invention can automatically update the summary by only processing the new document; the present invention can eliminate the adverse effects of different document capacities on summary generation by representing each document, each document set and document set stream as a vector of the same dimension.

Description

Technical Field

[0001] The present invention belongs to the technical field of document processing, and in particular relates to a method and device for automatically generating an extractive summary. Background Art

[0002] With the explosive growth of document information on the Internet, the problem of text information overload is becoming increasingly serious. Users urgently need to automatically summarize a large number of documents and convert them into short texts containing key information. According to the input type, document summarization can be divided into single-document summarization and multi-document summarization. Single-document summarization obtains summaries from a single document; multi-document summarization obtains summaries from a collection of documents. According to the output type, it can be divided into generative summarization and extractive summarization. Generative summarization starts from the original text and generates new words and sentences to form a summary; extractive summarization extracts key sentences from the source document to form a summary, and the summary all comes from the original text. Extractive summarization technology directly selects key content from the original text to form a summary, which is mainly divided into traditional methods and deep learning methods. The simplest traditional method directly combines the title and the first paragraph of the document into a summary; in addition, high-frequency words, phrases, short sentences, etc. can also be combined into a summary; the deep learning method models the extractive summarization task as a sequence labeling task, and classifies each sentence in the text into two categories. The positive class represents that the sentence belongs to the summary, and the negative class represents that the sentence does not belong to the summary. All sentences with positive labels constitute the summary.

[0003] Existing automatic summary generation technologies can only show good results when processing a fixed document set. However, when a new document is added, if new summary content is to be obtained, the existing technology can only combine the already processed documents and the new documents into one set and reprocess all the documents. Summary of the invention

[0004] In order to solve the above problems existing in the prior art, the present invention provides a method and device for automatically generating an extractive summary.

[0005] In order to achieve the above object, the present invention adopts the following technical solutions.

[0006] In a first aspect, the present invention provides a method for automatically generating an extractive summary, comprising the following steps:

[0007] Get the current moment i joined by m i The document set D consists of i , get the document collection stream DL i ={D r}, r = 1, 2, ..., i;

[0008] D i Each sentence of each document is represented as a vector of equal dimension, and D is obtained by weighted summation of the sentence vectors. iThe vector representation of each document of D i and DL i The vector representation of ;

[0009] Based on the vector representation of D i The importance of each sentence is scored, and the document collection flow DL at the previous moment is analyzed based on the score i-1 The summary is updated to generate DL i Summary.

[0010] Furthermore, we use the open source sentence embedding tool model CoSENT to transform D i The jth document Represented as a sentence vector set {V k ij}, the sentence vectors are all 768-dimensional column vectors; where j = 1, 2, ..., m i ,k=1,2,…,m ij , V k ij for The kth sentence in The vector representation of m ij for The number of sentences in .

[0011] Furthermore, D i The jth document in The vector representation V j i for:

[0012]

[0013]

[0014]

[0015] In the formula, is a scalar calculated based on known quantities and learnable parameters, w α 、b α is a learnable parameter, where w α is a 768-dimensional row vector, b α is a scalar.

[0016] Furthermore, D i The vector representation methods include:

[0017] Right D i Each sentence in each document in the word segmentation process is obtained is the qth word, q = 1, 2, ..., Q, Q is the total number of words;

[0018] Calculate the score for each word:

[0019]

[0020]

[0021]

[0022] In the formula, for The score, for In D i The number of occurrences in W (D i ) is D i The sum of the number of occurrences of all words in , D i Contains The number of documents;

[0023] D i The vector representation of is:

[0024]

[0025]

[0026] Where V i D i The vector representation of .

[0027] Furthermore, DL i The vector representation of is:

[0028]

[0029] In the formula, E i is the document collection stream DL at the current moment i i ={D r}, where r = 1, 2,…, i.

[0030] Furthermore, DL i The vector representation of is:

[0031]

[0032] In the formula, i≥2.

[0033] Furthermore, for D i The methods for scoring the importance of each sentence include:

[0034] Calculate D as follows i The jth document The kth sentence of Score:

[0035]

[0036] In the formula, and They are document-level importance score, sentence-level importance score and word-level importance score, respectively. The calculation formula is as follows:

[0037]

[0038]

[0039]

[0040] In the formula, “·” indicates calculating the dot product of vectors, and “||” indicates finding the modulus of the vector.

[0041] Furthermore, the method further includes performing gradient descent optimization on the learnable parameters, and the loss function is:

[0042]

[0043] Where, L j D i The jth document The loss function of .

[0044] Furthermore, DL is generated i The summary methods include:

[0045] According to the order of sentence importance from high to low, i Sentences and DL i-1 The sentences of the abstract are uniformly sorted;

[0046] Generate DL based on the top N sentences i Summary.

[0047] In a second aspect, the present invention provides an extractive summary automatic generation device, comprising:

[0048] The document acquisition module is used to obtain the document that is added by m at the current moment. i The document set D consists of i , get the document collection stream DL i ={D r}, r = 1, 2, ..., i;

[0049] Vector representation module, used to transform D i Each sentence of each document is represented as a vector of equal dimension, and D is obtained by weighted summation of the sentence vectors. i The vector representation of each document of Di and DL i The vector representation of ;

[0050] A summary generation module is used to represent D based on the vector i The importance of each sentence is scored, and the document collection flow DL at the previous moment is analyzed based on the score i-1 The summary is updated to generate DL i Summary.

[0051] Compared with the prior art, the present invention has the following beneficial effects.

[0052] The present invention obtains the document set D added at the current time i i , get the document collection stream D, and convert D i Each sentence of each document is represented as a vector of equal dimension, and D is obtained by weighted summation of the sentence vectors. i The vector representation of each document of D i and D, based on the vector representation of D i The importance of each sentence is scored, and the document collection flow DL at the previous moment is analyzed based on the score i-1 The summary is updated to generate DL i The present invention can automatically update the summary by processing the newly added document only when a new document is added; the present invention can eliminate the influence of different document capacities on summary generation by representing each document, each document set and document set stream as a vector of the same dimension. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 The present invention is a flowchart of an automatic method for generating an extractive summary according to an embodiment of the present invention.

[0054] Figure 2 A schematic diagram is generated for the embodiment of the present invention.

[0055] Figure 3 The block diagram is a device for automatically generating an extractive summary according to an embodiment of the present invention. DETAILED DESCRIPTION

[0056] In order to make the purpose, technical solution and advantages of the present invention clearer and more understandable, the present invention is further described below in conjunction with the accompanying drawings and specific implementation methods. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0057] Figure 1This is a flowchart of an extractive summary automatic generation method according to an embodiment of the present invention, comprising the following steps:

[0058] Step 101, obtain the current moment i joined by m i The document set D consists of i , get the document collection stream DL i ={D r}, r = 1, 2, ..., i;

[0059] Step 102: D i Each sentence of each document is represented as a vector of equal dimension, and D is obtained by weighted summation of the sentence vectors. i The vector representation of each document of D i and DL i The vector representation of ;

[0060] Step 103: based on the vector representation, i The importance of each sentence is scored, and the document collection flow DL at the previous moment is analyzed based on the score i-1 The summary is updated to generate DL i Summary.

[0061] This embodiment provides a method for automatically generating summaries based on a document collection stream. A large number of documents are produced and processed and then published on the Internet. Users can search and browse efficiently according to the topics they are interested in, and organize document collections under specific topics; at the same time, users will continuously enrich such document collections and continue to append single or multiple new documents to them. Obviously, this process forms the data form of a document collection stream. For this dynamically growing document collection stream, this embodiment proposes an automatic summary generation method to extract its core content into the summary, and realize automatic updating of the summary after the arrival of a new document collection.

[0062] In this embodiment, step 101 is mainly used to obtain the document set at the current moment. In this embodiment, the current moment is represented by moment i or the i-th moment. Each moment is a moment when a new document set is added. The moments when a new document set is added are generally not at equal intervals. If the document set added at the first moment is D1, the document set added at the second moment is D2, and the document set added at the i-th moment is D i , then the document collection stream obtained is DL i ={D r}, r = 1, 2, ..., i; can also be expressed as DL i ={D1,D2,…,D i}.like Figure 2 shown.

[0063] In this embodiment, step 102 is mainly used to represent the document as a vector. In this embodiment, each sentence is first represented as a vector. Different sentences have different lengths (the number of words they contain is different). In order to facilitate processing of different sentences (such as weighted summation), this embodiment represents all sentences as vectors of equal dimensions. Then, by performing D i The vector representations of all sentences in each document are weighted summed to obtain the vector representation of each document; D is obtained by weighted summing the vector representations of each document. i The document set flow DL is obtained by weighted summing the vector representations of each document set. i This embodiment can eliminate the impact of different document capacities on summary generation by representing each document, each document set, and document set stream as a vector.

[0064] In this embodiment, step 103 is mainly used to automatically generate a summary. i The importance of each sentence is scored, and then the document collection flow DL at the previous moment is analyzed based on the score i-1 The summary is updated to obtain the document collection stream DL at the current moment i Abstract. i Replace the sentences with the highest scores in DL i The existing automatic summarization technology can only show good results when processing a fixed document set. However, when a new document is added, if new summary content is to be obtained, the already processed documents and the new document can only be combined into one set and all documents can be processed again. i , to achieve the purpose of summary updating, and at the same time, ensure that the obtained summary can cover both past documents and new documents.

[0065] As an optional embodiment, the open source sentence vector tool model CoSENT is used to transform D i The jth document Represented as a sentence vector set {V k ij}, the sentence vectors are all 768-dimensional column vectors; where j = 1, 2, ..., m i ,k=1,2,…,m ij , V k ij for The kth sentence in The vector representation of m ij for The number of sentences in .

[0066] This embodiment provides a technical solution for representing a document as a set of sentence vectors. This embodiment uses the open source sentence vector tool model CoSENT. By representing each sentence as a sentence vector, a sentence vector set for each document can be obtained. For example, D i The jth document The kth sentence of Represented as a vector V k ij , get the jth document The sentence vector set {V k ij}, k=1,2,…,m ij ,Right now Of course, all sentence vectors are vectors of equal dimensions. This embodiment also limits the dimension (size) of the sentence vector, and the sentence vectors of this embodiment are all 768-dimensional column vectors. It is worth noting that this embodiment only provides a preferred implementation method and does not deny or exclude other feasible implementation methods, such as sentence vectors with dimensions different from 768.

[0067] As an optional embodiment, D i The jth document in The vector representation V j i for:

[0068]

[0069]

[0070]

[0071] In the formula, is a scalar calculated based on known quantities and learnable parameters, w α 、b α is a learnable parameter, where w α is a 768-dimensional row vector, b α is a scalar.

[0072] This embodiment provides a technical solution for vectorizing a document. The vector set {V k ij Each sentence vector V in} k ij Perform weighted summation to obtain the document The vector representation V j i , see formula (1) above. V k ij The weighting coefficient is It reflects the importance of the sentence, and its expression is shown in the above formula (2). satisfy Therefore, for V k ij The weighted sum is actually the weighted mean. Calculated by formula (3), it is based on the known quantity V k ij and learnable parameters w α 、b α The calculated scalar. Since w α is a 768-dimensional row vector, V k ij is a 768-dimensional column vector, so is a scalar, b α It is also a scalar.

[0073] As an optional embodiment, D i The vector representation methods include:

[0074] Right D i Each sentence in each document in the word segmentation process is obtained is the qth word, q = 1, 2, ..., Q, Q is the total number of words;

[0075] Calculate the score for each word:

[0076]

[0077]

[0078]

[0079] In the formula, for The score, for In D i The number of occurrences in W (D i ) is D i The sum of the number of occurrences of all words in , D i Contains The number of documents;

[0080] D i The vector representation of is:

[0081]

[0082]

[0083] Where V i Di The vector representation of .

[0084] This embodiment provides a method for document set D i A technical solution for vector representation. i The vector representation V of each document in j i Perform weighted summation to get D i The vector representation V i , see formula (7). V j i The weight coefficient of is a fraction, and the numerator of the fraction is document d j i The sum of the scores of all words in the fraction, with the denominator being D i The sum of the scores of all words in D i Each sentence in each document in the word segmentation process is obtained. Score The calculation is based on formulas (4) to (6), and the calculation process complies with the TF-IDF (term frequency-inverse document frequency) feature calculation method. Score Can be used to measure words In D i The greater the value, the higher the importance.

[0085] As an optional embodiment, DL i The vector representation of is:

[0086]

[0087] In the formula, E i is the document collection stream DL at the i-th moment i ={D r}, where r = 1, 2,…, i.

[0088] This embodiment provides a method for processing a document collection stream DL i A technical solution for vector representation. This embodiment obtains the vector representation of the document set stream by calculating the average value of the vector representation of each document set, as shown in formula (9). The average value is actually a weighted summation, but the weighting coefficient is 1 / i.

[0089] As an optional embodiment, DL i The vector representation of is:

[0090]

[0091] In the formula, i≥2.

[0092] This embodiment provides a method for DL i Another technical solution for vector representation. This embodiment is actually a variation of the previous embodiment, and is a recursive expression obtained by varying equation (9), as shown in equation (10). According to the recursive expression, only E i-1 and V i Two quantities quickly calculate E i When i = 1, E1 = V1; when i ≥ 2, E i It can be regarded as a vector representation of a document collection stream E i-1 and D i The vector representation V i The weighted sum of .

[0093] As an optional embodiment, i The methods for scoring the importance of each sentence include:

[0094] Calculate D as follows i The jth document The kth sentence of Score:

[0095]

[0096] In the formula, and They are document-level importance score, sentence-level importance score and word-level importance score, respectively. The calculation formula is as follows:

[0097]

[0098]

[0099]

[0100] In the formula, “·” indicates calculating the dot product of vectors, and “||” indicates finding the modulus of the vector.

[0101] This example provides a i A technical solution for scoring the importance of each sentence. In this embodiment, the score of each sentence is equal to the product of three scoring factors, namely, the document-level importance score, the sentence-level importance score, and the word-level importance score, as shown in formula (11). This embodiment uses three scoring factors to score each sentence, which can more comprehensively examine the importance of the sentence, making the score of each sentence more objective and reasonable, thereby improving the accuracy of summary generation. The document-level importance score formula is as shown in formula (12), and its score is composed of the sentence Document j i The vector representation V ji Streaming DL with document collections i The vector representation E i The cosine similarity of , that is, document d j i Streaming DL with document collections i The more similar, the sentences The higher the score, the higher the importance score. The sentence-level importance score formula is as follows (13), and its score is obtained by The vector representation of the document d is weighted summed up j i The vector represents the weighting coefficient The calculation method is as shown in formula (2). The word-level importance score formula is as shown in formula (14), and its score expression is a fraction, the numerator is equal to the sentence The sum of all word scores in the document set D i The sum of the scores of all words in .

[0102] As an optional embodiment, the method further includes performing gradient descent optimization on the learnable parameters, and the loss function is:

[0103]

[0104] Where, L j D i The jth document The loss function of .

[0105] This embodiment provides a technical solution for gradient descent optimization of learnable parameters. This embodiment uses the gradient descent method to optimize the learnable parameters (such as w α 、b α ) is optimized, and its loss function is as shown in formula (15). From formula (15), it can be seen that the loss function corresponds to document d j i , that is, different documents use different loss functions. This embodiment sets the loss function in this way to ensure that the document representation of each document is highly consistent with the feature representation of the document set to which it belongs.

[0106] As an optional embodiment, generate DL i The summary methods include:

[0107] According to the order of sentence importance from high to low, i Sentences and DL i-1 The sentences of the abstract are uniformly sorted;

[0108] Generate DL based on the top N sentences i Summary.

[0109] This embodiment provides a method for generating DL i This embodiment is based on the document collection stream DL at the last moment. i-1 The summary is updated to generate the current document collection stream DL i The update method is: first, according to the importance score of the sentence, D i Sentences and DL i-1 The sentences of the abstract are uniformly sorted, and the higher the score, the higher the ranking, that is, the two parts of sentences are sorted in order from high to low scores; then the top N sentences are selected to generate the abstract. When generating the abstract, the N sentences can be sorted in order from high to low scores, or in the order of the time when the document was added and the order in which the sentences appear in the document.

[0110] Figure 3 This is a schematic diagram of the composition of an embodiment of the present invention, wherein the device comprises:

[0111] The document acquisition module 11 is used to obtain the document added by m at the current moment i i The document set D consists of i , get the document collection stream DL i ={D r}, r = 1, 2, ..., i;

[0112] The vector representation module 12 is used to represent D i Each sentence of each document is represented as a vector of equal dimension, and D is obtained by weighted summation of the sentence vectors. i The vector representation of each document of D i and DL i The vector representation of ;

[0113] The summary generation module 13 is used to represent D based on the vector representation. i The importance of each sentence is scored, and the document collection flow DL at the previous moment is analyzed based on the score i-1 The summary is updated to generate DL i Summary.

[0114] The device of this embodiment can be used to perform Figure 1 The technical solution of the method embodiment shown has similar implementation principles and technical effects, which will not be repeated here.

[0115] The above is only a specific embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed by the present invention should be included in the protection scope of the present invention. Therefore, the protection scope of the present invention shall be subject to the protection scope of the claims.

Claims

1. A method for automatically generating an extractive summary, characterized in that: The following steps are involved: Get the current moment i joined by m i The document set D consists of i , get the document collection stream DL i ={D r }, r = 1, 2, ..., i; D i Each sentence of each document is represented as a vector of equal dimension, and D is obtained by weighted summation of the sentence vectors. i The vector representation of each document of D i and DL i The vector representation of ; Based on D i The vector representation of D i The importance of each sentence is scored, and the document collection flow DL at the previous moment is analyzed based on the score i-1 The summary is updated to generate DL i Summary of Using the open source sentence embedding tool model CoSENT, D i The jth document Represented as a set of sentence vectors The sentence vectors are all 768-dimensional column vectors; where j = 1, 2, ..., m i ,k=1,2,…,m ij , for The kth sentence in The vector representation of m ij for The number of sentences in D i The jth document in Vector representation of for: In the formula, is a scalar calculated based on known quantities and learnable parameters, w α 、b α is a learnable parameter, where w α is a 768-dimensional row vector, b α is a scalar; D i The vector representation methods include: Right D i Each sentence in each document in the word segmentation process is obtained is the qth word, q = 1, 2, ..., Q, Q is the total number of words; Calculate the score for each word: In the formula, for The score, for In D i The number of occurrences in W (D i ) is D i The sum of the number of occurrences of all words in , D i Contains The number of documents; D i The vector representation of is: Where V i D i The vector representation of ; DL i The vector representation of is: or: In the formula, E i is the document collection stream DL at the current moment i i ={D r }, where r = 1, 2, …, i, i ≥ 2; Right D i The methods for scoring the importance of each sentence include: Calculate D as follows i The jth document The kth sentence of Score: In the formula, and They are document-level importance score, sentence-level importance score and word-level importance score, respectively. The calculation formula is as follows: In the formula, "●" indicates calculating the dot product of vectors, and "||" indicates finding the modulus of the vector.

2. The method for automatically generating an extractive summary according to claim 1, characterized in that: The method also includes performing gradient descent optimization on the learnable parameters, and the loss function is: Where, L j D i The jth document The loss function of .

3. The method for automatically generating an extractive summary according to claim 1, characterized in that: Generate DL i The summary methods include: According to the order of sentence importance from high to low, i Sentences and DL i-1 The sentences of the abstract are uniformly sorted; Generate DL based on the top N sentences i Summary.

4. An extractive summary automatic generation device, characterized in that: include: The document acquisition module is used to obtain the document that is added by m at the current moment. i The document set D consists of i , get the document collection stream DL i ={D r }, r = 1, 2, ..., i; Vector representation module, used to transform D i Each sentence of each document is represented as a vector of equal dimension, and D is obtained by weighted summation of the sentence vectors. i The vector representation of each document of D i and DL i The vector representation of ; A summary generation module is used to represent D based on the vector i The importance of each sentence is scored, and the document collection flow DL at the previous moment is analyzed based on the score i-1 The summary is updated to generate DL i Summary of Using the open source sentence embedding tool model CoSENT, D i The jth document Represented as a set of sentence vectors The sentence vectors are all 768-dimensional column vectors; where j = 1, 2, ..., m i ,k=1,2,…,m ij , for The kth sentence in The vector representation of m ij for The number of sentences in D i The jth document in Vector representation of for: In the formula, is a scalar calculated based on known quantities and learnable parameters, w α 、b α is a learnable parameter, where w α is a 768-dimensional row vector, b α is a scalar; D i The vector representation methods include: Right D i Each sentence in each document in the word segmentation process is obtained is the qth word, q = 1, 2, ..., Q, Q is the total number of words; Calculate the score for each word: In the formula, for The score, for In D i The number of occurrences in W (D i ) is D i The sum of the number of occurrences of all words in , D i Contains The number of documents; D i The vector representation of is: Where V i D i The vector representation of ; DL i The vector representation of is: or: In the formula, E i is the document collection stream DL at the current moment i i ={D r }, where r = 1, 2, …, i, i ≥ 2; Right D i The methods for scoring the importance of each sentence include: Calculate D as follows i The jth document The kth sentence of Score: In the formula, and They are document-level importance score, sentence-level importance score and word-level importance score, respectively. The calculation formula is as follows: In the formula, "●" indicates calculating the dot product of vectors, and "||" indicates finding the modulus of the vector.

Images