Text summarization system, method, and program

Abstract

To properly and clearly summarize the whole text.SOLUTION: There is provided a sentence summarization system comprising: a cutting part 10 for cutting a text T as two blocks from a beginning sentence (text1) for every 5 sentences, then deviating the blocks for one sentence, for sequentially cutting blocks; a dispersion expression part 11 for digitizing the sentences for embedding the same in a vector space; a similarity determining part 12 for determining a level of a cosine similarity to the dispersion expression; a topic dividing part 13 for determining that sentences for which it is determined that similarities between the sentences are great, as a same topic, and determining that sentences for which it is determined that similarities between the sentences are low, as different topics, then estimating a position where the topic is switched, and dividing the sentences for each topic; and an extraction type summarization part 14 for inputting a learned model which has learned the divided sentences divided for each topic in advance, in an extraction type summarization model, coupling output and summarized sentences, for generating a summarized sentence of the text, where the whole sentence of the text is divided for each topic, then the sentence is summarized for each topic, for generating the summarized sentence of the text.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a system, method, device and program for summarizing texts used in minutes of meetings and call records of call centers, etc. [Background technology]

[0002] Conventionally, minutes of meetings, conferences, and other events have been converted into text documents using a speech recognition system, which are then manually summarized by a person in charge. However, because this process requires a lot of time and effort, a text summary generation technology has been proposed in recent years that simply summarizes the converted minutes (see, for example, Patent Document 1). In this technology, before summarization, formulaic expressions are registered as unnecessary sentences in a training data table for determining unnecessary sentences, and the formulaic expressions are removed to improve the accuracy of the summary. In addition, a certain number of sentences are extracted from the document to be summarized in order of appearance in a window, and the document within the window is summarized using an extractive summarization technology that uses distributed representations. A summary of the entire document is generated by sliding the window sentence by sentence. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-35272 Summary of the Invention [Problem to be solved by the invention]

[0004] However, while Patent Document 1 can delete unnecessary sentences, it requires time-consuming registration of the sentences in a training data table for unnecessary sentence identification. Furthermore, identifying unnecessary sentences not only takes time, but insufficient identification can result in reduced summarization accuracy. Furthermore, a certain number of sentences must be extracted from the document to be summarized in order of appearance into a window. For example, to extract 100 lines in 10 sentence increments, the window must be shifted by one sentence every 10 sentences, repeatedly, which is time-consuming. Furthermore, in systems that summarize text, summarization using a pre-trained model such as BERT limits the number of tokens that can be input, so long sentences must be divided into segments according to their input length. Here, tokens refer to words that divide a sentence into meaningful units. However, dividing a sentence into segments by topics can make it difficult to grasp semantic relationships within the input sentence, or topics may be omitted from the output summary.

[0005] The present invention has been made to solve the above-mentioned problems, and aims to provide a text summarization system, method, device, and program that can accurately and clearly summarize the entire text by focusing on topics (agenda, topics, topics or events, reports, schedules) as components of the text, dividing the topics, and summarizing each topic. [Means for solving the problem]

[0006] A text summarization system according to one aspect of the present invention includes an extraction unit that extracts a text containing a plurality of sentences into two consecutive blocks of a predetermined number of sentences each, starting from the first sentence, and sequentially extracts the blocks by shifting the blocks by one sentence at a time until the last sentence is reached; a similarity determination unit that determines the similarity between the blocks contained in each of the two sequentially extracted blocks from a vector representation obtained by vectorizing the sentences in each of the two blocks; a topic division unit that divides the text into topics based on the relationship between the position and similarity between each of the blocks; and an extraction-type summarization unit that summarizes the text for each divided topic, wherein the topic division unit calculates a parameter that indicates the depth of the similarity at the position between each block relative to the similarity peak before and after it, automatically sets a threshold value after calculating the parameter, and calculates the similarity between the blocks by using the parameter but Threshold The position where the value exceeds the limit and is a peak, and there is no dividing point nearby. Switching topics in the main text It is a position Estimate Then, the text is extracted based on the estimated transition position. Split the topic. In addition, the extraction-type summarization unit sequentially inputs the topic-divided sentences into an extraction-type summarization model that is a fine-tuned version of a pre-trained model, and combines the output summarized sentences to create summarized sentences of the main text.

[0007] According to this, Each divided topic is expressed as a specific trend or grouping within a certain topic within the whole, so the summary for each topic is accurate and clear, redundancy is eliminated, and the summary of the entire text is also accurate and clear. Furthermore, it is possible to accurately estimate the points where topics change and segment the text into topics, resulting in accurate and clear summaries for each topic.

[0010] The text summarization system of the present invention preferably includes an extraction-type summarization unit that sequentially inputs topic-divided text into an extraction-type summarization model that is fine-tuned from a pre-trained model, combines the output summarized text, and creates a summarized text of the main text. With this configuration, the entire text can be summarized automatically and quickly.

[0011] In one aspect of the present invention, a method for summarizing text includes the steps of: On the computer, The text containing multiple sentences is extracted into two consecutive blocks of a predetermined number of sentences from the first sentence. Let , and shift this block sentence by sentence until the last sentence is reached. to makea step of determining the similarity between the blocks contained in each of the two blocks from the vector representation obtained by vectorizing the sentences in each of the two blocks extracted sequentially; a step of dividing the topics of the text based on the relationship between the positions and similarities between the blocks; and a step of summarizing the text for each divided topic. Let it run In the step of dividing the topic of the text, a parameter is calculated that indicates the depth of the similarity at the position between each block relative to the peak of similarity before and after it. Let ,After finding the parameters, the threshold is automatically set. Let , parameters but Threshold The position where the value exceeds the limit and is a peak, and there is no dividing point nearby. Switching topics in the main text It is a position Estimate Based on the estimated transition position, Split Topic Let do. In addition, the topic-divided sentences are sequentially input into an extraction-type summarization model that is a fine-tuned version of a pre-trained model, and the output summarized sentences are combined to create a summarized sentence of the main text.

[0012] According to this, Each divided topic is expressed as a specific trend or grouping within a certain topic within the whole, so the summary for each topic is accurate and clear, redundancy is eliminated, and the summary of the entire text is also accurate and clear.

[0021] A program according to one aspect of the present invention causes a computer to execute the following operations: extracting a main text containing a plurality of sentences into two consecutive blocks each consisting of a predetermined number of sentences from the first sentence, shifting the blocks by one sentence at a time until the last sentence is reached; determining the similarity between the blocks contained in each of the two blocks from a vector representation obtained by vectorizing the sentences in each of the sequentially extracted blocks; dividing the main text into topics based on the relationship between the position and similarity between each of the blocks; and summarizing the text for each divided topic; and in dividing the main text into topics, causes the computer to calculate a parameter representing the depth of the similarity at the position between each block relative to the similarity peak before and after it, and automatically set a threshold value after calculating the parameter, and sets the parameter but Threshold The position where it exceeds and reaches its peak Switching topics in the main text It is a position Estimate Based on the estimated switching position, Split the topic. In addition, the computer inputs the topic-divided sentences in order into an extraction-type summarization model that is a fine-tuned version of a pre-trained model, and combines the output summarized sentences to create a summarized sentence of the main text.

[0022] According to this, Each divided topic is expressed as a specific trend or grouping within a certain topic within the whole, so the summary for each topic is accurate and clear, redundancy is eliminated, and the summary of the entire text is also accurate and clear. Furthermore, it is possible to accurately estimate the points where topics change and segment the text into topics, resulting in accurate and clear summaries for each topic. [Effects of the Invention]

[0026] In the text summarization system according to claim 1 of the present invention, a text containing multiple sentences is cut into two consecutive blocks, each consisting of a predetermined number of sentences, starting from the first sentence, and these blocks are sequentially cut out by shifting them by one sentence at a time. For each block cut out sequentially, the text within the block is vectorized to determine its similarity, and topic changes are estimated based on the determined similarity, and the text is summarized for each divided topic. As a result, the summary for each topic is accurate and clear, and the summary of the entire text is also accurate and clear.

[0027] Furthermore, a text summarization system according to claim 2 of the present invention includes an extraction unit that extracts a main text containing multiple sentences into two consecutive blocks of a predetermined number of sentences starting from the first sentence and extracts the blocks sequentially by shifting each block by one sentence, a distributed representation unit that quantifies the sentences and embeds them in a vector space, a similarity determination unit that determines the cosine similarity of the distributed representations, and a topic segmentation unit that estimates the positions where the topics change and divides the text into topics by regarding sentences that are determined to have a high similarity as belonging to the same topic and sentences that are determined to have a low similarity as belonging to different topics, and by dividing the text into topics by dividing the entire text into topics and summarizing the text for each topic, it is possible to reliably estimate the positions where the topics change and divide the text into topics, resulting in accurate and clear summaries for each topic.

[0028] Furthermore, the method for summarizing text according to claim 4 of the present invention includes the steps of: extracting a main text containing multiple sentences into two consecutive blocks each having a predetermined number of sentences starting from the first sentence; shifting these blocks by one sentence and extracting them sequentially; vectorizing the sentences in the extracted block each time they are extracted and determining their similarity; and estimating topic changes based on the determined similarity and dividing the topics; and since the text is summarized for each divided topic, the summary for each topic is accurate and clear, and the summary of the entire text is also accurate and clear.

[0029] Furthermore, a text summarization method according to claim 5 of the present invention includes an extraction unit that extracts a main text including a plurality of sentences into two consecutive blocks of a predetermined number of sentences from the first sentence and extracts the blocks sequentially by shifting each sentence, a distributed representation unit that digitizes these sentences and embeds them in a vector space, a similarity determination unit that determines the degree of similarity between the distributed representations, and a topic segmentation unit that estimates the position where the topics change by regarding sentences determined to have a high similarity as the same topic and regarding sentences determined to have a low similarity as different topics, and divides the text into each topic, and the first step extracts a predetermined number of consecutive blocks from the first sentence of the main text by the extraction unit, and the second step extracts a character string for each of the sentences in the two consecutive blocks by the distributed representation unit. The method includes a second step of creating a distributed representation of the expression using a vector representation, a third step of using a similarity discriminator to discriminate the similarity of the distributed representation and output the discrimination result, and a fourth step of using a segmentation unit to shift one sentence from the block and divide it into two new consecutive blocks, repeating steps two and three for each division. When the method reaches the last sentence, the topic segmentation unit classifies sentences that are determined to have a high degree of similarity as belonging to the same topic and sentences that are determined to have a low degree of similarity as belonging to different topics, estimating the positions where the topics change, and dividing the text by topic. Since the entire text is divided into topics and the text is summarized for each topic, it is possible to reliably estimate the positions where the topics change and divide the text by topic. This results in accurate and clear summaries for each topic.

[0030] Furthermore, in the text summarization device according to claim 7 of the present invention, a text containing multiple sentences is cut into two consecutive blocks, each consisting of a predetermined number of sentences, starting from the first sentence, and these blocks are sequentially cut out by shifting them by one sentence at a time. For each block cut out sequentially, the text within the block is vectorized to determine its similarity, and topic changes are estimated based on the determined similarity, and the text is summarized for each divided topic. As a result, the summary for each topic is accurate and clear, and the summary of the entire text is also accurate and clear.

[0031] Furthermore, a text summarization device according to claim 8 of the present invention includes an extraction unit that extracts a main text containing multiple sentences into two consecutive blocks of a predetermined number of sentences starting from the first sentence and extracts the blocks sequentially by shifting each block by one sentence, a distributed representation unit that digitizes the sentences and embeds them in a vector space, a similarity determination unit that determines the degree of similarity between the distributed representations, and a topic segmentation unit that estimates the positions where the topics change and divides the text into topics by regarding sentences that are determined to have a high similarity as belonging to the same topic and sentences that are determined to have a low similarity as belonging to different topics, and by dividing the entire text into topics and summarizing the text for each topic, the device can reliably estimate the positions where the topics change and divide the text into topics, thereby resulting in accurate and clear summaries for each topic.

[0032] Furthermore, in the program according to claim 10 of the present invention, a computer is configured to extract a main text containing multiple sentences into two consecutive blocks each consisting of a predetermined number of sentences starting from the first sentence, and sequentially extract these blocks by shifting them by one sentence at a time; vectorize the sentences within each sequentially extracted block to determine their similarity; estimate topic changes based on the determined similarity and divide the topics; and summarize the sentences for each divided topic, thereby making the summary for each topic accurate and clear, and also making the summary of the entire text accurate and clear.

[0033] In addition, the program according to claim 11 of the present invention causes a computer to execute the following operations: extract a main text containing multiple sentences into two consecutive blocks of a predetermined number of sentences starting from the first sentence, and extract the blocks sequentially by shifting each block by one sentence; create a distributed representation of the character expressions of these sentences using vector representations; determine the similarity of the distributed representations; classify sentences determined to have a high similarity as belonging to the same topic and sentences determined to have a low similarity as belonging to different topics, estimate the positions where the topics change, and divide the text into topics; and divide the entire text into topics and summarize the text for each topic. This makes it possible to reliably estimate the positions where the topics change and divide the text into topics, resulting in accurate and clear summaries for each topic. [Brief explanation of the drawings]

[0034] [Figure 1] FIG. 1 is a system configuration diagram showing the overall configuration of a text summarization system according to one embodiment of the present invention. [Figure 2] FIG. 2 is an explanatory diagram that schematically illustrates the concept of the text summarization system of FIG. [Figure 3] FIG. 3 is an explanatory diagram showing an example of cutting out a sentence into blocks in the sentence summarization system of FIG. [Figure 4] FIG. 4 is an explanatory diagram showing an example in which a block is set for each predetermined number of sentences by the cutout unit. [Figure 5] FIG. 5 is an explanatory diagram conceptually showing the steps from dividing text into blocks to deriving a graph for determining similarity in the text summarization system of FIG. [Figure 6] Figures 6(A) and 6(B) are a flowchart showing the steps of repeatedly extracting blocks from the first sentence to the last sentence and determining similarity, and a flowchart showing the steps of extracting topics from the similarity graph, respectively. [Figure 7] FIG. 7 is a graph showing topic breaks based on cosine similarity in the text summarization system of FIG. [Figure 8] Figure 8 shows the graph from which the text similarity curve for the main text was derived. [Figure 9] FIG. 9 is an explanatory diagram showing an example of a topic-divided text. [Figure 10] FIG. 10 is an explanatory diagram that schematically shows the steps of inputting the sentences divided by the extraction-type summarization unit into the extraction-type summarization model in order and combining the output summary sentences. [Figure 11] FIG. 11 is an explanatory diagram showing an example of numerical values output from the model in the extraction-type summarization unit of FIG. DETAILED DESCRIPTION OF THE INVENTION

[0035] The present invention will be described below with reference to an embodiment shown in the drawings. As shown in Figures 1 and 2, a text summarization system 2 according to one embodiment of the present invention cuts a main text containing multiple sentences into two consecutive blocks spaced a predetermined number of times from the beginning, sequentially cutting out these blocks by shifting them by one sentence, vectorizing the sentences within each block and determining their similarity, estimating topic changes based on the determined similarity, and dividing the text into topics, and summarizing the text for each divided topic. In other words, in order to eliminate redundancy and unclearness in summarizing long input text, the main text is divided into topics and summarized.

[0036] As shown in Figure 1, the text summarization system 2 according to this embodiment is a system for summarizing a text T such as minutes of a meeting. The text T is input as character data in advance into hardware (hard disk, information processing unit, memory unit), a computer, or a cloud computer. In this example, a computer (PC) 3 having a central processing unit (CPU) 4, an input unit 5, an output unit 6, a display unit 7, a transmission / reception unit 8, and a memory unit 9 will be described as an example. The PC 3 stores software (programs) that perform the operations described below.

[0037] The text summarization system 2 according to this embodiment comprises an extraction unit 10, a distributed representation unit 11, a similarity determination unit 12, a topic division unit 13, and an extraction-type summarization unit 14. As shown in FIGS. 3 and 4, the extraction unit 10 extracts a plurality of texts (sentence 1, sentence 2, sentence 3, ... sentence 4) from the text. N ( N A predetermined number of sentences (5 in this embodiment) starting from the first sentence (sentence 1) of the entire text of the main text T, including the text T (where s is an arbitrary integer equal to or greater than 1), are sequentially extracted as blocks while being shifted in order. As shown in FIG. 5, the distributed representation unit 11 distributes the character representations of these sentences using vector representations. The similarity determination unit 12 determines the cosine similarity of this distributed representation (similarity 0.0 to 1.0). The topic segmentation unit 13 then considers sentences determined to have a high degree of similarity to be part of the same topic, and sentences determined to have a low degree of similarity to be part of different topics, estimating the positions where the topics change, and segmenting the text into individual topics. Furthermore, the extraction-type summarization unit 14 sequentially inputs the topic-divided sentences into an extraction-type summarization model, which is a fine-tuned version of a pre-trained model, and combines the output summarized sentences to create summarized sentences of the main text.

[0038] The extraction unit 10 extracts a certain number of sentences (five sentences in this embodiment) from the first sentence (sentence 1) of the text T (T is, for example, the story of "Momotaro" shown in FIG. 4) as shown in FIGS. 3 and 4, and defines these as one block. Two consecutive blocks are extracted from the beginning (see B1-1 and B2-1 in FIGS. 3 and 4) (first step S1). In other words, if one block of sentences consists of five sentences (sentences 1 to 5), the first 10 sentences (sentences 1 to 5) are extracted. 10 ) is divided into two parts (B1-1: (Sentence 1 to Sentence 5), B2-1: (Sentence 6 to Sentence 10)). Then, as shown in FIG. 5, whether or not the two extracted blocks B1-1 and B2-1 are similar to each other is determined by the distributed representation unit 11 using vector representation (second step S2), and the similarity determination unit 12 determines this distributed representation from the cosine similarity and outputs it as a numerical value between 0 and 1 (third step S3). This similarity data is stored in the memory unit 9 Next, the segmentation unit 10 deletes the first sentence (sentence 1) of the sentences targeted for block segmentation, and extracts two blocks of five sentences from the second sentence (sentence 2) (B1-2: (sentences 2 to 6) in FIG. 3, B2-2: (sentences 7 to 8) in FIG. 3). 11 )). And then, these two blocks, B1-2: (sentences 2 to 6), B2-2: (sentences 7 to 8), 11 ) the two blocks B1-2 and B2-2 are similar to each other, the distributed representation unit 11 and the similarity determination unit 12 determine whether they are similar to each other from the cosine similarity of the distributed representations, and output a numerical value between 0 and 1. That is, for the blocks B1-2 and B2-2 newly extracted from the second sentence (sentence 2), the second step S2 and the third step S3 are repeated to process the data, and the data is recorded in the storage unit 9. The extraction of the blocks (B1-1:B2-1)...(B1-n:B2-n (n is an arbitrary integer)) is performed from the last sentence (sentence 1). N ) can be continued until it reaches

[0039] In this way, the target sentences (sentence 1 to sentence 2) N The last sentence of N ) is reached, As shown in FIG. 5, the topic segmentation unit 13 graphs the output similarity values on the vertical axis and the positions between blocks on the horizontal axis. The closer the cosine similarity is to 0, the less similar the blocks are, and the closer it is to 1, the more similar the blocks are. Therefore, the valleys in the graph can be estimated as the locations where topics change. Therefore, as shown in FIG. 7, if the entire text T is segmented according to the positions of the valleys in the graph, the topics within the segmented sentences will be united. In other words, the sentences will be grouped under the same topic. In this way, the topic segmentation unit 13 considers sentences determined to have a high similarity to be the same topic, and sentences determined to have a low similarity to be different topics, and estimates the locations where topics change and segments the text by topic (fourth step S4). FIG. 8 shows a text similarity curve for the text T (the story of Momotaro). In the figure, the vertical lines indicate the topic division positions, the left axis represents similarity, and the right axis defines the depth of the valley using depth-scores. In other words, the depth is defined as how deep it is in relation to the left and right peaks, and the deeper it is, the more of a peak where the topic switches (the lower the left and right values, the more of a peak it is), i.e., it indicates a topic switch. The threshold α is set after the text of the main text T (the story of Momotaro) is graphed, and the point where this threshold α is exceeded and the depth-scores (right axis) peaks is the switch point. However, this must be ensured that there is no division point nearby. Figure 9 shows an example of text T (the story of Momotaro) divided into topics in this way.

[0040] Next, for the sentences divided into topics, the extraction-type summarization unit 14 sequentially inputs the topic-divided sentences into an extraction-type summarization model that is a fine-tuned version of a pre-trained model, as shown in Figures 2 and 10, combines the output summarized sentences, and finally creates a summarized sentence for the main text (fifth step S5). Figure 11 is an explanatory diagram showing an example of a numerical value output from the model for a topic-divided sentence. Note that the higher this numerical value (see the right side of Figure 11), the higher the probability that the sentence will be a candidate for a summary.

[0041] Next, a text summarization method according to the present invention will be described based on the operation of the text summarization system 2 according to the above embodiment. As shown in Figures 1 and 2, the text summarization system 2 according to the above embodiment is configured to include, as software (programs), an extraction unit 10, a distributed representation unit 11, a similarity determination unit 12, a topic segmentation unit 13, and an extraction-type summarization unit 14. First, in a first step S1, the extraction unit 10 extracts five sentences from the text T, starting with the first sentence (sentence 1), and defines each block as one block. Two successive blocks, B1-1 and B2-1, are extracted from the first block. Next, in a second step S2, the distributed representation unit 11 performs a distributed representation using vector representations to determine whether the extracted two blocks, B1-1 and B2-1, are similar to each other. In a third step S3, the similarity determination unit 12 determines the distributed representations based on the cosine similarity and outputs a value between 0 and 1. In this case, as shown in Figure 5, the closer to 0 the lower the similarity, and the closer to 1 the higher the similarity. After being graphed, depth-scores The threshold value α is set. The sentences in the two blocks B1-1 and B2-1 (sentences 1 to 5) and (sentences 6 to 7) are 10 ), the similarity is calculated and recorded. Next, the first sentence (sentence 1) is deleted by the extraction unit 10, and two blocks of five sentences are extracted from the second sentence (sentence 2) (B1-2: (sentences 2 to 6) in FIG. 3, B2-2: (sentences 7 to 8) in FIG. 3). 11 )), and the two blocks B1-2: (sentences 2 to 6), B2-2: (sentences 7 to 8) are shifted by one sentence. 11 ) and calculate and record the similarity between these two blocks B1-2 and B2-2 by repeating the second and third steps S2 and S3. In this way, the sentences are shifted one by one to extract the two blocks B1 and B2 in order and calculate the similarity. The extraction of the two blocks is performed starting from the last sentence (sentence N ) can be continued until it reaches

[0042] Target sentences (sentence 1 to sentence N The last sentence of N), in the fourth step S4, the topic division unit 13 graphs the output similarity values on the vertical axis and the positions between blocks on the horizontal axis (see FIG. 5), estimates that the valleys in the graph are positions where topics switch, and divides the entire text T according to the positions of the valleys in the graph (see FIG. 7). In this way, the text T is divided into topics (see FIG. 9).

[0043] Once the sentences in the main text T have been divided into topics, in the fifth step, the extraction-type summary unit 14 sequentially inputs the topic-divided sentences into an extraction-type summary model that is a fine-tuned version of a pre-trained model, combines the output summarized sentences, and finally creates summarized sentences of the main text (see Figures 2 and 10).

[0044] In this way, in the text summarization system and method according to the above embodiment, by dividing the text T into topics and summarizing each topic, it is possible to accurately, clearly, and simply summarize the entire text.

[0045] Although the text summarization system and method described above are referred to as a system, the present invention is not limited to this. It is also possible to use the software (program) (the segmentation unit 10, the distributed representation unit 11, the similarity determination unit 12, the topic segmentation unit 13, and the extractive summarization unit 14) installed on a PC 3, a terminal, or hardware (a device equipped with a CPU, an input / output unit, a display unit, and a storage unit) as a text summarization device. Furthermore, the present invention may be implemented as a program that causes a computer to segment a text containing multiple sentences into two consecutive blocks, each consisting of a predetermined number of blocks starting from the beginning, and sequentially segmenting the blocks by shifting each block by one sentence, vectorizing the sentences in each segment and determining their similarity, estimating topic changes based on the determined similarity, and summarizing the text for each segmented topic. In the above embodiment, blocks of five sentences are segmented starting from the first sentence (sentence 1), but this is not limiting. The number of sentences to be segmented may be appropriately set depending on the content and type of the text T. Furthermore, while the text summarization system and method according to the above embodiment are ultimately designed to summarize the text T, the system is not limited to this and can also be used as a language processing program solely for the purpose of dividing text into topics. In other words, since the text, such as minutes of a meeting, can be divided into topics, users can extract and use only the topics they need from the text without summarizing them, or they can summarize and use only those topics. [Explanation of symbols]

[0046] 2. Text summarization system 10 Cutout 11 Similarity determination unit 13 Topic division 14 Extract type summary section B1, B2, B1-1, B2-1, B1-2, B2-2 Blocks T Main text Sentence 1 ~ Sentence N sentence

Claims

1. an extracting unit that extracts a body of text including a plurality of sentences into two consecutive blocks each having a predetermined number of sentences from the first sentence, and sequentially extracts the blocks by shifting them by one sentence until it reaches the last sentence; a similarity determination unit that determines the similarity between the blocks included in each of the two blocks from a vector representation obtained by vectorizing the sentences in each of the two blocks that are sequentially extracted; a topic division unit that divides the topics of the text based on the relationship between the positions and similarities between the blocks; an extractive summarization unit that summarizes the text for each divided topic; The topic division unit A parameter representing the depth of the similarity at each inter-block position relative to the previous and next similarity peaks is calculated. After determining the parameters, a threshold is automatically set; A position where the parameter exceeds the threshold and reaches a peak, and where there is no dividing position nearby, is estimated to be a topic switching position in the main text, and the topic of the main text is divided based on the estimated switching position; The extraction-type summarization unit sequentially inputs the topic-divided sentences into an extraction-type summarization model that is a fine-tuned version of a pre-trained model, and combines the output summarized sentences to create summarized sentences of the main text.

2. A computer, A step of extracting a body of text including a plurality of sentences into two consecutive blocks each consisting of a predetermined number of sentences from the first sentence, and sequentially extracting the blocks by shifting them by one sentence until the last sentence is reached; a step of determining the similarity between the blocks included in each of the two blocks sequentially extracted from the vector representation obtained by vectorizing the sentences in each of the two blocks; A step of dividing the topic of the text based on the relationship between the position and similarity between each block; a step of summarizing the text for each divided topic; In the step of dividing the topic of the text, A parameter representing the depth of the similarity at each inter-block position relative to the previous and next similarity peaks is calculated. After determining the parameters, the threshold is automatically set; A position where the parameter exceeds the threshold and reaches a peak, and where there is no dividing position nearby, is estimated to be a topic switching position in the main text, and the topic of the main text is divided based on the estimated switching position; A method for summarizing text, characterized in that topic-divided text is input in order into an extraction-type summarization model that is fine-tuned from a pre-trained model, and the output summarized text is combined to create a summarized text of the main text.

3. On the computer, A text including a plurality of sentences is cut out into two consecutive blocks each having a predetermined number of sentences from the first sentence, and the blocks are shifted by one sentence each to cut out sequentially until the last sentence is reached; determining the similarity between the blocks contained in each of the two blocks from a vector representation obtained by vectorizing the sentences in each of the blocks extracted sequentially; Dividing the text into topics based on the relationship between the positions and similarities between the blocks; Summarizing the text for each divided topic, In dividing the topics in the text, A parameter representing the depth of the similarity at each inter-block position relative to the previous and next similarity peaks is calculated. After determining the parameters, the threshold is automatically set; A position where the parameter exceeds the threshold and reaches a peak, and where there is no dividing position nearby, is estimated to be a topic switching position in the main text, and the topic of the main text is divided based on the estimated switching position; A program that inputs topic-divided sentences in order into an extraction-type summary model that is a fine-tuned version of a pre-trained model, combines the output summarized sentences, and creates summarized sentences of the main text.

Images