Summarization device, summarization method, and summarization program

Abstract

To provide a summarization device, summarization method, and summarization program that can appropriately summarize documents of any size. [Solution] The summarization device 1 comprises: a division unit 11 that divides an input document into units of sentences or groups of sentences; a labeling unit 12 that extracts named entities contained in each of the divided sentences or groups of sentences and assigns labels to the extracted named entities; a score calculation unit 13 that calculates a score indicating the importance of each sentence or group of sentences by predetermined calculations based on predefined weights for the labels; and an output unit 14 that outputs a predetermined number of sentences or groups of sentences with the highest scores as a summary of the document.

Description

[Technical Field]

[0001] This invention relates to a technique for summarizing documents. [Background technology]

[0002] Conventionally, there are many large language models (LLMs) based on Transformers (see, for example, Non-Patent Document 1). For document summarization tasks, several models have been proposed, including BERTSUMExt (Non-Patent Document 2), BART (Non-Patent Document 3), T5 (Non-Patent Document 4), and Pegasus (Non-Patent Document 5). [Prior art documents] [Non-patent literature]

[0003] [Non-Patent Document 1] A. Vaswani et al., "Attention is all you need", Advances in Neural Information Processing Systems, MIT Press, 2017, pp. 5998-6008. [Non-Patent Document 2] Y. Liu and M. Lapata, "Text summarization with pretrained encoders", EMNLP-IJCNLP 2019 - 2019 Conference on Empirical Methods in Natural Language Processing and 9th International Joint Conference on Natural Language Processing Proceedings of the Conference, 2019, pp. 3730-3740. [Non-Patent Document 3] M. Lewis et al., "BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension", arXiv preprint arXiv:1910.13461 (2019). [Non-Patent Document 4] C. Raffel et al., "Exploring the limits of transfer learning with a unified text-to-text transformer", Journal of Machine Learning Research 21, 1 (2020), 5485-5551. [Non-Patent Document 5] J. Zhang et al., "PEGASUS: Pre-training with extracted gap-sentences for abstractive summarization", Proceedings of the 37th International Conference on Machine Learning (ICML '20), PMLR, 11328-11339. [Non-Patent Document 6] X. Wang et al., "Dnrti: A largescale dataset for named entity recognition in threat intelligence", 2020 IEEE 19th International Conference on Trust, Security, and Privacy in Computing and Communications (TrustCom), pp. 1842-1848. [Non-Patent Document 7] X. Wang et al., "Aptner: A specific dataset for ner missions in cyber threat intelligence field", 2022 IEEE 25th International Conference on Computer Supported Cooperative Work in Design (CSCWD), pp. 1233-1238. [Non-Patent Document 8] I. Mouiche and S. Saad, "TI-NERmerger: Semi-Automated Framework for Integrating NER Datasets in Cybersecurity", 2024, 21th International Conference on Security and Cryptography (SECRYPT). [Non-Patent Document 9] "Summarize Text", 2024, LangChain, Inc.,<https: / / python.langchain.com / docs / tutorials / summarization / > [Overview of the Initiative] [Problems that the invention aims to solve]

[0004] However, many models are pre-trained using general-purpose data, and fine-tuning using training data (pairs of original text and summarized text) is necessary to appropriately summarize documents in a specific field. Furthermore, the points to focus on in a document can change depending on the purpose, but it was difficult to prepare corresponding training data each time. Furthermore, because LLMs typically have an upper limit on the input token size, summarizing extremely large documents has been difficult. For example, methods for handling documents with large token sizes, such as Map-Reduce in Non-Patent Document 9, have been proposed, but they have problems such as the content becoming too abstract and the same content being repeated.

[0005] An object of the present invention is to provide a summarization device, a summarization method, and a summarization program that can appropriately summarize documents of any size.

Means for Solving the Problems

[0006] The summarization device according to the present invention includes a splitting unit that splits an input document into units of sentences or groups of sentences, and for each of the split sentences or groups of sentences, extracts the included proper expressions and assigns labels to the extracted proper expressions; a label assignment unit; a score calculation unit that calculates a score indicating the importance of each of the sentences or groups of sentences by a predetermined operation based on weights defined in advance for the labels; and an output unit that outputs, as a summary sentence of the document, the sentences or groups of sentences with the top predetermined number of scores.

[0007] The score calculation unit may calculate an information score based on the sum of the weights for each label assigned to the sentence or group of sentences as the score.

[0008] The score calculation unit may calculate a predetermined information score for a specific set of labels included in the sentence or group of sentences as the score.

[0009] The score calculation unit may calculate a propagation score according to the information score and the distance as a propagation effect of the information score calculated for neighboring sentences or groups of sentences as the score.

[0010] The splitting unit may further split the document into larger semantic groups than the sentences or groups of sentences, and the score calculation unit may calculate the propagation score within the range of the group.

[0011] The score calculation unit may set a dummy information score used only for calculating the propagation score outside the range of the group by statistically processing the information scores within the range of the group.

[0012] The output unit may take a predetermined number of sentences or groups of sentences with the highest scores as input to a predetermined language model and output a summary sentence obtained from the language model.

[0013] The summarization method according to the present invention involves a computer dividing an input document into units of sentences or groups of sentences using a division unit, extracting named entities from each of the divided sentences or groups of sentences using a labeling unit, assigning labels to the extracted named entities, calculating a score indicating the importance of each of the sentences or groups of sentences using a predetermined calculation based on predefined weights for the labels using a score calculation unit, and outputting a predetermined number of the sentences or groups of sentences with the highest scores as a summary of the document using an output unit.

[0014] The summarization program according to the present invention is for causing a computer to function as the summarization device. [Effects of the Invention]

[0015] According to the present invention, documents of any size can be appropriately summarized. [Brief explanation of the drawing]

[0016] [Figure 1] This is a block diagram showing the functional configuration of the summarization device in an embodiment. [Figure 2] This flowchart shows the processing flow by the summarization device in the embodiment. [Figure 3] This is a schematic diagram illustrating the procedure of the summary method in the embodiment. [Modes for carrying out the invention]

[0017] An example of an embodiment of the present invention will be described below. The summarization method of this embodiment is an extraction-type method that weights named entities of interest to the user. Based on named entity labels, it outputs a document summary by calculating scores for sentences or groups of sentences such as paragraphs using the procedure described later. While the types of documents that can be targeted are not limited and the application can be made to any field, this embodiment will describe the case where a document related to cyber threat information is input as an example.

[0018] Figure 1 is a block diagram showing the functional configuration of the summarization device 1 in this embodiment. The summarization device 1 is an information processing device equipped with a control unit 10 and a storage unit 20, as well as various input / output interfaces.

[0019] The control unit 10 is the part that controls the entire summarization device 1, and realizes each function in this embodiment by appropriately reading and executing various programs stored in the storage unit 20. The control unit 10 may be a CPU. The memory unit 20 is a storage area for various programs and data that enable the hardware group to function as a summarization device 1, and may be ROM, RAM, flash memory, or a hard disk drive (HDD).

[0020] The control unit 10 functions as a division unit 11, a labeling unit 12, a score calculation unit 13, and an output unit 14 by executing the software (summary program) stored in the storage unit 20.

[0021] The splitting unit 11 divides the input document into units of sentences or units of sentence groups such as paragraphs.

[0022] The labeling unit 12 extracts named entities from each of the divided sentences or groups of sentences and assigns labels to the extracted named entities. The method of label assignment is not limited. For example, in Non-Patent Documents 6 to 8, the construction method and training data of an entity expression extraction model that identifies entity expressions and classifies them into categories are disclosed, and entity expression labels can be automatically assigned according to the STIX format.

[0023] The score calculation unit 13 calculates a score indicating the importance of each sentence or sentence group by a predetermined operation based on the weights defined in advance for the labels. Define an information score for weighting the sentence itself and a propagation score propagated from the weights of neighboring sentences as the scores.

[0024] The information score (Information Score) is calculated based on the predefined weight information (Defined Score). For example, the information score may be based on the sum of the weights for each label assigned to the sentence or sentence group. Let the weights for the labels of Defined Score be r1,…,r i , i , i ,

[0025] , k , k , , be w1,…w respectively. Assuming that the number of words (or tokens) is T m and there is a sentence S i with the number of words (or tokens) being T i (i = 1,…,n), when focusing on S i and there are R i 1,…,R i l words respectively labeled with r i 1,…,r i l i i The information score of S i can be expressed as IS k=1 l =(Σ i k w i k ·R i ) / T

[0025] Specifically, for example, to focus on what attack method the attacker used, let's say we define weights of 3 for the attacker label and 5 for the attack method label. i When we focus on S, i Suppose that for each word, one attacker label and two attack method labels are assigned. In this case, T i If we set =20, then S i The information score is expressed as (3×1+5×2) / 20=0.65.

[0026] Furthermore, the method for calculating the information score is not limited to this. For example, the IS may be calculated regardless of the number of words. i =Σ k=1 l w i k ·R i k Alternatively, by utilizing relational extraction techniques, a predetermined information score may be assigned to expressions considered more important, such as specific pairs of labels that have a subject-predicate relationship.

[0027] The propagation score is calculated by spreading the information score of each sentence or group of sentences to the surrounding sentences or groups of sentences. For example, the information score of the p-th sentence is IS p Therefore, the propagation score of the p±q (q≠0)th sentence is: PS p±q =w·IS p log 1 / R This can be expressed as (q / R) (where w is a real number and R is a natural number that influences the sentence). The function used to calculate the propagation score is not limited to this; any function that gradually decreases as the distance q increases is acceptable. Furthermore, the propagation effect may only occur on one side, such as at the pq (q≠0)th propagation event.

[0028] Furthermore, if the splitting unit 11 identifies and splits the input document into larger semantic groups than sentences or groups of sentences (for example, paragraphs for sentences, chapters for paragraphs, etc.), the score calculation unit 13 may calculate the propagation score within the scope of this group. That is, when calculating the score for each sentence, if there is a change in paragraph, the subsequent (or previous) propagation scores are treated as 0.

[0029] Furthermore, since the propagation scores of the first and last sentences of the group become small in this case, the score calculation unit 13 may set a dummy information score outside the range of the group, which is used solely for calculating the propagation score, by statistically processing the information score within the range of this group. Specifically, for example, IS0, IS n+1 or, further IS -1 IS n+2 These should be prepared as dummy data. Here, the dummy information scores can be set as appropriate, for example, the average, minimum, and maximum values ​​of all information scores within the group.

[0030] The output unit 14 outputs a summary of the input document, consisting of a predetermined number (k) of sentences or groups of sentences whose combined information score and propagation score result in the highest score.

[0031] Furthermore, the summarization results can also be reused. The output unit 14 may extract a predetermined number of important sentences or groups of sentences with high scores from a massive document, then apply another language model, such as a generative AI, and output a summary sentence obtained from this language model. In this case, by increasing the weight of the propagation score, particularly by increasing the parameter w in the propagation score, and by increasing k, it becomes possible to extract consecutive sentence clusters. This allows for the removal of unnecessary information from large documents and the application of other summarization models, resulting in an effective summary.

[0032] Figure 2 is a flowchart showing the processing flow by the summarization device 1 in this embodiment. In step S1, the splitting unit 11 divides the input document into sentence units. In step S2, the labeling unit 12 assigns a named entity label to each of the words and other elements contained in the divided sentence.

[0033] In step S3, the score calculation unit 13 calculates the information score and propagation score for each sentence based on the weights (Defined Score) and score function (Score Function) corresponding to the predefined labels, and then calculates the total score. In step S4, the output unit 14 outputs the top k sentences with the highest calculated scores as a summary.

[0034] Figure 3 is a schematic diagram illustrating the procedure of the summarization method in this embodiment. The input document is divided into multiple sentences (Sentence 1, 2, …, n). Each sentence is then assigned a named entity label, such as threat-actor or attack-pattern, by applying a Named Entity Recognition (NER) model trained on a predetermined training dataset.

[0035] If the user pre-defines weights (Defined Score) for each label, such as Attacker: 3, Attack Method: 5, the summarization device 1 calculates the score for each sentence according to a predetermined function (Score Function). In this example, the number of sentences to output as a summary is set to k=3, so the three sentences with the highest scores, Sentences 2, 5, and 8, are output.

[0036] According to this embodiment, the summarization device 1 extracts important sentences or groups of sentences by assigning scores based on named entity labels to sentences or groups of sentences, and uses them to create a summary. If a method for assigning named entity labels is established, training data is not required, and the summarization device 1 does not need to incorporate the entire document into its learning model, so it can handle documents of any size and summarize them appropriately.

[0037] Furthermore, since sentence importance is determined solely by labels, it is possible to avoid the positional bias problem (bias on information at the beginning and end of sentences) that occurs in generative summarization models. Furthermore, the summarization device 1 allows for easy adjustment of what aspects of the summary (sentence extraction) are focused on by designing weights and a score function for named entity labels.

[0038] The summarization device 1 can preferentially extract sections of a document that contain named entities of interest to the user by calculating an information score based on the sum of the weights assigned to each label for each sentence or group of sentences. Furthermore, by using relational extraction technology, the summarization device 1 can extract texts that better suit the user's needs by calculating a predetermined information score for specific sets of labels.

[0039] Furthermore, based on the assumption that sentences adjacent to important sentences are also important, the summarization device 1 calculates a ripple effect of information scores, including the information score and a ripple score corresponding to the distance, thereby obtaining a more reliable score and enabling the output of an appropriate summary.

[0040] In this case, if the summarization device 1 can divide the document into semantic groups larger than sentences or groups of sentences, it can calculate a more reliable score by calculating the propagation score within the scope of these groups, without adding unnecessary propagation scores at points where the content changes, such as paragraph breaks. Furthermore, by setting dummy information scores outside the scope of such groups, used solely for calculating propagation scores, by statistically processing the information scores within the group scope, the propagation scores attached to the first and last sentences in a document or group can be appropriately adjusted.

[0041] The summary text obtained by this embodiment retains the important sentences in the document while reducing the number of sentences, making it possible to reuse it as input for various language models with limited input token size, such as generative AI. For example, summarization device 1 can generate a more appropriate summary with a reduced character count by inputting the extracted important sentences into another summarization model.

[0042] Furthermore, this embodiment makes it possible to create summaries from any document, for example, and thus contribute to Goal 9 of the United Nations-led Sustainable Development Goals (SDGs), "Build resilient infrastructure, promote sustainable industrialization and foster innovation."

[0043] Although embodiments of the present invention have been described above, the present invention is not limited to the embodiments described above. Furthermore, the effects described in the embodiments described above are merely a list of the most preferred effects resulting from the present invention, and the effects of the present invention are not limited to those described in the embodiments.

[0044] The summarization method by summarization device 1 is implemented by software. When implemented by software, the programs constituting this software are installed on an information processing device (computer). These programs may be distributed to users by being recorded on removable media such as a CD-ROM, or by being downloaded to the user's computer via a network. Furthermore, these programs may be provided to the user's computer as a web service via a network without being downloaded. [Explanation of Symbols]

[0045] 1. Summarization device 10 Control Unit 11 Division 12 Labeling section 13. Score Calculation Section 14 Output section 20 Memory section

Claims

1. A division unit that divides the input document into units of sentences or groups of sentences, A labeling unit that extracts named entities from each of the divided sentences or groups of sentences and assigns labels to the extracted named entities, A score calculation unit calculates a score indicating the importance of each sentence or group of sentences by performing a predetermined calculation based on predefined weights for the aforementioned labels. A summarization device comprising: an output unit that outputs a predetermined number of sentences or groups of sentences with the highest scores as a summary of the document.

2. The summarizing device according to claim 1, wherein the score calculation unit calculates an information score based on the sum of the weights assigned to each label in the sentence or group of sentences as the score.

3. The summarizing device according to claim 1, wherein the score calculation unit calculates a predetermined information score for a specific set of labels included in the sentence or group of sentences as the score.

4. The summarizing device according to claim 2 or 3, wherein the score calculation unit calculates, as the score, the information score and a ripple score corresponding to the distance as the ripple effect of the information score calculated for neighboring sentences or groups of sentences.

5. The division section further divides the document into semantic groups larger than the sentences or groups of sentences, The summarizing device according to claim 4, wherein the score calculation unit calculates the ripple score within the range of the group.

6. The summarizing device according to claim 5, wherein the score calculation unit sets a dummy information score, used solely for calculating the propagation score, outside the range of the group, by statistically processing the information score within the range of the group.

7. The summarization device according to any one of claims 1 to 3, wherein the output unit takes a predetermined number of sentences or groups of sentences with the highest scores as input to a predetermined language model and outputs a summary sentence obtained from the language model.

8. Computers The splitting unit divides the input document into units of sentences or groups of sentences. The labeling unit extracts named entities from each of the divided sentences or groups of sentences, and assigns labels to the extracted named entities. The score calculation unit calculates a score indicating the importance of each sentence or group of sentences based on predefined weights for the labels, through a predetermined calculation. A summarization method in which an output unit outputs a predetermined number of sentences or groups of sentences with the highest scores as a summary of the document.

9. A summarization program for causing a computer to function as a summarization device according to any one of claims 1 to 3.

Images