Generation apparatus, generation method, and generation program

Abstract

Using a language model, generate event reports with a quality comparable to that of an expert. [Solution] The generation device 10 includes a past report perspective DB 121 that stores past report perspectives, which are perspectives that should be summarized from the original document when creating a report from scratch, extracted using LLM 31 from past reports generated about past events using LLM (Large language Models) 31 and revised reports in which modifications have been made to past reports; a new perspective generation unit 123 that generates new perspectives from documents related to the events to be reported using LLM 31 based on the past report perspectives; and a generation unit 13 that generates a report summarizing the events to be reported using LLM 31 based on the new perspectives generated by the new perspective generation unit 123.

Description

Technical Field

[0001] The present invention relates to a generation device, a generation method, and a generation program.

Background Art

[0002] In departments dealing with security such as CSIRT (Computer Security Incident Response Team), there are many cases where security reports summarizing public and non - public security incidents are issued to related organizations.

[0003] In the issuance of this security report, the application of a summarization generation technique using a large - language model (LLM: Large Language Models) has been under consideration. There are various conventional techniques for generating summaries of security information using LLM.

[0004] For example, as a technique for generating summaries of security information, there is one that gives information serving as a summary framework to the LLM to aim for a summary of security information (Non - Patent Document 1). In the technique described in Non - Patent Document 1, incident cases are structured using six major items [pre - condition / attack / detection / damage / investigation / countermeasure] and ten minor items [who / whose / to whom / what was done / when / where / why / how / how much / example of attack].

Prior Art Documents

Non - Patent Documents

[0005]

Non - Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0006] When creating a report summarizing security incidents, experienced individuals can independently consider and create the report, determining what perspectives are necessary to present the incidents.

[0007] Report writing ability is highly subjective, and the first challenge is that a novice in report writing cannot create a report that effectively inherits that know-how simply by reviewing past reports and comments. Therefore, attempts are being made to provide LLMs with accumulated information such as past report examples and revision comments, and have the LLMs create the reports. However, simply providing LLMs with accumulated information is not enough, as they cannot effectively handle that information, making it difficult to solve the first challenge.

[0008] Furthermore, when reporting security incidents, there are many cases where it's desirable to convey information using shorter text spaces, such as slides, rather than just long sentences. Examples include using slides to present information or adding short explanations to links.

[0009] While many attempts have been made to generate summaries using LLM, summaries generated with LLM tend to prioritize accuracy, comprehensiveness, and naturalness of writing, rather than focusing on summarizing key points clearly in short sentences.

[0010] Furthermore, in the field of security and other areas, there have been various attempts to generate summaries using LLMs when creating concise reports based on publicly available and private multilingual natural text sources. However, conventional technologies have a second challenge: they struggle to handle complex events with multiple time series.

[0011] For example, the method described in Non-Patent Document 1 is limited to structuring that uniformly addresses incidents, and therefore cannot handle vulnerability explanations other than incidents or complex events with multiple timelines.

[0012] The present invention has been made in view of the above, and aims to provide a generation apparatus, generation method, and generation program that can generate event reports using LLM with a quality comparable to that of a skilled professional. [Means for solving the problem]

[0013] To solve the above-mentioned problems and achieve the objective, the generation device according to the present invention is characterized by having: a storage unit that stores at least past report perspectives, which are perspectives to be summarized from the original document when creating the report from scratch, extracted using the language model from past reports generated about past events using the language model and revised reports in which modifications have been made to the past reports; a first generation unit that generates new perspectives from documents related to the events to be reported using the language model based on the past report perspectives; and a second generation unit that generates a report summarizing the events to be reported using the language model based on the new perspectives generated by the first generation unit.

[0014] Furthermore, the generation method according to the present invention is a generation method executed by a generation device, wherein the generation device has a storage unit that stores at least past report perspectives, which are perspectives to be summarized from the original document when creating the report from scratch, extracted using the language model from past reports generated for past events using the language model and revised reports to which modifications have been made to the past reports, and the generation method includes: a first generation step of newly generating perspectives from documents relating to the events to be reported using the language model based on the past report perspectives; and a second generation step of generating a report summarizing the events to be reported using the language model based on the new perspectives generated in the first generation step.

[0015] In addition, the generation program according to the present invention causes a computer to execute: a first generation step of newly generating, using the language model, a past report viewpoint, which is a viewpoint for summarizing from the original document when creating the report from scratch, from a past report generated using the language model about past events and a corrected report in which corrections have been made to the past report; and a second generation step of generating a report summarizing the event to be reported using the language model based on the new viewpoint generated in the first generation step.

Advantages of the Invention

[0016] According to the present invention, a report regarding an event can be generated with a quality comparable to that of an expert using a language model.

Brief Description of the Drawings

[0017] [Figure 1] FIG. 1 is a diagram for explaining the outline of an embodiment. [Figure 2] FIG. 2 is a diagram showing an example of the configuration of a generation device according to an embodiment. [Figure 3] FIG. 3 is a diagram for explaining the processing of a viewpoint extraction unit. [Figure 4] FIG. 4 is a diagram showing an example of a past report generated by an LLM. [Figure 5] FIG. 5 is a diagram showing an example of a corrected report. [Figure 6] FIG. 6 is a diagram showing an example of a correction comment. [Figure 7] FIG. 7 is a diagram showing an example of a prompt set for an LLM. [Figure 8] FIG. 8 is a diagram showing an example of a past report viewpoint. [Figure 9] FIG. 9 is a diagram for explaining other processing of the viewpoint extraction unit. [Figure 10] FIG. 10 is a diagram showing an example of a prompt set for an LLM. [Figure 11] FIG. 11 is a diagram showing an example of a prompt to be set in the LLM. [Figure 12] FIG. 12 is a diagram showing an example of a detailed summary. [Figure 13] FIG. 13 is a diagram showing an example of a report output by the report generation unit. [Figure 14] FIG. 14 is a flowchart showing the processing procedure of the generation process according to the embodiment. [Figure 15] FIG. 15 is a flowchart showing the processing procedure of the preprocessing shown in FIG. 14. [Figure 16] FIG. 16 is a flowchart showing the processing procedure of the viewpoint generation process shown in FIG. 14. [Figure 17] FIG. 17 is a flowchart showing the processing procedure of the report generation process shown in FIG. 14. [Figure 18] FIG. 18 is a diagram showing an example of a computer by which a generation device is realized when a program is executed.

Embodiments of the Invention

[0018] <0000​​​​​​​​​​​​In this embodiment, historical report perspectives are extracted from historical reports generated using LLM and revised reports that have been modified from historical reports. The extracted historical report perspectives are then associated with the revised reports and stored in the historical report perspective DB121 (described later). Perspectives are those that should be compiled from the original document when creating a report from scratch. Historical report perspectives are perspectives extracted using LLM from at least historical reports and revised reports.

[0021] The generation device searches for past security events similar to the article of the new security event being reported by calculating the similarity with data stored in the past report perspective DB121 (Figure 1 (1)).

[0022] The generator extracts past report perspectives V21 and revised reports corresponding to the searched past events as key perspectives of past articles (Figure 1 (2)). The generator provides past report perspectives V21 and revised reports to the LLM as example sentences (few-shot prompting) (Figure 1 (3)). Then, the generator instructs the LLM to generate a new perspective from the article D11 of the new security event (Figure 1 (4)), which becomes key perspective V11 (new perspective).

[0023] The generation device generates a detailed summary R11 (first summary) that fully reflects the content of the key points V11 (Figure 1 (5)). From the detailed summary R11, the generation device generates and outputs a final report R21 (Figure 1 (6)) formatted into the specified report format.

[0024] In this way, the generation device refers to useful past report perspectives when creating summaries and causes LLM to create new perspectives. Using these new perspectives, the generation device creates a summary in LLM from the information necessary for drafting the report, making it possible to generate accurate summaries that reflect the perspectives even for complex events.

[0025] [Generation device] Next, a description of the generation apparatus according to the embodiment will be given. Figure 2 is a diagram showing an example of the configuration of the generation apparatus according to the embodiment.

[0026] The generation device 10 according to this embodiment communicates with an analyst input device 20, which is a terminal device operated by an analyst (for example, a skilled person proficient in report creation), and a server 30 that controls the input and output of data to and from the LLM 31. The LLM 31 is a natural language processing model that processes input data (for example, text information) in natural language according to prompts set in the LLM 31, and generates and outputs data such as viewpoints, summaries, and reports. The LLM 31 may be provided as an external device to the generation device 10, or it may be provided inside the generation device 10.

[0027] The generation device 10 according to this embodiment is a generation device that is realized by loading a predetermined program into a computer including, for example, ROM (Read Only Memory), RAM (Random Access Memory), CPU (Central Processing Unit), etc., and having the CPU execute the predetermined program. The generation device 10 also has a communication interface for sending and receiving various information with other devices connected via a network, etc. (for example, an analyst input device 20, a server 30).

[0028] The generation device 10 includes a pre-processing unit 11, a viewpoint generation unit 12 (first generation unit), and a generation unit 13 (second generation unit).

[0029] The preprocessing unit 11 performs preprocessing to extract and format the information necessary for creating a report draft by LLM 31 from the information input by the analyst via the analyst input device 20. This information can be reviewed by the analyst, for example, and edited or unnecessary information can be omitted.

[0030] The perspective generation unit 12 generates new perspectives from documents related to the event to be reported, using LLM31 based on past report perspectives. For the event for which a new report is to be created, the perspective generation unit 12 creates new "perspectives," which are the points to summarize, using past report perspectives stored in the past report perspective DB121 as examples.

[0031] The generation unit 13 generates a report summarizing the events to be reported, using the LLM 31 based on the new perspectives (new viewpoints) generated by the perspective generation unit 12.

[0032] [Preprocessing step] Next, the preprocessing unit 11 will be described. The preprocessing unit 11 includes an information source acquisition unit 111, an information extraction unit 112, and an information verification unit 113.

[0033] The information source acquisition unit 111 acquires information sources to be used for report generation. The information source acquisition unit 111 acquires documents (articles), past reports, revised reports, and / or revision comments related to various security events, including the report generation target. Past reports are reports generated using LLM31 about past security events. Revised reports are reports to which experts have made revisions to past reports generated by LLM31. Experts are people who are skilled in creating reports on security events. Revision comments are comments made by experts (reasons for revision, content of revision, etc.) regarding the revised parts of past reports.

[0034] The information extraction unit 112 extracts text information and / or important image information from the input security information (documents related to the events to be reported) and converts the input information into information that can be interpreted by the LLM 31. The information extraction unit 112 outputs the text information extracted from the input security events.

[0035] The information extraction unit 112 takes security information in a specified format as input. Examples of the specified format include document files such as PDF (Portable Document Format) and HTML (Hyper Text Markup Language). However, it is not necessary to limit the input to these formats; any security information that contains natural language is acceptable. Furthermore, the input security information may be a single item or multiple items.

[0036] The information extraction unit 112 may, in addition to implementing an implementation that extracts only text, extract information by converting images to text using an LLM model such as GPT-4o that can extract interpretable information from images. Possible input examples include the results of searches conducted in databases using specific keywords or vulnerability identifiers.

[0037] The information verification unit 113 presents the information extracted by the information extraction unit 112 to the analyst, accepts the corrected and verified text information from the analyst, and outputs it to the perspective generation unit 12. The information verification unit 113 takes the text information extracted from the input security event as input and outputs the verified text information extracted from the input security event.

[0038] The information verification unit 113 presents the information extracted by the information extraction unit 112 to the analyst input device 20. The analyst can then verify and edit the presented information to confirm that the information extraction is functioning correctly. For example, if there is an error in the interpretation of text information generated from an image, the analyst can manually correct the information. The text information that the analyst has verified and corrected is then output to the viewpoint generation unit 12.

[0039] [Perspective Generation Unit] Next, the perspective generation unit 12 will be described. The perspective generation unit 12 includes a past report perspective DB 121, a perspective extraction unit 122, and a new perspective generation unit 123.

[0040] The past report perspective DB121 stores at least the past report perspectives. The past report perspective DB121 stores each perspective by associating its identifier with the content it should summarize. A perspective is a key:value pair consisting of the perspective's name and the content it should summarize (description).

[0041] Specific examples of perspectives include: "Disclosure of Information Source": "Based on information regarding a vulnerability in product A announced by security company T on 2024 / xx / xx," and "Vulnerability Details": "The name of the vulnerability (XXX), the CVE number (CVE-XXX-XXXXX), and the type of vulnerability (heap-based buffer overflow) are clearly stated." The past report perspective DB121 stores past report perspectives and the corresponding correction reports. The past report perspective DB121 may also store past report perspectives, the corresponding correction reports, and correction comments.

[0042] The perspective extraction unit 122 uses LLM 31 to extract past report perspectives from past reports, revised reports, and / or revised comments. The perspective extraction unit 122 stores the extracted past report perspectives in the past report perspective DB 121, linked to the corresponding revised reports. In addition to past report perspectives and revised reports, the perspective extraction unit 122 may also store the corresponding revised comments in the past report perspective DB 121.

[0043] Figure 3 is a diagram illustrating the processing of the viewpoint extraction unit 122. Figure 4 is a diagram showing an example of a past report generated by LLM31. Figure 5 is a diagram showing an example of a revised report. Figure 6 is a diagram showing an example of a revised comment.

[0044] The perspective extraction unit 122, based on an article D21 about a certain security incident, causes LLM31 to extract the perspective V21 from the past report (Figure 3(3)) using the following data: LLM generated report D22 (a past report summarized and generated by LLM31) (Figure 3(1)) (Figure 4), revised report D23 (Figure 5) which is a revised version of LLM generated report D22 by expert U1, and expert revision comments D24 (Figure 3(2)) (Figure 6). Note that revision comments D24 are not necessarily required.

[0045] Figure 7 shows an example of a prompt to be set in LLM31. The perspective extraction unit 122 sets prompt P1 shown in Figure 7 in LLM31, for example, and inputs the LLM generation report D22 (original_document), the revision report D23 (reviewed_document), and / or the revision comment D24 (reviewer_comment) into LLM31.

[0046] This causes LLM31 to generate and output the past report perspective V21. Prompt P1 instructs you, as a security report expert, to describe the perspectives that should be summarized from the original document when creating a report equivalent to the revised security report from scratch, referring to the differences and comments (D24) of the report (D22, D23). Prompt P1 also instructs you on the structure of the perspectives to be output.

[0047] Figure 8 shows an example of past report perspectives. LLM31 extracts one or more key:value pairs of perspective names and the content to be summarized from the LLM generated report D22, revised report D23, and / or revised comment D24, for example, as shown in Figure 8. The perspective extraction unit 122 stores the past report perspectives V21 extracted by LLM31 in the past report perspective DB 121, linking them with the corresponding revised report D23 and / or revised comment D24 (Figure 3 (4)).

[0048] In addition, there are cases where, aside from situations where an expert modifies a report created by LLM31, only reports previously created by humans exist. Figure 9 illustrates other processes of the viewpoint extraction unit 122.

[0049] In this case, the perspective extraction unit 122 performs the process shown in Figure 9 to generate pseudo-data from which to extract past report perspectives. Specifically, let's explain the case where only a report summarized by expert U1 exists for article D21 of a security incident. The perspective extraction unit 122 treats the report newly created by LLM31 for article D21 of the same incident as LLM-generated report D22 (Figure 9 (2)). Then, the report created by expert U1 is designated as revised report D23' (Figure 9 (1)).

[0050] The perspective extraction unit 122 generates past report perspectives, just as if there were no correction comment D24. For example, the prompt P1 shown in Figure 7 is set to LLM31, and the LLM generated report D22 and the correction report D23' are input to LLM31. As a result, LLM31 extracts past report perspective V21' (Figure 9 (3)). The perspective extraction unit 122 links past report perspective V21' with correction report D23' and stores it in the past report perspective DB 121 (Figure 9 (4)).

[0051] Similarly, if there is a "human-generated report" and a "human-revised report" that has been reviewed and corrected, the "human-generated report" can be designated as the LLM-generated report, and the "human-revised report" as the revised report.

[0052] Furthermore, the viewpoint extraction unit 122 can further enrich the past report viewpoints by adding newly created reports (final reports) by the generation device 10, as well as viewpoints previously extracted by experts, etc., to the past report viewpoints DB 121. Note that if sufficient data has been accumulated in the past report viewpoints DB 121, the viewpoint extraction unit 122 may be deleted. Also, the viewpoint extraction unit 122 may be provided in a separate device.

[0053] The new perspective generation unit 123 generates new perspectives from documents (articles) related to security events for which reports are to be generated, using LLM31, based on past report perspectives stored in the past report perspective DB121. The new perspective generation unit 123 takes correction reports, past perspectives, and verified text information extracted from the input security events (output of the information verification unit 113) as input. The new perspective generation unit 123 outputs the newly generated perspectives (new perspectives) to the generation unit 13.

[0054] The new perspective generation unit 123 searches the past report perspective DB 121 for past security events similar to the security event for which a report is to be generated, and obtains the correction report and past report perspective corresponding to that past security event.

[0055] The new perspective generation unit 123 performs similarity determination by, for example, converting text data into a vector form and then calculating COS similarity. In this case, techniques that enable the vectorization of documents, such as Sentence BERT (Bidirectional Encoder Representations from Transformers), can be utilized.

[0056] For example, the new perspective generation unit 123 determines the similarity between the titles of past reports (revision reports, final reports), the titles of the articles that form the basis of the reports, and the titles of the articles concerning the new security events for which a report is to be generated. The new perspective generation unit 123 obtains revision reports and past report perspectives for security events that show a similarity above a predetermined threshold. The new perspective generation unit 123 may also determine the similarity between past reports and the articles concerning the new security events for which a report is to be generated, or it may determine the similarity between articles. The new perspective generation unit 123 may use any of the conventional methods as long as the similarity can be determined.

[0057] The new perspective generation unit 123 performs few-shot prompting by providing LLM31 with one or more past correction reports and past report perspectives obtained from the past report perspective DB 121. Then, the new perspective generation unit 123 uses LLM31 to generate a new perspective specific to the newly entered security event article (the security event article for which a report is to be generated). In addition to past report perspectives and correction reports, the new perspective generation unit 123 may also generate the new perspective using correction comments.

[0058] The newly generated perspectives are in key:value format, similar to the perspectives included in the past report perspective DB121, and their corresponding content includes the content extracted from the input.

[0059] Figure 10 shows an example of a prompt to be set in LLM31. The new perspective generation unit 123 sets, for example, the prompt P2 shown in Figure 10 to LLM31.

[0060] Prompt P2 instructs you, as a security report expert, to create detailed descriptions of the points (perspective names) and corresponding content (perspective content) necessary to clearly explain target_document (the security incident article to be reported). Prompt P1 instructs you to structure the perspectives to be output. Prompt P1 also instructs you to create the perspectives by referring to sample_document (correction report) and sample_output (past report perspectives). required_points is included as input during implementation to adjust the output, and should contain at least the content you want to refer to in LLM31.

[0061] [Generation part] Next, the generation unit 13 will be described. The generation unit 13 includes an intermediate summary generation unit 131 (summary generation unit) and a report generation unit 132. In the generation unit 13, instead of narrowing down the articles of the events to be reported to a short summary all at once, it first creates an intermediate detailed summary and then decides what to further narrow down within that summary.

[0062] The intermediate summary generation unit 131 generates a detailed summary (intermediate summary) that fully reflects the content of the new perspective (new viewpoint) generated by the new perspective generation unit 123, based on the verified text information extracted from the input security event. The intermediate summary generation unit 131 takes the new perspective and the verified text information extracted from the input security event as input and outputs the intermediate summary (detailed summary) to the report generation unit 132.

[0063] Figure 11 shows an example of a prompt to be set in LLM31. Figure 12 shows an example of a detailed summary. The prompt P3 shown in Figure 11 instructs the LLM31 to create a detailed summary that satisfies a given perspective (new perspective) for the verified text information extracted from the input security event (new security event article D11). As a result, LLM31 generates, for example, a detailed summary R11 (Figure 12) about the new security event article D11 and outputs it to the intermediate summary generation unit 131.

[0064] The report generation unit 132 generates a final report corresponding to the specified report format from the detailed summary generated by the intermediate summary generation unit 131, and outputs it to, for example, the analyst input device 20. Figure 13 shows an example of a report output by the report generation unit 132. The report generation unit 132 takes the intermediate summary (natural language) (detailed summary), the content to be narrowed down (natural language), and the report format (natural language) generated by the intermediate summary generation unit 131 as input, and outputs a report (natural language or slides) (for example, report R21 in Figure 13).

[0065] The format of the report is specified, for example, by the analyst. Therefore, the operation of the report generation unit 132 differs depending on the final report format specified by the analyst. In other words, the output of the report generation unit 132 differs depending on the final report format specified by the analyst. The report format may be specified as, for example, a single-slide style or a snippet consisting of a short document of about three lines. In the case of a slide, the report generation unit 132 converts the document generated by LLM31 into a slide format using a combination of LLM31 and a library such as python-pptx.

[0066] The generation unit 13 generates a detailed explanation as an intermediate summary, determines the content to be narrowed down from that detailed explanation, and then removes information accordingly to generate a consistent report. The content to be narrowed down can also be automatically extracted by LLM 31. Alternatively, an analyst may review the intermediate summary (detailed explanation) and concisely instruct LLM 31 on the content to be narrowed down.

[0067] [Generation process] Next, the generation process according to the embodiment will be described. Figure 14 is a flowchart showing the processing procedure of the generation process according to the embodiment.

[0068] As shown in Figure 14, in the generation device 10, the preprocessing unit 11 performs preprocessing to extract and format the information necessary for creating a report draft by LLM 31 from the information group input by the analyst via the analyst input device 20 (step S1).

[0069] Next, the perspective generation unit 12 performs a perspective generation process that generates new perspectives from documents related to the event to be reported, using the LLM 31 based on past report perspectives (step S2).

[0070] The generation unit 13 performs a report generation process that generates a report summarizing the events to be reported, using the LLM 31 based on the new perspective (new perspective) generated by the perspective generation unit 12 (step S3).

[0071] [Pre-processing procedure] Figure 15 is a flowchart showing the pre-processing steps shown in Figure 14.

[0072] As shown in Figure 15, the information source acquisition unit 111 performs an information source acquisition process to acquire information sources to be used for report generation (step S11). The information extraction unit 112 performs an information extraction process to extract text information and / or important image information from the input security information and convert the input information into information that can be interpreted by the LLM 31 (step S12).

[0073] The information verification unit 113 presents the information extracted by the information extraction unit 112 to the analyst, receives the corrected and verified text information from the analyst, and performs information verification processing to output it to the perspective generation unit 12 (step S13).

[0074] [New perspective generation process] Figure 16 is a flowchart showing the processing steps for the viewpoint generation process shown in Figure 14.

[0075] As shown in Figure 16, the perspective extraction unit 122 extracts past report perspectives from past reports, revised reports, and / or revised comments using LLM 31 and performs past report perspective storage processing to store them in the past report perspective DB 121 (step S21). If sufficient data has been stored in the past report perspective DB 121, step S21 may be omitted.

[0076] The new perspective generation unit 123 generates new perspectives from articles related to security events for which reports are to be generated, using LLM31, based on past report perspectives accumulated in the past report perspective DB121 (step S22).

[0077] [Report generation process] Figure 17 is a flowchart showing the processing steps for the report generation process shown in Figure 14.

[0078] As shown in Figure 17, the intermediate summary generation unit 131 performs an intermediate summary generation process to generate an intermediate summary (intermediate summary) for the article concerning the event to be reported, which reflects all of the content of the new perspectives generated in step S2 (step S31).

[0079] The report generation unit 132 performs a final report generation process (step S32) to generate and output a final report corresponding to the specified report format from the detailed summary generated in step S31.

[0080] [Effects of the embodiment] The generation device 10 according to this embodiment formalizes the know-how of experts in report creation into a form called "perspectives" that are easy for LLM31 to handle, based on previously accumulated revised reports and / or revised comments. Then, for each new security event to be reported, the generation device 10 newly generates perspectives appropriate to that security event, based on past report perspectives.

[0081] In this way, the generation device 10 creates a new perspective suitable for each new security event, based on past report perspectives, that is, the know-how of experts in report creation. Using this new perspective, the generation device 10 generates a new report for the LLM 31 that summarizes the minimum necessary information about the event to be reported in a short document such as a single slide.

[0082] Therefore, according to the generation device 10, even for vulnerability explanations other than incidents or complex security events with multiple timelines, it is possible to generate accurate reports of comparable quality to those produced by experts using LLM 31 by reflecting new perspectives.

[0083] Furthermore, the generation device 10 does not narrow down the security events to be reported into short content all at once. Instead, it first generates a detailed explanation as an intermediate summary, then determines what to narrow down from that detailed explanation, and removes information accordingly. This allows the generation device 10 to produce a consistent report that summarizes the key points clearly in short sentences. In addition, the generation device 10 extracts multiple perspectives and corresponding content for each slide and filters the content according to the creator's theme, allowing it to summarize important content on a single slide, going beyond the structure of the original document.

[0084] Furthermore, this embodiment can be used not only for security incidents but also for fields other than security, when generating short reports based on publicly available and private multilingual natural language information sources.

[0085] [System configuration of the embodiment] Each component of the generating device 10 is a functional concept and does not necessarily have to be physically configured as shown in the diagram. In other words, the specific forms of distribution and integration of the functions of the generating device 10 are not limited to those shown in the diagram, and all or part of it can be configured by functionally or physically distributing or integrating it in any unit according to various loads and usage conditions.

[0086] Furthermore, each process performed in the generation device 10 may be implemented, in whole or in part, by a CPU, a GPU (Graphics Processing Unit), and a program that is analyzed and executed by the CPU and GPU. Alternatively, each process performed in the generation device 10 may be implemented as hardware using wired logic.

[0087] Furthermore, among the processes described in the embodiments, all or part of the processes described as being performed automatically can be performed manually. Alternatively, all or part of the processes described as being performed manually can be performed automatically by known methods. In addition, the processing procedures, control procedures, specific names, and information including various data and parameters described above and illustrated may be changed as appropriate unless otherwise specified.

[0088] [program] Figure 18 shows an example of a computer in which the generation device 10 is realized when a program is executed. The computer 1000 has, for example, memory 1010 and CPU 1020. The computer 1000 also has a hard disk drive interface 1030, a disk drive interface 1040, a serial port interface 1050, a video adapter 1060, and a network interface 1070. These components are connected by a bus 1080.

[0089] Memory 1010 includes ROM 1011 and RAM 1012. ROM 1011 stores, for example, a boot program such as the BIOS (Basic Input Output System). The hard disk drive interface 1030 is connected to the hard disk drive 1090. The disk drive interface 1040 is connected to the disk drive 1100. For example, a removable storage medium such as a magnetic disk or optical disk is inserted into the disk drive 1100. The serial port interface 1050 is connected to, for example, the mouse 1110 and the keyboard 1120. The video adapter 1060 is connected to, for example, the display 1130.

[0090] The hard disk drive 1090 stores, for example, an OS (Operating System) 1091, an application program 1092, a program module 1093, and program data 1094. That is, the program that defines each process of the generation device 10 is implemented as a program module 1093 in which code executable by the computer 1000 is written. The program module 1093 is stored, for example, in the hard disk drive 1090. For example, a program module 1093 for performing the same processes as the functional configuration in the generation device 10 is stored in the hard disk drive 1090. Note that the hard disk drive 1090 may be replaced by an SSD (Solid State Drive).

[0091] Furthermore, the configuration data used in the processing of the above-described embodiment is stored as program data 1094 in, for example, memory 1010 or hard disk drive 1090. The CPU 1020 then reads the program module 1093 and program data 1094 stored in memory 1010 or hard disk drive 1090 into RAM 1012 as needed and executes them.

[0092] Furthermore, the program module 1093 and program data 1094 are not limited to being stored in the hard disk drive 1090; for example, they may be stored in a removable storage medium and read by the CPU 1020 via a disk drive 1100 or the like. Alternatively, the program module 1093 and program data 1094 may be stored in another computer connected via a network (LAN (Local Area Network), WAN (Wide Area Network), etc.). The program module 1093 and program data 1094 may then be read by the CPU 1020 from the other computer via a network interface 1070.

[0093] Although embodiments applying the invention made by the present inventors have been described above, the present invention is not limited by the descriptions and drawings that constitute part of the disclosure of the present invention in these embodiments. That is, all other embodiments, examples, and operational techniques made by those skilled in the art based on these embodiments are included in the scope of the present invention. [Explanation of Symbols]

[0094] 10 Generator 11 Pre-processing section 12. Perspective Generation Unit 13 Generation part 20 Analyst Input Devices 30 servers 31 LLM 111 Information Source Acquisition Department 112 Information extraction part 113 Information Verification Department 121 Past Report Perspectives Database 122 Perspective Extraction Section 123 New Perspective Generation Unit 131 Intermediate Summary Generation Unit 132 Report Generation Department

Claims

1. A storage unit that stores at least past report perspectives, which are perspectives that should be summarized from the original document when creating a report from scratch, extracted using the language model from past reports generated about past events using the language model, and revised reports that have been modified from the past reports, Based on the aforementioned past report perspectives, a first generation unit generates new perspectives from documents related to the events to be reported using the language model, A second generation unit generates a report summarizing the events targeted for report generation, based on the new perspective generated by the first generation unit, using the language model. A generating apparatus characterized by having the following features.

2. The generation apparatus according to claim 1, characterized in that the storage unit stores the perspectives of the past report extracted using a language model based on the past report, the revised report, and the revised comments on the past report.

3. The memory unit stores the past report perspective and the revised report corresponding to the past report perspective. The generation apparatus according to claim 1, characterized in that the first generation unit searches the storage unit for past events similar to the event to be reported, provides the language model with the revised report and the past report perspective corresponding to the searched past event as example sentences, and causes the language model to generate the new perspective from documents relating to the event to be reported.

4. The second generation unit is, A summary generation unit generates an intermediate summary that reflects all of the content of the new perspectives in the documents related to the events for which the report is to be generated, A report generation unit generates a final report corresponding to the specified report format from the aforementioned interim summary, The generating apparatus according to claim 1, characterized by having the following features.

5. A generation method performed by a generation device, The generation device has a storage unit that stores at least past report perspectives, which are perspectives that should be summarized from the original document when creating the report from scratch, extracted using the language model from past reports generated about past events using the language model, and revised reports in which modifications have been made to the past reports. Based on the aforementioned past report perspectives, a first generation step is performed to generate new perspectives from documents related to the events to be reported, using the language model; A second generation step, based on the new perspective generated in the first generation step, uses the language model to generate a report summarizing the events targeted for report generation, A method for generating a product that includes the following:

6. A first generation step involves using a language model to generate new perspectives from documents related to the events to be reported, based on past report perspectives, which are perspectives that should be summarized from the original documents when creating the report from scratch, extracted using the language model from past reports generated about past events and revised reports that have been modified from the past reports, using the language model; A second generation step involves generating a report summarizing the events targeted for report generation, using the language model based on the new perspective generated in the first generation step, A program generated for execution by a computer.

Images