Information display device, information display method, and program

Abstract

The presenter provides an information display device that can provide attack instructions, including specific attack methods to be implemented next, to operators conducting cyberattack tests, regardless of their skill level. [Solution] The information presentation device comprises: an acquisition unit that acquires attack information including at least one attack method performed in an ongoing cyberattack test; a search unit that searches past attack records including the attack method included in the attack information; a generation unit that generates instructions that indicate the next attack method to be performed by referring to a series of attack methods included in the searched past attack records; and an output unit that outputs attack instruction information including candidates for the next attack method output from the model in response to the input of instructions. The generation unit is an information presentation device that generates instructions including knowledge indicating the reason why the attack procedures of multiple attack records differ when multiple attack records that include a common attack method but have different attack procedures are searched.

Description

Technical Field

[0001] The present disclosure relates to an information presentation device, an information presentation method, and a program.

Background Art

[0002] As the importance of cyber security increases, the demand for cyber attack tests is increasing. In ordinary cyber attack tests, due to the complexity of the system and the diversity of attack methods, it is difficult to clarify the attack procedures. Therefore, specific attack procedures corresponding to the target system and tools could not be provided to the operator together with specific attack methods. As a result, when an inexperienced operator conducts a cyber attack test, there is a possibility that the vulnerabilities of the system to be pointed out may be overlooked.

[0003] Patent Document 1 discloses an information processing device aimed at comprehensively extracting attack paths assumed in a target system and deriving attack cases of each device that enables each attack path. The device of Patent Document 1 extracts attack paths from a list of the device group in the system and the connection relationship. The device of Patent Document 1 stores past attack cases in association with attack uses and node conditions. The device of Patent Document 1 searches for attack cases based on the node conditions of each device on the extracted attack path.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] With the method of Patent Document 1, an attack instruction including a specific attack method to be next executed cannot be presented to the operator. Therefore, when using the method of Patent Document 1, the selection of the next attack method depends on the proficiency of the operator.

[0006] The purpose of this disclosure is to provide an information presentation device, an information presentation method, and a program that can present attack instructions, including specific attack methods to be performed next, to an operator, regardless of the operator's skill level while conducting cyberattack tests. [Means for solving the problem]

[0007] An information presentation device in one aspect of the present disclosure includes: an acquisition unit that acquires attack information including at least one attack method performed in an ongoing cyberattack test; a search unit that searches past attack records including the attack methods included in the attack information; a generation unit that generates instructions for presenting the next attack method to be performed by referring to a series of attack methods included in the retrieved past attack records; and an output unit that outputs attack instruction information including candidates for the next attack method output from a model in response to the input of instructions. The generation unit generates instructions including knowledge indicating the reasons for the multiple attack records when multiple attack records are retrieved that include a common attack method but have different attack procedures.

[0008] In one aspect of the information presentation method of this disclosure, a computer obtains attack information including at least one attack method performed in an ongoing cyberattack test, searches for past attack records including the attack methods included in the attack information, refers to a series of attack methods included in the retrieved past attack records to generate instructions for the next attack method to be performed, outputs attack instruction information including candidates for the next attack method output from a model in response to the input of the instructions, and in the generation process, if multiple attack records are found that include a common attack method but have different attack procedures, the computer generates instructions including knowledge indicating the reasons why the attack procedures of the multiple attack records differ.

[0009] A program in one aspect of this disclosure causes a computer to perform the following processes: acquiring attack information including at least one attack method performed in an ongoing cyberattack test; searching for past attack records including the attack methods included in the attack information; generating instructions that suggest the next attack method to be performed by referring to a series of attack methods included in the searched past attack records; outputting attack instruction information including candidates for the next attack method output from a model in response to the input of instructions; and, if, in the generation process, multiple attack records that include a common attack method but have different attack procedures are found, generating instructions that include knowledge indicating the reason why the attack procedures of the multiple attack records differ. [Effects of the Invention]

[0010] According to this disclosure, it becomes possible to provide an information presentation device, an information presentation method, and a program that can present attack instructions, including specific attack methods to be performed next, to an operator, regardless of the operator's skill level when conducting cyberattack tests. [Brief explanation of the drawing]

[0011] [Figure 1] This block diagram shows an example of a configuration related to an information presentation device in this disclosure. [Figure 2] This block diagram shows an example of the configuration of an information presentation device in this disclosure. [Figure 3] This is a table showing an example of attack records stored in the database referenced by the information presentation device in this embodiment. [Figure 4] This table shows an example of knowledge stored in the database referenced by the information presentation device in this embodiment. [Figure 5] This is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. [Figure 6] This is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. [Figure 7]It is a conceptual diagram showing an example in which attack instruction information output from an information presentation device in the present disclosure is displayed on the screen of a terminal device. [Figure 8] It is a flowchart showing an example of the operation of an information presentation device in the present disclosure. [Figure 9] It is a flowchart showing an example of attack instruction information generation processing by an information presentation device in the present disclosure. [Figure 10] It is a conceptual diagram showing an example of a prompt generated by an information presentation device in the present disclosure. [Figure 11] It is a conceptual diagram showing an example of a prompt generated by an information presentation device in the present disclosure. [Figure 12] It is a conceptual diagram showing an example in which a user interface that receives an input of an answer to a question output by a large language model in response to a prompt input by an information presentation device in the present disclosure is displayed on the screen of a terminal device. [Figure 13] It is a conceptual diagram showing an example of a prompt generated by an information presentation device in the present disclosure. [Figure 14] It is a conceptual diagram showing an example in which attack instruction information output from an information presentation device in the present disclosure is displayed on the screen of a terminal device. [Figure 15] It is a conceptual diagram showing an example of a prompt generated by an information presentation device in the present disclosure. [Figure 16] It is a conceptual diagram showing an example in which attack instruction information output from an information presentation device in the present disclosure is displayed on the screen of a terminal device. [Figure 17] It is a conceptual diagram showing an example of a prompt generated by an information presentation device in the present disclosure. [Figure 18] It is a conceptual diagram showing an example of a prompt generated by an information presentation device in the present disclosure. [Figure 19] It is a conceptual diagram showing an example of a prompt generated by an information presentation device in the present disclosure. [Figure 20] It is a conceptual diagram showing an example in which attack instruction information output from an information presentation device in the present disclosure is displayed on the screen of a terminal device. [Figure 21]It is a block diagram showing an example of the configuration of the knowledge accumulation device in the present disclosure. [Figure 22] It is a conceptual diagram showing an example of the display of the information output from the knowledge accumulation device in the present disclosure. [Figure 23] It is a flowchart showing an example of the operation of the knowledge accumulation device in the present disclosure. [Figure 24] It is a block diagram showing an example of the configuration of the information presentation device in the present disclosure. [Figure 25] It is a flowchart showing an example of the operation of the information presentation device in the present disclosure. [Figure 26] It is a block diagram showing an example of the hardware configuration for executing the processing in the present disclosure.

Embodiments for Carrying Out the Invention

[0012] Hereinafter, embodiments for carrying out the present disclosure will be described with reference to the drawings. In the present disclosure, the drawings used in the description of each embodiment are associated with one or more embodiments. Also, the elements included in each drawing may apply to one or more embodiments. The embodiments described below have technically preferable limitations for carrying out the present disclosure, but do not limit the scope of the disclosure below. In all the drawings used in the description of the following embodiments, the same reference numerals are given to the same parts unless there is a particular reason. In the following embodiments, repeated descriptions may be omitted for the same configurations and operations. The direction of the arrows in the drawings indicates an example of the flow of signals, data, etc., and does not limit the flow of signals, data, etc.

[0013] (First Embodiment) First, the information presentation device according to the first embodiment will be described with reference to the drawings. The information presentation device of this embodiment presents attack instructions, including the specific attack method to be performed next, to an operator performing a penetration test, which is one type of cyberattack test. Penetration testing is also referred to as intrusion testing. In typical penetration tests, it is difficult to document attack procedures due to the complexity of the system and the diversity of attack methods. In this embodiment, based on the attack methods performed so far, attack instruction information, including the specific attack method to be performed next, is documented and presented.

[0014] (composition) Figure 1 is a block diagram showing an example configuration related to the information presentation device in this disclosure. The information presentation device 10 is connected to the terminal device 180 and the LLM (Large Language Models) system 150 via a network such as the Internet or an intranet. The information presentation device 10 is a device that performs processing related to penetration testing. For example, the information presentation device 10 may have functions such as analyzing the results of penetration testing, evaluating vulnerabilities, and proposing security measures. Details of the information presentation device 10 will be described later.

[0015] Terminal device 180 is an information processing device (computer) used by the worker. Terminal device 180 provides an interface for the worker to access the results and analysis information of the penetration test. Application software for conducting the penetration test is installed on terminal device 180. The terminal device used to conduct the penetration test and the terminal device connected to the information presentation device 10 may be configured as separate devices. For example, terminal device 180 may be implemented in the cloud or on a server. The functions of the application software for conducting the penetration test may be built on a server or cloud accessible from terminal device 180. Terminal device 180 may be implemented by a general-purpose computer. Alternatively, terminal device 180 may be implemented by a dedicated computer for conducting the penetration test.

[0016] Terminal device 180 accepts the selection of an attack method for conducting a simulated attack against the system under evaluation. Terminal device 180 conducts a simulated attack against the system under evaluation by executing processing according to the attack method selected by the operator. For example, terminal device 180 executes processing according to the attack method by using a tool to execute the attack method. Terminal device 180 outputs the attack work history (also called attack information) of the penetration test conducted by the operator to the information display device 10. The attack work history includes at least one attack method performed for each attack step and the attack details regarding the results of the performed attack method. The attack method is selected by the operator. For example, the attack method can be a Tactic, Technique, Procedure, or tool name. For example, the attack method may be defined as a combination of Tactic, Technique, Procedure, and tool name. For example, the attack details may be the command used in the attack, its options, and the execution result of the command.

[0017] For example, attack methods include network attacks, web application attacks, authentication / access control attacks, social engineering, system-level attacks, and highly targeted attacks. For example, network attacks include port scanning, man-in-the-middle attacks, denial-of-service attacks, and DNS (Domain Name System) poisoning. For example, network attacks include ARP (Address Resolution Protocol) spoofing and wireless network attacks. For example, web application attacks include SQL (Structured Query Language) injection, cross-site scripting, session hijacking, and directory traversal. For example, authentication / access control attacks include password cracking and privilege escalation. For example, social engineering includes phishing. For example, system-level attacks include buffer overflow attacks, memory corruption attacks, and reverse shells. For example, highly targeted attacks include exploiting zero-day vulnerabilities, ransomware attack simulations, and supply chain attack simulations.

[0018] The LLM system 150 is a system that performs processing using a large-scale language model (not shown). The large-scale language model (also called the model) is a deep learning model trained on a large-scale language dataset. The LLM system 150 uses the large-scale language model to output text information corresponding to the content of text information composed of natural language. The LLM system 150 uses the large-scale language model to provide penetration test results in easily understandable text information. In other words, the LLM system 150 converts complex security information into a format that is easy for humans to understand. For example, the LLM system 150 outputs an answer in response to a question input. The LLM system 150 may also be a model capable of inputting and outputting images and audio. For example, the LLM system 150 is a system that can be used via an API (Application Programming Interface). The LLM system 150 may be configured to use a dedicated model built for conducting penetration tests. As long as it can be accessed from the information presentation device 10, there are no limitations on the type of large-scale language model used by the LLM system 150 or the location where the LLM system 150 is deployed.

[0019] [Information presentation device] Next, an example of the configuration of the information presentation device 10 will be described with reference to the drawings. Figure 2 is a block diagram showing an example of the configuration of the information presentation device in this disclosure. The information presentation device 10 includes an acquisition unit 11, a search unit 13, an instruction unit 15, and an output unit 17. The information presentation device 10 also includes a database 130. The database 130 may be configured outside the information presentation device 10, provided that it is accessible from the information presentation device 10. The instruction unit 15 is connected to the LLM system 150.

[0020] The acquisition unit 11 is connected to a terminal device 180 used by the worker. The acquisition unit 11 acquires an attack work history from the terminal device 180 used by the worker, including the attack method and attack details for each attack step performed in the penetration test. In this embodiment, the attack work history includes attack methods for multiple attack steps that were executed in sequence.

[0021] Database 130 stores records of past attacks. Each attack record includes at least one attack method used in past penetration tests. Multiple attack records stored in Database 130 may include those with the same attack method, as well as those with different attack methods. Furthermore, multiple attack records stored in Database 130 may include a common attack method but differ in their attack procedures. Among the multiple attack records stored in Database 130, each of the multiple attack records that include a common attack method but differ in their attack procedures is called a similar attack record. For example, Database 130 uses a relational database management system that enables high-speed query processing and efficient management of large amounts of data. If attack records are normalized and stored using multiple tables, data integrity is maintained, and flexible searching and analysis become possible.

[0022] Figure 3 is a table showing an example of attack records stored in the database referenced by the information presentation device in this embodiment. Database 130 stores attack records (attack record R1, attack record R2, attack record R3, ...) related to attacks carried out in past penetration tests. Attack record R1 includes attack methods A, B, and C. Attack record R2 includes attack methods A, D, and E. Attack record R3 includes attack methods A, B, and F. Attack records R1, R2, and R3 have similarities and differences. Attack records R1, R2, and R3 are common in that attack method A was carried out in attack step 1. Attack record R1 and R2 are common in attack step 1, but differ in attack steps 2 and 3. Attack record R1 and R3 are common in attack steps 1 and 2, but differ in attack step 3. In other words, attack records R1, R2, and R3 are similar attack records that include common attack methods but differ in their attack procedures. The expression "different attack procedures while including common attack methods" refers to cases where some of the attack methods used in the attack records are different, or where the attack methods used are the same but the order is different. To put it another way, "different attack procedures while including common attack methods" refers to cases where the order, type, or combination of attack methods used in the attack records is different.

[0023] Furthermore, database 130 stores knowledge that shows the reasons why multiple similar attack records, including a common attack method, differ from one another. The knowledge includes commonalities and differences, as well as Q&A (Question and Answer), regarding multiple similar attack records that include a common attack method. Commonalities are attack methods that are common to multiple similar attack records that include a common attack method. In the example in Figure 3, the commonality between attack record R1 and attack record R2 is attack method A. Differences are attack methods that differ among multiple similar attack records that include a common attack method. In the example in Figure 3, the differences between attack record R1 and attack record R2 are attack methods B and attack method D. Q&A is knowledge that combines question Q, which asks why multiple attack records including a common attack method were carried out, and answer A to that question Q. For example, Q&A is created based on the knowledge of a skilled worker. In this embodiment, Q&A is assumed to be pre-created using answers to questions asked to workers who carried out similar attack records. For example, Q&A is knowledge compiled from questions and answers given to operators who have performed similar attacks in past penetration tests. Preferably, Q&A is knowledge compiled from questions and answers given to operators with extensive experience in penetration testing.

[0024] Figure 4 is a table showing an example of knowledge stored in the database referenced by the information presentation device in this embodiment. Figure 4 shows knowledge corresponding to similar attack records shown in Figure 3. Figure 4 shows an example of Q&A regarding multiple similar attack records in which some attack methods are common but other attack methods differ.

[0025] Knowledge item number 1 is an example of similar attack records, with attack record R1 and attack record R2 being examples. Attack record R1 and attack record R2 are similar in that attack method A was implemented. Attack record R1 and attack record R2 differ in the attack method implemented after attack method A. In attack record R1, attack method B was implemented after attack method A. In attack record R2, attack method D was implemented after attack method A. The Q&A includes question Q, which asks why the procedures for attack record R1 and attack record R2 are different, and answer A to that question. For example, question Q is asked to the person who carried out attack record R1 and attack record R2. Question Q may include the question, "In R1, B was implemented after A, but why was D implemented after A in R2?" Answer A may include the person's answer, "In R2, based on the results of A, it was determined that service T was running on the target system, and therefore D was effective after A."

[0026] Knowledge item number 2 is an example of similar attack records, specifically attack record R1 and attack record R3. Attack record R1 and attack record R3 are similar in that attack method B was executed following attack method A. Attack record R1 and attack record R3 differ in the attack method executed after attack method B. In attack record R1, attack method C was executed after attack method B. In attack record R3, attack method F was executed after attack method B. The Q&A includes question Q, which asks for the reason why attack record R1 and attack record R3 differ, and answer A to that question. Question Q is asked to the workers who executed attack record R1 and attack record R3. Question Q includes the question, "In R1, C was executed after A and B, but in R3, F was executed after A and B?" Answer A includes the worker's answer, "This was at the customer's request."

[0027] The search unit 13 searches the database 130 for attack records that include attack methods included in the attack work history. For example, the search unit 13 extracts attack records that match at least one of the attack methods included in the attack work history. For example, the search unit 13 extracts attack records that include an attack method that matches the last attack method executed among multiple attack methods included in the attack work history. For example, the search unit 13 extracts attack records that include more attack methods included in the attack work history. The search unit 13 may be configured to search for attack records that include attack methods included in the attack work history via the internet. For example, the search unit 13 may be configured to extract attack records that include attack methods included in the attack work history from publicly available information such as published threat reports. If there are multiple attack records that include attack methods included in the attack work history, and the procedures of the attack methods included in the extracted multiple attack records differ, those attack methods correspond to similar attack records.

[0028] The instruction unit 15 retrieves attack records retrieved by the search unit 13. Attack records include attack methods included in the attack work history. The instruction unit 15 generates a prompt (also called an instruction) that instructs the system to refer to the retrieved past attack records and present the next attack method to be performed. The functional configuration of the instruction unit 15 that generates prompts (instructions) is also called the generation unit. The instruction unit 15 inputs the generated prompt into the LLM system 150. If similar attack records containing attack methods included in the attack work history are retrieved, the instruction unit 15 retrieves knowledge from the database 130 that includes the reasons why those similar attack records differ. If there are multiple similar attack records containing attack methods included in the attack work history and the reasons why those similar attack records differ, the instruction unit 15 generates a first prompt that inputs the reasons why those similar attack records differ into the LLM system 150. The instruction unit 15 inputs the generated first prompt into the LLM system 150. The instruction unit 15 retrieves text information output from the LLM system 150 in response to the input of the first prompt. If there are no similar attack records, including the attack record included in the attack operation history, the instruction unit 15 does not input the reason for the difference in similar attack records to the LLM system 150 at the first prompt. Similarly, if there are no reasons for the difference in similar attack records, including the attack record included in the attack operation history, the instruction unit 15 does not input the reason for the difference in similar attack records to the LLM system 150 at the first prompt.

[0029] The first prompt includes information about each of the multiple attack methods that make up a similar attack record, and the reasons why the procedures in which those attack methods were carried out differ. The instruction unit 15 generates the first prompt using a pre-configured template. The template for generating the first prompt includes instructions to understand the results of the penetration test. The template for generating the first prompt also includes the content of the attack methods carried out for each similar attack record, and the reasons for the differences between those similar attack records. The instruction unit 15 inputs the generated first prompt to the LLM system 150. If there are no similar attack records that include the attack record included in the attack work history, the first prompt includes information about each of the multiple attack methods that make up the attack record that includes the attack method included in the attack work history. Also, if there are no reasons for the differences between similar attack records that include the attack record included in the attack work history, the first prompt includes information about each of the multiple attack methods that make up the attack record that includes the attack method included in the attack work history.

[0030] The instruction unit 15 acquires text information output from the LLM system 150 in response to the input of the first prompt. The text information output from the LLM system 150 in response to the input of the first prompt corresponds to the answer to the first prompt. The answer to the first prompt becomes a trigger for the information presentation device 10 to input the second prompt to the LLM system 150.

[0031] Figure 5 is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. Figure 5 shows the first prompt P1 that the information presentation device 10 inputs to the LLM system 150. The first prompt P1 includes an instruction, the content of each similar attack record, and information indicating the reason why similar attack records differ. The instruction includes the text information, "Please understand the results of the following penetration test." Regarding similar attack records, it includes text information about the multiple attack records that constitute that similar attack record and the attack methods that constitute those attack records. The reason why the attack methods included in the similar attack records differ (the reason for the difference between attack record R1 and attack record R2) is provided as text information, "In attack record R2, from the result of attack method A, ..." Also, Figure 5 shows the response A1 output from the LLM system 150 in response to the input of the first prompt P1. Response A1 includes the text information, "I understand," indicating that the LLM system 150 understood the content of the first prompt P1.

[0032] The instruction unit 15 generates a second prompt instructing the LLM system 150 to suggest the next attack method to be performed in the ongoing penetration test. If there are no attack records that include the attack method included in the attack work history, the instruction unit 15 generates the second prompt without generating the first prompt. If there are attack records that include the attack method included in the attack work history, the instruction unit 15 inputs the first prompt to the LLM system 150. In particular, if there are similar attack records that include the attack method included in the attack work history, and there is knowledge indicating the reason why these similar attack records differ, the instruction unit 15 generates a first prompt including that reason and inputs the generated first prompt to the LLM 150. The instruction unit 15 inputs the generated second prompt to the LLM system 150. The instruction unit 15 obtains text information (attack instruction information) output from the LLM system 150 in response to the input of the second prompt. The attack instruction information includes candidates for the next attack method to be performed.

[0033] The second prompt includes instructions that present candidate attack methods to be performed after the attack methods performed in the penetration test being conducted. The instruction unit 15 generates the second prompt using a pre-configured template. The instruction unit 15 inputs the generated second prompt to the LLM system 150. The instruction unit 15 obtains text information output from the LLM system 150 in response to the input of the second prompt. The text information output from the LLM system 150 in response to the input of the second prompt corresponds to the answer to the second prompt. The answer to the second prompt includes candidate attack methods to be performed next.

[0034] Figure 6 is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. Figure 6 shows the second prompt P2 that the information presentation device 10 inputs to the LLM system 150. The second prompt P2 includes an instruction and the attack method performed in the penetration. The instruction includes the text information, "Attack method A has been performed on the penetration target. Based on the results, please suggest the next attack method to perform." The performed attack method includes the results obtained by attack method A. Figure 6 shows the response A2 output from the LLM system 150 in response to the input of the second prompt P2. Response A2 includes the text information, "We recommend performing attack method D." Response A2 includes a candidate for the next attack (attack method D) based on the information input by the first prompt P1.

[0035] The output unit 17 is connected to a terminal device 180 used by the operator. The output unit 17 obtains attack instruction information, including candidate attack methods to be performed next, from the instruction unit 15. The output unit 17 outputs the obtained attack instruction information to the terminal device 180. The attack instruction information output to the terminal device 180 is displayed on the screen of the terminal device 180. The output unit 17 may further perform a process to extract candidate attack methods to be performed next from the response to the second prompt. If the response to the second prompt includes multiple candidate attack methods to be performed next, the output unit 17 may output some or all of them to the terminal device 180. For example, the output unit 17 may refer to a separately defined priority for each attack method, select the one with a higher priority than the attack methods included in the response, and output it to the terminal device 180.

[0036] Figure 7 is a conceptual diagram showing an example of how attack instruction information output from the information presentation device in this disclosure is displayed on the screen of a terminal device. The IP (Internet Protocol) address and port number of the target of the attack are displayed at the top of the screen of terminal device 180. Text information indicating the attack work history, such as "Attack method A was performed last time," is displayed on the screen of terminal device 180. Text information indicating the next attack method candidate, such as "We recommend performing attack method D next," is also displayed on the screen of terminal device 180. Furthermore, an attack selection UI (User Interface) that accepts the execution of an attack method is displayed on the screen of terminal device 180. In the example in Figure 7, the button for selecting attack method D, which is the next attack method candidate, is highlighted to stand out more than the buttons for the other attack methods. A cursor for selecting the button for attack method D is hovered over the button for selecting that button. By viewing the information displayed on the screen of terminal device 180, the operator can consider which attack method to perform next.

[0037] In the above description, the instruction unit 15 inputs information to the LLM system 150 using the first prompt and the second prompt to obtain candidate attack methods to be implemented next, but is not limited to this. For example, instead of the first prompt in the above description, the instruction unit 15 may input information to the LLM system 150 using RAG (Retrieval-Augmented Generation) or fine tuning. For example, the instruction unit 15 may generate a single prompt that includes both the information included in the first prompt and the information included in the second prompt. In that case, the instruction unit 15 inputs the single prompt to the LLM system 150 and obtains candidate attack methods to be implemented next by obtaining the text information output from the LLM system 150.

[0038] (operation) Next, an example of the operation of the information presentation device in this disclosure will be described with reference to the drawings. Figure 8 is a flowchart of an example of the operation of the information presentation device in this disclosure. In the explanation of the process according to the flowchart in Figure 8, the components of the information presentation device 10 will be considered the operating entities. The operating entities of the process according to the flowchart in Figure 8 may also be the information presentation device 10. For example, the process according to the flowchart in Figure 8 is realized by a processor executing a program stored in the memory installed in a computer (not shown) on which the information presentation device 10 is implemented.

[0039] In Figure 8, first, the acquisition unit 11 acquires an attack activity history, including the attack methods used in the ongoing penetration test (step S11). The attack activity history includes the attack methods used and the attack details, including the results of those attack methods. The attack record includes at least one attack method.

[0040] Next, the search unit 13 searches for attack records that include attack methods included in the attack work history (step S12). The search unit 13 searches for attack records from the database 130. The search unit 13 may be configured to search for attack records via the internet.

[0041] Next, the instruction unit 15 executes the attack instruction information generation process (step S13). Details of the attack instruction information generation process in step S13 will be described later.

[0042] Next, the output unit 17 outputs attack instruction information that includes candidate attack methods (step S14). The attack instruction information output from the information display device 10 is displayed on the screen of the terminal device 180 used to conduct the penetration test.

[0043] [Attack instruction information generation process] Next, an example of the attack instruction information generation process by the information presentation device in this disclosure (step S13 in Figure 8) will be described with reference to the drawings. Figure 9 is a flowchart of an example of the attack instruction information generation process by the information presentation device in this disclosure. In the explanation of the process according to the flowchart in Figure 9, the components of the information presentation device 10 (instruction unit 15) will be considered the main operating entity. The main operating entity of the process according to the flowchart in Figure 9 may also be the information presentation device 10.

[0044] In Figure 9, first, the instruction unit 15 determines whether there is an attack record that includes the attack method included in the attack work history (step S131). If there is an attack record that includes the attack method included in the attack work history (Yes in step S131), the instruction unit 15 determines whether there are multiple attack records that include a common attack method but have different attack procedures (step S132). If there is no attack record that includes the attack method included in the attack work history (No in step S131), the process proceeds to step S137.

[0045] If there are multiple attack records that include a common attack method but differ in their attack procedures (Yes in step S132), the instruction unit 15 determines whether there is a reason why the multiple similar attack records differ from one another (step S133). If there is no reason why the multiple similar attack records differ from one another (No in step S132), the process proceeds to step S135.

[0046] If there are reasons why multiple similar attack records differ from each other (Yes in step S133), the instruction unit 15 searches for the differences between the similar attack records and the reasons for those differences (step S134). If there are no reasons why multiple similar attack records differ from each other (No in step S133), the process proceeds to step S135.

[0047] If the answer in step S132 or step S133 is No, or if the answer in step S131 is No, the instruction unit 15 generates a first prompt for input to the LLM system 150 (step S135).

[0048] Next, the instruction unit 15 inputs the generated first prompt to the LLM system 150 (step S136). The instruction unit 15 then retrieves the text information output from the LLM system 150 in response to the input of the first prompt.

[0049] If the answer in step S131 is No, or if, following step S136, the instruction unit 15 generates a second prompt instructing the LLM system 150 to present the next attack method to be performed (step S137).

[0050] Next, the instruction unit 15 inputs the generated second prompt to the LLM system 150 (step S138).

[0051] Next, the instruction unit 15 acquires attack instruction information, including the next attack method, output from the LLM system 150 (step S139). Following step S139, the process proceeds to step S14 in the flowchart of Figure 8.

[0052] (modified version) Next, modifications of this embodiment will be described with reference to the drawings. Three modifications are given here. The following modifications are examples of processing by the information presentation device of this embodiment and do not limit the processing by the information presentation device of this embodiment.

[0053] [Variation 1] Figures 10 to 14 are conceptual diagrams relating to Modification 1. Modification 1 is an example of presenting a user interface to the operator that requests input of auxiliary information regarding the next attack method. The auxiliary information is requested from the LLM system 150 to the information presentation device 10 in order to determine the next attack method to be implemented.

[0054] Figure 10 is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. Figure 10 shows an example of the first prompt P1-1 that the information presentation device 10 inputs to the LLM system 150. The first prompt P1-1 is a prompt for inputting knowledge into the LLM system 150 regarding the reasons why multiple attack records that fall under the category of similar attack records differ. The first prompt P1-1 includes an instruction that reads, "Please understand the results of the following penetration test." The first prompt P1-1 also includes the attack details of attack records R1 and R3 that fall under the category of similar attack records. Furthermore, the first prompt P1-1 includes the knowledge that "the reason why attack records R1 and R3 differ is due to customer requests..."

[0055] Figure 10 also shows the response A1-1 output from the LLM system 150 in response to the input of the first prompt P1-1. Response A1-1 includes the text information, "I understand." Response A1-1 indicates that the content of the first prompt P1-1 was input to the LLM system 150 as a prerequisite.

[0056] Figure 11 is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. Figure 11 shows the second prompt P2-1-1 that the information presentation device 10 inputs to the LLM system 150. The second prompt P2-1-1 is a prompt that requests the LLM system 150 to suggest the next attack method to be implemented. The second prompt P2-1-1 includes the instruction, "Attack method A has been implemented against the penetration target. Based on the results, please suggest the next attack method to be implemented." The second prompt P2-1-1 also includes the text information, "Ask questions as necessary." This text information includes a request to the LLM system 150 for auxiliary information to determine the next attack method to be implemented.

[0057] Figure 11 also shows the response A2-1-1 output from the LLM system 150 in response to the input of the second prompt P2-1-1. Response A2-1-1 includes the text information, "Does the customer have any requests regarding attack methods?" Response A2-1-1 corresponds to the request for auxiliary information to determine the next attack method to be implemented, output from the LLM system 150 in response to the instruction "Ask questions as needed" included in the second prompt P2-1-1.

[0058] Figure 12 is a conceptual diagram showing an example of a user interface displayed on the screen of a terminal device that accepts input of answers to questions output from the information presentation device in this disclosure. The IP address and port number of the target of the attack are displayed at the top of the screen of the terminal device 180. The screen of the terminal device 180 displays text information requesting input of auxiliary information to determine the next attack method, such as "Do you have any requests for attack methods?" The screen of the terminal device 180 also displays a user interface (auxiliary information reception UI) for inputting auxiliary information. The auxiliary information reception UI displays a text area for inputting auxiliary information. If the number of characters that can be entered is small, a text box may be placed instead of a text area. The auxiliary information reception UI also displays a button for sending the auxiliary information entered by the worker to the information presentation device 10. For example, a worker who has viewed the information prompting the input of auxiliary information displayed on the screen of the terminal device 180 enters the auxiliary information in response to the request from the LLM system 150 into the text area. The auxiliary information entered in the text area is sent to the information presentation device 10 in response to a click of the button displayed as "Yes". If there is no auxiliary information to enter, the worker does not need to enter any. In that case, the worker can simply click the button labeled "None". When the button labeled "None" is clicked, information indicating that there is no auxiliary information is sent to the information display device 10. If the button labeled "Yes" is clicked when no auxiliary information has been entered in the text area, the system may be configured to send information indicating that there is no auxiliary information to the information display device 10. If the button labeled "Yes" is clicked when no auxiliary information has been entered in the text area, the system may be configured to display a pop-up on the terminal device 180 screen containing a message requesting the input of auxiliary information.

[0059] Figure 13 is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. Figure 13 shows the second prompt P2-1-2 input by the information presentation device 10 to the LLM system 150. The second prompt P2-1-2 is a response to the input of auxiliary information requested by the LLM system 150. In the example in Figure 13, since no auxiliary information was input by the operator, the second prompt P2-1-2 contains the text information "None". If auxiliary information is input by the operator, the information presentation device 10 inputs the auxiliary information into the second prompt P2-1-2 and inputs it to the LLM system 150. The LLM system 150 outputs a response corresponding to the instruction contained in the second prompt P2-1-1 in Figure 11, including the text information contained in the second prompt P2-1-2.

[0060] Figure 13 shows the response A2-1-2 output from the LLM system 150 in response to the input of the second prompt P2-1-2. Response A2-1-2 includes the text information, "We recommend implementing attack method C." Response A2-1-2 corresponds to the response to the instruction contained in the second prompt P2-1-1 in Figure 11, "Attack method A has been implemented against the penetration target. Based on the results, please suggest the next attack method to implement." Response A2-1-2 is a response that takes into account the content of the second prompt P2-1-2.

[0061] Figure 14 is a conceptual diagram showing an example of how attack instruction information output from the information display device in this disclosure is displayed on the screen of a terminal device. The IP address and port number of the target of the attack are displayed at the top of the screen of terminal device 180. The screen of terminal device 180 also displays information indicating the attack work history, stating, "Last time, attack method A was performed." The screen of terminal device 180 also displays a suggestion indicating the next attack method candidate, stating, "We recommend performing attack method C next." Furthermore, the screen of terminal device 180 displays an attack selection UI that accepts the execution of an attack method. In the example in Figure 14, the button for selecting attack method C, which is the next attack method candidate, is highlighted to stand out more than the buttons for the other attack methods. A cursor for selecting the button for attack method C is hovered over the button for selecting that button. By viewing the information displayed on the screen of terminal device 180, the operator can consider which attack method to perform next.

[0062] In this modified version, a large-scale language model is used to present the operator with an input screen that accepts supplementary information to determine the next attack method. According to this modified version, the operator can select a more appropriate next attack method considering past attack results and customer requests. Therefore, this modified version makes it possible to conduct penetration testing efficiently and effectively.

[0063] [Variation 2] Figures 15 and 16 are conceptual diagrams relating to Modification 2. Modification 2 is an example in which multiple attack method candidates are presented depending on the conditions. In this modification, the display example of the first prompt is omitted.

[0064] Figure 15 is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. Figure 15 shows an example of a second prompt P2-2 that the information presentation device 10 inputs to the LLM system 150. The second prompt P2-2 is a prompt that requests the LLM system 150 to suggest the next attack method to be implemented. The second prompt P2-2 includes the text information: "Attack method A has been implemented against the penetration target. Based on the results, please suggest the next attack method to be implemented."

[0065] Figure 15 also shows the response A2-2 output from the LLM system 150 in response to the input of the second prompt P2-2. Response A2-2 includes the text information: "We recommend implementing attack method B or attack method D. If service T is running, attack method D is more suitable." Response A2-2 includes several candidate attack methods. Response A2-2 also includes information indicating that attack method D is more suitable under the condition that a specific service is running.

[0066] Figure 16 is a conceptual diagram showing an example of how attack instruction information output from the information presentation device in this disclosure is displayed on the screen of a terminal device. The IP address and port number of the target of the attack are displayed at the top of the screen of terminal device 180. The attack history is displayed on the screen of terminal device 180, stating, "Last time, attack method A was performed." The screen of terminal device 180 also displays a suggestion, stating, "Next, we recommend performing the following attack methods." The screen of terminal device 180 displays attack methods B and D as the next attack methods. Attack method D has the condition, "if service T is running." Furthermore, the screen of terminal device 180 displays an attack selection UI that accepts the execution of an attack method. In the example in Figure 16, the buttons for selecting attack methods B and D, which are candidates for the next attack methods, are highlighted to stand out more than the buttons for other attack methods. In Figure 16, it is assumed that a specific service was not running. A cursor for selecting attack method B is overlaid on the button for selecting that button. The worker can consider the next attack method by viewing the information displayed on the screen of terminal device 180.

[0067] In this modified version, a large-scale language model is used to conditionally present multiple candidate attack methods. According to this modified version, the operator can select the most appropriate attack method depending on the target system's state and environment. Therefore, this modified version enables flexible and effective penetration testing.

[0068] [Variation 3] Figures 17 to 20 are conceptual diagrams relating to Modification 3. Modification 3 is an example in which multiple candidate attack methods with set priorities are presented. In this modification, the display example of the first prompt is omitted. Hereafter, it is assumed that the LLM system 150 outputs the candidate attack methods to be performed next in order of priority.

[0069] Figure 17 is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. Figure 17 shows an example of a second prompt P2-3-1 that the information presentation device 10 inputs to the LLM system 150. The second prompt P2-3-1 is a prompt that requests the LLM system 150 to suggest the next attack method to be implemented. The second prompt P2-3-1 includes the text information, "Attack method A has been implemented against the penetration target. Based on the results, please suggest the next attack method to be implemented." The implemented attack method includes the results obtained by that attack method. Figure 17 shows the response A2-3-1 output from the LLM system 150 in response to the input of the second prompt P2-3-1. Response A2-3-1 includes the text information, "We recommend implementing attack method D."

[0070] Figure 18 is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. Figure 18 shows a scene following Figure 17. Figure 18 shows an example of a second prompt P2-3-2 that the information presentation device 10 inputs to the LLM system 150 in response to the answer A2-3-1 output by the LLM system 150. The second prompt P2-3-2 is a prompt that requests the LLM system 150 to output further attack methods. The second prompt P2-3-2 contains the text information, "Anything else?". Figure 18 shows the answer A2-3-2 output by the LLM system 150 in response to the input of the second prompt P2-3-2. The answer A2-3-2 contains the text information, "We recommend implementing attack method B." Attack method B is the next highest priority attack method after attack method D.

[0071] Figure 19 is a conceptual diagram showing an example of a prompt generated by the information presentation device in this disclosure. Figure 19 shows a scene following Figure 18. Figure 19 shows an example of a second prompt P2-3-3 that the information presentation device 10 inputs to the LLM system 150 in response to the answer A2-3-2 output by the LLM system 150. The second prompt P2-3-3 is a prompt that requests the LLM system 150 to output further attack methods. The second prompt P2-3-3 contains the text information, "Are there any others?". Figure 19 shows an example of the answer A2-3-3 output by the LLM system 150 in response to the input of the second prompt P2-3-3. The answer A2-3-3 contains the text information, "Attack methods C and F can also be implemented." Attack methods C and F are attack methods with the next highest priority after attack method B.

[0072] Figure 20 is a conceptual diagram showing an example of how attack instruction information output from the information presentation device in this disclosure is displayed on the screen of a terminal device. The IP address and port number of the target of the attack are displayed at the top of the screen of terminal device 180. The attack history is displayed on the screen of terminal device 180, stating, "Last time, attack method A was performed." The screen of terminal device 180 also displays a suggestion, stating, "Next, we recommend performing the following attack methods." The screen of terminal device 180 displays attack methods D, B, C, and F as the next attack methods. Attack methods D, B, C, and F are listed in order of priority. Attack method D is the most promising candidate with the highest priority. Attack method B is the next candidate with the second highest priority after attack method D. Attack methods C and F are the next candidates with the third highest priority after attack method B. Furthermore, the screen of terminal device 180 displays an attack selection UI that accepts the execution of an attack method. In the example shown in Figure 20, the button for selecting attack method D, a strong candidate, is highlighted to be more conspicuous than the buttons for the other attack methods. The button for selecting attack method B, the next candidate, is less conspicuous than the button for attack method D, but is still highlighted to be more conspicuous than the buttons for the other attack methods. The buttons for selecting attack methods C and F, the next candidates, are less conspicuous than the button for attack method B, the next candidate, but are still highlighted to be more conspicuous than the buttons for the other attack methods. A cursor for selecting attack method D is displayed over the button. The operator can consider the next attack method to use by viewing the information displayed on the screen of terminal device 180.

[0073] In this modified version, a large-scale language model is used to present multiple attack methods in order of priority. Furthermore, in this modified version, the attack methods are visually distinguished and displayed so that their priority order can be distinguished. According to this modified version, the operator can intuitively understand and select the importance and effective order of the attack methods. Therefore, according to this modified version, strategic and efficient penetration testing becomes possible. In addition, according to this modified version, visual highlighting according to priority facilitates the operator's rapid decision-making and enables flexible selection of attack methods according to the situation. In the first embodiment and its modifications, the case in which penetration testing is treated as a cyberattack test was described as an example, but the present invention is not limited thereto. For example, the information presentation device 10 may be configured to handle vulnerability assessment in the same way as penetration testing.

[0074] As described above, the information presentation device of this embodiment comprises an acquisition unit, a search unit, an instruction unit, and an output unit. The acquisition unit acquires an attack work history that includes at least one attack method performed in the ongoing cyberattack test. The search unit searches for past attack records that include the attack methods included in the attack work history. For example, the search unit searches for past attack records that include the attack methods included in the attack work history from a database that stores attack records that include a series of attack methods performed in past cyberattack tests. The instruction unit refers to the series of attack methods included in the retrieved past attack records and generates instructions that present the next attack method to be performed. For example, the instruction unit inputs instructions to a large-scale language model requesting the presentation of candidates for attack methods to be performed after the previously performed attack method in the ongoing cyberattack test. For example, if the instruction unit finds multiple attack records that include a common attack method but have different attack procedures, it generates instructions that include knowledge indicating the reason why the order of consecutive attack methods included in the multiple attack records differs. The output unit outputs attack instruction information that includes candidates for the next attack method output from the large-scale language model in response to the input instructions.

[0075] In this embodiment, past attack records, including attack methods included in the attack work history of the ongoing cyberattack test, are searched. In this embodiment, a large-scale language model is used to extract candidate attack methods to be executed next based on past attack records. According to this embodiment, regardless of the skill level of the worker conducting the cyberattack test, attack instructions including specific attack methods to be executed next can be presented to the worker.

[0076] In one embodiment of this system, if a database contains multiple similar attack records, including attack methods included in the attack work history, the instruction unit inputs instructions to a large-scale language model that include knowledge explaining why the multiple similar attack records differ. The similar attack records are at least two attack records that include a common attack method but differ in their attack procedures. The output unit outputs attack instruction information that includes candidate attack methods for the next attack, output from the large-scale language model according to the knowledge. According to this embodiment, it is possible to present attack instructions that include candidate attack methods for the next attack, selected according to knowledge explaining why the multiple similar attack records differ.

[0077] In one embodiment of this system, the instruction unit receives a request for auxiliary information from a large-scale language model to determine the next attack method to be implemented. The output unit outputs a request for input of auxiliary information. The acquisition unit acquires the input auxiliary information in response to the input request. The instruction unit inputs instructions, including the acquired auxiliary information, into the large-scale language model. In this embodiment, the large-scale language model is used to present the operator with auxiliary information to determine the next attack method. According to this embodiment, the operator can select a more appropriate next attack method considering past attack results and customer requests. Therefore, according to this embodiment, it becomes possible to conduct cyberattack tests efficiently and effectively.

[0078] In one embodiment of this system, the instruction unit inputs an instruction to the large-scale language model requesting the presentation of multiple candidate attack methods. The output unit outputs attack instruction information containing multiple candidates for the next attack method output from the large-scale language model. In this embodiment, the large-scale language model is used to conditionally present multiple candidates as the next attack method. According to this embodiment, the operator can select a more appropriate attack method depending on the system status and environment of the target. Therefore, according to this embodiment, it becomes possible to conduct cyberattack tests flexibly and effectively.

[0079] In one embodiment of this system, the instruction unit continuously inputs instructions to a large-scale language model requesting multiple attack method candidates. The output unit outputs the multiple attack method candidates output from the large-scale language model. In this embodiment, the large-scale language model is used to present multiple attack methods in order of priority and display them visually distinctly. According to this embodiment, the operator can intuitively understand and select the importance and effective order of the attack methods. Therefore, this embodiment enables the implementation of strategic and efficient cyberattack tests. Furthermore, according to this embodiment, visual highlighting according to priority facilitates the operator's rapid decision-making and enables flexible selection of attack methods according to the situation.

[0080] (Second Embodiment) Next, the knowledge storage device according to the second embodiment will be described with reference to the drawings. The knowledge storage device of this embodiment is connected to a terminal device similar to that of the first embodiment. The knowledge storage device of this embodiment acquires attack records, including attack methods, that were carried out to attack the system under evaluation, via the terminal device. The knowledge storage device of this embodiment stores the carried-out attack records in a database. If the procedures of the attack methods that constitute the attack records differ, the knowledge storage device of this embodiment asks the operator about the reason for the difference and acquires the reason entered as the answer to that question as knowledge. The knowledge storage device of this embodiment stores knowledge indicating the reason why at least two attack records that fall under similar attack records differ in a database referenced by the information presentation device of the first embodiment.

[0081] (composition) Figure 21 is a block diagram showing an example of the configuration of a knowledge storage device in this disclosure. The knowledge storage device 20 comprises an acquisition unit 21, a storage unit 23, an extraction unit 25, and a questioning unit 27. The knowledge storage device 20 also comprises a database 230. The database 230 corresponds to the database 130 referenced by the information presentation device 10 of the first embodiment. The database 230 may be configured outside the knowledge storage device 20, provided that it is connectable from the knowledge storage device 20.

[0082] The acquisition unit 21 is connected to a terminal device (not shown) used by the operator. The acquisition unit 21 acquires attack results performed in cyberattack tests from the terminal device. The acquisition unit 21 may be configured to acquire previously accumulated attack results from an external database or the like. The attack results include attack methods performed in the past and the results of those attack methods. In this embodiment, the attack results include multiple attack methods that were executed in sequence.

[0083] The storage unit 23 stores the attack records acquired by the acquisition unit 21 in the database 230. For example, the storage unit 23 assigns a unique identifier to each attack record and stores the attack record, including the details of the attack performed using the attack method. For example, the attack details include information such as the tools and scripts used for each attack method, the success rate of the attack, the system vulnerabilities detected, and the impact of the attack, all stored in a structured format. For example, the storage unit 23 records the relationships and sequence of a series of attack methods and stores the entire attack scenario, composed of multiple attack methods, in a reproducible form.

[0084] The extraction unit 25 searches for multiple attack records stored in the database 230. From the multiple attack records, the extraction unit 25 extracts multiple attack records that contain a common attack method but have different attack procedures. Multiple attack records that contain a common attack method but have different attack procedures are considered similar attack records. First, the extraction unit 25 extracts multiple attack records that contain a common attack method from the multiple attack records. Next, the extraction unit 25 extracts multiple attack records that contain a common attack method but have different attack methods. The multiple attack records extracted here are similar attack records. The extraction unit 25 instructs the query unit 27 to ask why the attack procedures of the similar attack records differ. If no similar attack records are extracted from the attack records that contain a common attack method, the extraction unit 25 does not instruct the query unit to ask for the reason for those attack records.

[0085] The questioning unit 27 is connected to a terminal device (not shown) used by the operator. In response to instructions from the extraction unit 25, the questioning unit 27 sends a question Q to the terminal device asking for the reasons for the differences between multiple attack records that include a common attack method but have different attack procedures. The questioning unit 27 also obtains an answer A to the question Q sent from the terminal device. The questioning unit 27 stores the knowledge contained in the answer A in the database 230. The knowledge includes commonalities, differences, and Q&A sets of question Q and answer A for each similar attack record. An example of knowledge including Q&A is shown in Figure 4.

[0086] Figure 22 is a conceptual diagram showing an example of the display of information output from the knowledge storage device in this disclosure. The screen of the terminal device 280 used by the operator displays information showing the differences between attack record R1 and attack record R2. In attack record R1, attack method A, attack method B, and attack method C were carried out in that order. In attack record R2, attack method A, attack method D, and attack method E were carried out in that order. Attack record R1 and attack record R2 are common in that attack method A was carried out. Attack record R1 and attack record R2 differ in that the attack method carried out after attack method A is different. The screen of the terminal device 280 displays question Q, which asks for the reason why attack record R1 and attack record R2 are different. Question Q is text information output from the question unit 27. Question Q includes the text information, "In R1, B was carried out after A, but in R2, D was carried out after A, why?" The screen of the terminal device 280 also displays answer A to question Q. Answer A is entered by the worker who reviewed question Q. Answer A contains the text information, "In R2, based on the result of A, it was determined that service T was running on the target system, and that D was effective after A." For example, service T is a running service such as remote desktop. A button for sending the entered answer A to the knowledge storage device 20 is displayed in the lower right corner of the terminal device 280 screen. A cursor for selecting the button is hovered over the button. Answer A sent to the knowledge storage device 20 is stored in the database 230 as knowledge of similar attack methods, along with question Q.

[0087] (operation) Next, an example of the operation of the knowledge storage device in this disclosure will be described with reference to the drawings. Figure 23 is a flowchart illustrating an example of the operation of the knowledge storage device in this disclosure. In describing the process according to the flowchart in Figure 23, the components of the knowledge storage device 20 are considered the main operating entities. The main operating entity for the process according to the flowchart in Figure 23 may also be the knowledge storage device 20. For example, the process according to the flowchart in Figure 23 is realized by a processor executing a program stored in the memory of a computer (not shown) on which the knowledge storage device 20 is implemented.

[0088] In Figure 23, first, the acquisition unit 21 acquires attack records related to attacks carried out in past penetration tests (step S21).

[0089] Next, the storage unit 23 stores the attack methods and attack details related to the acquired attack records in the database 230 (step S22).

[0090] If there are similar attack records among the multiple attack records stored in the database 230 (Yes in step S23), the extraction unit 25 extracts the similar attack records (step S24). On the other hand, if there are no similar attack records among the multiple attack records stored in the database 230 (No in step S23), the process according to the flowchart in Figure 23 is completed.

[0091] Following step S24, the questioning unit 27 outputs a question to the operator's terminal device 280 asking for the reasons for the differences in attack methods in similar attack records (step S25).

[0092] Next, the questioning unit 27 obtains the answer corresponding to the question from the worker's terminal device 280 (step S26).

[0093] Next, the question unit 27 links the knowledge contained in the answer to similar attack records and stores it in the database 230 (step S27). The knowledge contained in the answer may also be configured to be stored in the database 230 by the storage unit 23.

[0094] As described above, the knowledge storage device of this embodiment comprises an acquisition unit, a storage unit, an extraction unit, and a questioning unit. The acquisition unit acquires attack results performed in cyberattack tests. The storage unit stores the attack methods and attack details included in the attack results in a database. The extraction unit searches for multiple attack results stored in the database. The extraction unit extracts multiple attack results from among the multiple attack results that include a common attack method but have different attack procedures. The questioning unit sends questions to the terminal device regarding the multiple attack results that include a common attack method but have different attack procedures, asking for the reasons why they differ. The questioning unit also acquires answers to the questions sent from the terminal device. The questioning unit stores the knowledge included in the answers in the database.

[0095] In this embodiment, attack records related to attacks carried out in cyberattack tests are stored in a database. In this embodiment, knowledge is stored, including the reasons for the differences, for multiple attack records that include a common attack method but have different attack procedures. According to this embodiment, a database of accumulated attack records can be constructed to present attack instructions, including specific attack methods to be carried out next, to operators, regardless of their skill level when conducting cyberattack tests. Furthermore, according to this embodiment, knowledge is stored in the database, including the reasons for the differences, for multiple attack records (similar attack records) that include a common attack method but have different attack procedures. The attack records and knowledge stored in the database are referenced by the information presentation device of the first embodiment.

[0096] (Third embodiment) Next, the information presentation device in the third embodiment will be described with reference to the drawings. The information presentation device in this embodiment has a simplified configuration compared to the information presentation device in the first embodiment. For example, the functions of the components of the information presentation device in this embodiment are realized by the functions of the components of the information presentation device according to the first embodiment.

[0097] (composition) Figure 24 is a block diagram showing an example of the configuration of an information presentation device in this disclosure. The information presentation device 30 comprises an acquisition unit 31, a search unit 33, a generation unit 35, and an output unit 37.

[0098] The acquisition unit 31 acquires attack information that includes at least one attack method performed in the ongoing cyberattack test. The search unit 33 searches for past attack records that include the attack methods included in the attack information. The generation unit 35 refers to the series of attack methods included in the searched past attack records and generates instructions that suggest the next attack method to be performed. If the generation unit 35 finds multiple attack records that include a common attack method but have different attack procedures, it generates instructions that include knowledge indicating the reason why the attack procedures of the multiple attack records differ. The output unit 37 outputs attack instruction information that includes the next candidate attack method output from the model in response to the input instructions.

[0099] (operation) Figure 25 is a flowchart illustrating an example of the operation of the information presentation device in this disclosure. In the explanation of the process according to the flowchart in Figure 25, the components of the information presentation device 30 are considered the operating entities. The operating entities of the process according to the flowchart in Figure 25 may also be the information presentation device 30.

[0100] The acquisition unit 31 acquires attack information that includes at least one attack method used in the ongoing cyberattack test (step S31).

[0101] The search unit 33 searches for past attack records that include the attack methods contained in the attack information (step S32).

[0102] The generation unit 35 refers to a series of attack methods included in the retrieved past attack records and generates instructions that present the next attack method to be performed (step S33).

[0103] If the generation unit 35 finds multiple attack records that include a common attack method but have different attack procedures (Yes in step S34), it generates instructions that include knowledge indicating the reason why the attack procedures of the multiple attack records differ (step S35). If no multiple attack records with different sequences of consecutive attack methods are found (No in step S34), the process proceeds to step S36.

[0104] If the answer is No in step S35, or if the answer is No in step S34, the output unit 37 outputs attack instruction information including the next candidate attack method output from the model in response to the input instruction (step S36).

[0105] In this embodiment, past attack records, including attack methods included in the attack work history of the ongoing cyberattack test, are searched. In this embodiment, a large-scale language model is used to extract candidate attack methods to be executed next based on past attack records. According to this embodiment, regardless of the skill level of the worker conducting the cyberattack test, attack instructions including specific attack methods to be executed next can be presented to the worker.

[0106] (Hardware) Next, the hardware configuration for performing the processing described in this disclosure will be described with reference to the drawings. Figure 26 is a block diagram showing an example of a hardware configuration for performing the processing described in this disclosure. Here, an information processing device 90 (computer) is shown as an example of a hardware configuration. The information processing device in Figure 26 is an example configuration for performing the processing described in this disclosure and does not limit the scope of this disclosure.

[0107] As shown in Figure 26, the information processing device 90 comprises a processor 91, memory 92, auxiliary storage device 93, input / output interface 95, and communication interface 96. In Figure 26, interface is abbreviated as I / F (Interface). The information processing device 90 may include at least one or more of the processor 91, memory 92, auxiliary storage device 93, input / output interface 95, and communication interface 96. The processor 91, memory 92, auxiliary storage device 93, input / output interface 95, and communication interface 96 are connected to each other via a bus 98 so that they can communicate data. In addition, the processor 91, memory 92, auxiliary storage device 93, and input / output interface 95 are connected to a network such as the Internet or an intranet via the communication interface 96.

[0108] The processor 91 loads a program (instruction) stored in an auxiliary storage device 93 or the like into memory 92. For example, the program is a software program for executing the processing described in this disclosure. The processor 91 executes the program loaded into memory 92. The processor 91 executes the processing described in this disclosure by executing the program. The processor 91 may be composed of a single piece of hardware or of multiple pieces of hardware.

[0109] Memory 92 is a storage device having an area where programs are deployed. The processor 91 deploys programs stored in auxiliary storage devices 93, etc., into memory 92. Memory 92 can be implemented using volatile memory such as DRAM (Dynamic Random Access Memory). Alternatively, non-volatile memory such as MRAM (Magnetoresistive Random Access Memory) may be used as memory 92. Memory 92 may be composed of a single piece of hardware or multiple pieces of hardware.

[0110] The auxiliary storage device 93 stores various data, such as programs. For example, the auxiliary storage device 93 can be implemented by a local disk such as a hard disk or flash memory. The auxiliary storage device 93 may be configured by a single piece of hardware or by multiple pieces of hardware. The auxiliary storage device 93 may also be configured as external hardware. It is also possible to configure the system to store various data in memory 92 and omit the auxiliary storage device 93.

[0111] The input / output interface 95 is an interface for connecting the information processing device 90 to peripheral devices based on standards and specifications. The communication interface 96 is an interface for connecting to external systems and devices via a network such as the Internet or an intranet, based on standards and specifications. The input / output interface 95 may be composed of a single piece of hardware or multiple pieces of hardware. The input / output interface 95 and the communication interface 96 may be common as interfaces for connecting to external devices.

[0112] The information processing device 90 may be connected to input devices such as a keyboard, mouse, or touch panel, as needed. These input devices are used to input information and settings. When a touch panel is used as an input device, the screen with touch panel functionality serves as the interface. The processor 91 and the input devices are connected via an input / output interface 95.

[0113] The information processing device 90 may be equipped with a display device for displaying information. If a display device is provided, the information processing device 90 is equipped with a display control device (not shown) for controlling the display of the display device. The information processing device 90 and the display device are connected via an input / output interface 95.

[0114] The information processing device 90 may be equipped with a drive device. The drive device mediates between the processor 91 and the recording medium (program recording medium) by reading data and programs stored on the recording medium and writing the processing results of the information processing device 90 to the recording medium. The information processing device 90 and the drive device are connected via an input / output interface 95.

[0115] The above is an example of a hardware configuration that enables the processing described in this disclosure. The hardware configuration in Figure 26 is an example of a hardware configuration for executing the processing described in this disclosure and does not limit the scope of this disclosure. A program that causes a computer to execute the processing described in this disclosure is also included in the scope of this disclosure.

[0116] A program recording medium that stores a program for performing the processing in this embodiment is also included in the scope of the present invention. For example, the program recording medium is a computer-readable, non-transient recording medium. The recording medium can be implemented as an optical recording medium such as a CD (Compact Disc) or DVD (Digital Versatile Disc). The recording medium may also be implemented as a semiconductor recording medium such as a USB (Universal Serial Bus) memory or an SD (Secure Digital) card. Furthermore, the recording medium may be implemented as a magnetic recording medium such as a flexible disk, or other recording media.

[0117] The components in this disclosure may be combined in any way. The components in this disclosure may be implemented by cloud computing. The components in this disclosure may be implemented by software. The components in this disclosure may be implemented by circuitry.

[0118] Although the present disclosure has been described above with reference to embodiments, the present disclosure is not limited to the embodiments described above. Various modifications to the structure and details of the present disclosure can be made as can be understood by those skilled in the art within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0119] Some or all of the above embodiments may also be described as follows, but are not limited to the following. In the following appendices, the dependents of each category may also be dependent on other categories. The descriptions included in the following appendices are significant as grounds for amendment. (Note 1) An acquisition unit that acquires attack information including at least one attack method used in ongoing cyberattack tests, A search unit that searches for past attack records including attack methods included in the aforementioned attack information, A generation unit that generates instructions to propose the next attack method to be executed by referring to a series of attack methods included in the searched past attack records, The system includes an output unit that outputs attack instruction information, including candidate attack methods output from the model in response to the input of the aforementioned instruction, The generating unit is An information presentation device that, when multiple attack records are found that share a common attack method but have different attack procedures, generates instructions that include knowledge explaining the reasons why the attack procedures of the multiple attack records differ. (Note 2) The aforementioned search unit, An information presentation device as described in Appendix 1, which searches a database containing attack records, including a series of attack methods, that have been carried out in past cyberattack tests, for past attack records that include the attack methods included in the aforementioned attack information. (Note 3) The generating unit is If the database contains multiple attack records that differ from each other but include the attack methods described in the attack information, then instructions including knowledge about why the multiple attack records differ from each other are input to the model. The output unit is, An information presentation device as described in Appendix 2, which outputs attack instruction information including candidate attack methods output from the model in accordance with the aforementioned knowledge. (Note 4) The information presentation device described in Appendix 3, wherein multiple attack records that differ from each other are at least two attack records that include a common attack method but have different attack procedures. (Note 5) The generating unit is An information presentation device as described in Appendix 1, which inputs instructions to the model requesting the presentation of candidate attack methods to be implemented following the previously implemented attack method in an ongoing cyberattack test. (Note 6) The generating unit is The model then receives a request for auxiliary information to determine the next attack method to be implemented. The output unit is, Output a request for input of supplementary information, The acquisition unit is, The auxiliary information entered in response to the aforementioned input request is acquired, The generating unit is An information display device according to any one of Appendix 1 to 5, which inputs instructions including the acquired auxiliary information into the model. (Note 7) The generating unit is The model is instructed to present a list of candidate attack methods. The output unit is, An information presentation device according to any one of the appendices 1 to 5 that outputs attack instruction information including multiple candidates for the following attack method output from the aforementioned model. (Note 8) The generating unit is The model is given a series of instructions to find candidates for multiple attack methods. The output unit is, An information presentation device described in any one of the appendices 1 to 5 that outputs a plurality of candidate attack methods output from the aforementioned model. (Note 9) Computers We obtain attack information that includes at least one attack method used in ongoing cyberattack tests. Search past attack records, including the attack methods included in the aforementioned attack information. Referencing a series of attack methods included in the searched past attack records, generate instructions that specify the next attack method to be executed. In response to the input of the aforementioned instructions, the model outputs attack instruction information including candidate attack methods, In the above generation, An information presentation method that generates instructions containing knowledge explaining the reasons for the differences in attack procedures among multiple attack records that include a common attack method but differ in their attack procedures. (Note 10) On the computer, A process to obtain attack information that includes at least one attack method used in ongoing cyberattack tests, A process to search for past attack records that include the attack methods included in the aforementioned attack information, A process that generates instructions to suggest the next attack method to be executed, by referring to a series of attack methods included in the searched past attack records, A process that outputs attack instruction information, including candidate attack methods output from the model in response to the input of the aforementioned instruction, In the above generation process, A program that, when multiple attack records are found that contain a common attack method but have different attack procedures, generates instructions containing knowledge explaining the reasons why the attack procedures of the multiple attack records differ. Furthermore, some or all of the configurations described in Appendices 2 to 8, which are subordinate to Appendice 1 above, may also be subordinate to Appendices 9 and 10 in the same way as those described in Appendices 2 to 8. Moreover, not limited to Appendices 1, 9, and 10, some or all of the configurations described as appendices may also be subordinate to various hardware, software, various recording means for recording software, or systems, without departing from the embodiments described above. [Explanation of symbols]

[0120] 10, 30 Information presentation device 11, 31 Acquisition Department 13, 33 Search section 15 Instruction section 17, 37 Output section 20 Knowledge Storage Device 21 Acquisition Department 23 Storage Unit 25 Extraction part 27 Question Section 35 Generation part 130, 230 databases 150 LLM System 180, 280 terminal devices

Claims

1. An acquisition unit that acquires attack information including at least one attack method used in ongoing cyberattack tests, A search unit that searches for past attack records including attack methods included in the aforementioned attack information, A generation unit that generates instructions to propose the next attack method to be executed by referring to a series of attack methods included in the searched past attack records, The system includes an output unit that outputs attack instruction information, including candidate attack methods output from the model in response to the input of the aforementioned instruction, The generating unit is An information presentation device that, when multiple attack records are found that share a common attack method but have different attack procedures, generates instructions that include knowledge explaining the reasons why the attack procedures of the multiple attack records differ.

2. The aforementioned search unit, The information presentation device according to claim 1, which searches for past attack records including attack methods included in the attack information from a database that has accumulated attack records including a series of attack methods carried out in past cyberattack tests.

3. The generating unit is If the database contains multiple attack records that differ from each other but include the attack methods described in the attack information, then instructions including knowledge about why the multiple attack records differ from each other are input to the model. The output unit is, The information presentation device according to claim 2, which outputs attack instruction information including candidate attack methods output from the model in accordance with the knowledge described above.

4. The information presentation device according to claim 3, wherein the multiple attack records that differ from each other are at least two attack records that include a common attack method but have different attack procedures.

5. The generating unit is The information presentation device according to claim 1, which inputs an instruction to the model requesting the presentation of candidate attack methods to be implemented after the previously implemented attack method in an ongoing cyberattack test.

6. The generating unit is The model then receives a request for auxiliary information to determine the next attack method to be implemented. The output unit is, Output a request for input of supplementary information, The acquisition unit is, The auxiliary information entered in response to the aforementioned input request is acquired, The generating unit is An information presentation device according to any one of claims 1 to 5, which inputs instructions including the acquired auxiliary information into the model.

7. The generating unit is The model is instructed to present a list of candidate attack methods. The output unit is, An information presentation device according to any one of claims 1 to 5, which outputs attack instruction information including a plurality of candidates for the following attack method output from the aforementioned model.

8. The generating unit is The model is given a series of instructions to find candidates for multiple attack methods. The output unit is, An information presentation device according to any one of claims 1 to 5, which outputs a plurality of candidate attack methods output from the aforementioned model.

9. Computers We obtain attack information that includes at least one attack method used in ongoing cyberattack tests. Search past attack records, including the attack methods included in the aforementioned attack information. Referencing a series of attack methods included in the searched past attack records, generate instructions that specify the next attack method to be executed. In response to the input of the aforementioned instructions, the model outputs attack instruction information including candidate attack methods, In the above generation, An information presentation method that generates instructions containing knowledge explaining the reasons for the differences in attack procedures among multiple attack records that include a common attack method but differ in their attack procedures.

10. On the computer, A process to obtain attack information that includes at least one attack method used in ongoing cyberattack tests, A process to search for past attack records that include the attack methods included in the aforementioned attack information, A process that generates instructions to suggest the next attack method to be executed, by referring to a series of attack methods included in the searched past attack records, A process that outputs attack instruction information, including candidate attack methods output from the model in response to the input of the aforementioned instruction, In the above generation process, A program that, when multiple attack records are found that contain a common attack method but have different attack procedures, generates instructions containing knowledge explaining the reasons why the attack procedures of the multiple attack records differ.

Images