Information processing device and information processing method

Abstract

To evaluate whether a device to be evaluated can detect an attack to equipment without actually attacking the equipment.SOLUTION: An information processing apparatus comprises: a pseudo-trace generation part which generates pseudo-trace information on a second attack based upon trace information on a first attack acquired during the first attack to first equipment on a communication network and information associated with an attacking method of the second attack to second equipment; and a pseudo-trace transmission part which transmits the pseudo-trace information on the second attack to the device to be evaluated which detects the attack based upon the trace information on the attack.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to an information processing apparatus and an information processing method.

Background Art

[0002] There is a security countermeasure device that detects or blocks cyberattacks on a system. In order to evaluate the performance of the security countermeasure device and the settings at the time of system introduction, there is a method of actually attacking the system and having the security countermeasure device read the communication data flowing through the network during the attack or the logs acquired by the host targeted during the attack. However, actually executing an attack on the system may have an adverse effect on the system.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present embodiment provides an information processing apparatus and an information processing method that enable evaluating whether an evaluation target device can detect an attack on a device without actually attacking the device.

Means for Solving the Problems

[0005] The information processing apparatus according to the present embodiment includes a pseudo trace generation unit that generates pseudo trace information of the second attack based on trace information of the first attack acquired during the first attack on the first device and information on the attack method of the second attack on the second device, and a pseudo trace transmission unit that transmits the pseudo trace information of the second attack to an evaluation target device that detects the attack based on the trace information of the attack.

Brief Description of the Drawings

[0006] [Figure 1] A diagram showing an example of a security system according to the first embodiment. [Figure 2] A diagram showing an example configuration of a monitored system incorporating the device under evaluation and the security measures evaluation device. [Figure 3] A functional block diagram showing an example configuration of a security measures evaluation device. [Figure 4] A diagram illustrating an example attack scenario. [Figure 5] A diagram showing an example of network configuration information. [Figure 6] A diagram showing an example of attack trace data stored in the attack trace memory unit. [Figure 7] Figure 6 shows an example of trace information in attack trace data. [Figure 8] A diagram showing an example of pseudo-trace information generated from trace information based on parameters. [Figure 9] A diagram illustrating specific examples of trace information and pseudo-traces information. [Figure 10] A flowchart showing an example of the processing procedure of the security countermeasure evaluation device according to this embodiment. [Figure 11] A flowchart illustrating an example of a process for generating pseudo-traces when the trace information is communication data. [Figure 12] A flowchart illustrating an example of a processing procedure for generating pseudo-traces when the trace information is in the form of logs. [Figure 13] A functional block diagram showing an example configuration of a security countermeasure evaluation device according to the second embodiment. [Figure 14] A diagram showing an example of evaluation result data. [Figure 15] A flowchart showing an example of the processing procedure for the security countermeasure evaluation device according to the second embodiment. [Figure 16] This flowchart shows an example of a processing procedure for generating evaluation results in Example 1 of the second embodiment. [Figure 17]This flowchart shows an example of a processing procedure for generating evaluation results in Example 2 of the second embodiment. [Figure 18] This flowchart shows an example of a processing procedure for generating evaluation results in Example 3 of the second embodiment. [Figure 19] The hardware configuration of the information processing device according to each embodiment is shown. [Modes for carrying out the invention]

[0007] This embodiment will be described below with reference to the drawings.

[0008] (First Embodiment) Figure 1 shows an example of a security system according to the first embodiment. The security system in Figure 1 comprises a security evaluation device 100 and an evaluation target device 200. The security evaluation device 100 corresponds to an information processing device according to this embodiment. The security evaluation device 100 evaluates the security measures in the monitored system, such as whether the evaluation target device 200, which is responsible for the security functions in the monitored system, can detect or block attacks. The monitored system is a communication network including one or more devices. The communication network is, for example, a local area network such as a wireless LAN or Ethernet.

[0009] The device under evaluation 200 may be incorporated as part of the monitored system, or it may be incorporated as software into a device (host) in the monitored system. Alternatively, the device under evaluation 200 may exist separately from the monitored system, in which case the device under evaluation 200 may be directly connected to the security measures evaluation device 100 by a communication cable or the like.

[0010] The device under evaluation 200 is a device that plays a role in executing security functions such as detecting or blocking attacks based on communication data acquired from a communication network or host logs. The function of detecting or blocking attacks is, for example, a function of detecting abnormal traffic or a function of blocking abnormal traffic. Specific examples of the device under evaluation 200 include a network - type intrusion detection system (IDS: Intrusion Detection System) capable of detecting attacks, a network - type intrusion prevention system (IPS: Intrusion Prevention System) capable of detecting and blocking attacks, and an attack detection tool (log detection tool) as software capable of detecting attacks.

[0011] Figure 2 shows an example configuration of a monitored system incorporating the device under evaluation 200 and the security measures evaluation device 100. The monitored system is configured as a communication network. In this example, the device under evaluation 200 is a network-type IDS. The system in Figure 2 is configured as a local area network. In addition to the device under evaluation 200 and the security measures evaluation device 100, two other devices (hosts) are placed in the communication network: one is a gateway (GW) 11 and the other is an arbitrary device 12. Device 12 can be a PC (Personal Computer), industrial equipment, or other device. For reference, the figure shows a device (attacker device) 31 located outside the network that can be operated by a malicious party to attack the communication network in Figure 2. Consider the case where the attacker device 31 sends communication data into the communication network for the purpose of attacking (attack A1), hijacks the gateway 11, and uses the gateway 11 as a stepping stone to attack device 12 (attack A2). If the device under evaluation 200 has the correct security settings (security measures) configured, the device under evaluation 200 can acquire communication data flowing through the communication network during an attack and detect an attack from the attacker device 31 from the acquired communication data. The acquisition of communication data may be performed using packet capture software or the like. In this embodiment, assuming such an attack has occurred, it is possible to evaluate in advance whether the security measures evaluation device 100 can detect or block the attack without actually attacking the communication network (attacking GW11 and device 12). When performing the evaluation, the devices to be monitored (GW11 and device 12 in the example of Figure 2) may or may not actually be located on the communication network.

[0012] Specifically, when assuming an attack based on one or more attack scenarios, the security countermeasure evaluation device 100 generates simulated communication data or simulated logs (simulated trace information described later), transmits it to the evaluation target device 200, and causes the evaluation target device 200 to perform an attack detection operation using the simulated communication data or simulated logs. The present embodiment has one of the major features in the method of generating such simulated communication data or simulated logs (simulated trace information).

[0013] FIG. 3 is a functional block diagram showing a configuration example of the security countermeasure evaluation device 100. The security countermeasure evaluation device 100 includes an analysis unit 110, an attack scenario storage unit 120, an attack trace storage unit 130, a simulated trace generation unit 140, and a simulated trace transmission unit 150.

[0014] The analysis unit 110 acquires an attack scenario 300 (see FIG. 4 described later) and network configuration information 500 (see FIG. 5 described later). The attack scenario 300 may be, for example, generated by another device based on the system configuration information or vulnerability information of the monitoring target system and acquired by the analysis unit 110. Alternatively, the analysis unit 110 may acquire the attack scenario 300 input by the user from an operating device. The attack scenario 300 may be single or plural. The analysis unit 110 may acquire the network configuration information 500 from the user via an operating device, or may acquire it from a device that manages the network configuration information.

[0015] FIG. 4 shows an example of the attack scenario 300. The attack scenario 300 includes a plurality of attack data 310 to 330. The attack data 310 includes an attack order 311 and an attack method 410. The attack method 410 includes information on the attack method, and more specifically, includes attack identification information 312 and one or more parameters 313a, 313b, ···. When the parameters 313a, 313b, ··· are described without particular distinction, they are collectively referred to as parameters 313.

[0016] Other attack data 320, 330, etc. have a similar structure to attack data 310. That is, attack data 320 includes attack sequence 321 and attack method 420. Attack method 420 includes attack identification information 322 and one or more parameters 323a, 323b.... When parameters 323a, 323b... are not specifically distinguished, they are collectively referred to as parameter 323. Attack data 330 includes attack sequence 331 and attack method 430. Attack method 430 includes attack identification information 332 and one or more parameters 333a, 333b.... When parameters 333a, 333b... are not specifically distinguished, they are collectively referred to as parameter 333.

[0017] Attack sequences 311-331 are information used to determine the attack procedure according to attack scenario 300. For example, if attack data 310 simulates attack A1 in Figure 2 and attack data 320 simulates attack A2, then attack sequence 311 represents "1" and attack sequence 321 represents "2".

[0018] Attack identification information 312-332 is information that can identify the content of the attack. This may be identification information that identifies an attack module implemented in a penetration testing tool, or identification information that identifies a penetration testing script, or simply an ID that identifies multiple attack data from one another.

[0019] Parameters 313-333 are information that must be specified to execute an attack, and are parameters specified when performing an attack. Parameters may include all or at least one of the following: IP address, port number, username, password, and type of target. For example, when generating one or more packets as pseudo-communication data (pseudo-traces information described later) to be used when attacking GW11 from attacker device 31 in Figure 2 based on attack data 310, parameters 313-333 specifying the source IP address of device 31 and the destination address of GW11 are used. If the attack is carried out in multiple phases (for example, attacking GW11 in the first phase and device 12 in the next phase), attack data 310-330 may each correspond to one of the multiple phases. In this case, parameters 313-333 are used corresponding to each of the multiple phases. Note that while the example in Figure 4 includes multiple parameters in the attack data, it may contain only one parameter. Furthermore, configurations where the attack data does not include parameters are also possible. For example, instead of parameters, a function (program) that generates pseudo-trace information by rewriting the trace information described later may be stored.

[0020] Figure 5 shows an example of network configuration information 500. As shown in Figure 5, network configuration information 500 includes one or more host information items 510 to 530. Host information items 510 to 530 are information about one or more devices or equipment present in the communication network. For example, in the network shown in Figure 2, GW11, device 12, IDS200, and security countermeasure evaluation device 100 may each be included as host information. The configuration of the communication network can be identified by this collection of host information.

[0021] More specifically, the host information 510 includes a hostname 511 and one or more network parameters 512. The network parameters 512 are information about the network to which the host of the host information 510 is connected. The network parameters 512 include one or more pieces of information, such as the host's IP address 512a. An example of information other than an IP address is a MAC address.

[0022] Other host information 520, 530, etc., have a similar structure to host information 510. That is, host information 520 includes a hostname 521 and one or more network parameters 522. Network parameters 522 include one or more pieces of information such as the host's IP address 522a. Host information 530 includes a hostname 531 and one or more network parameters 532. Network parameters 532 include one or more pieces of information such as the host's IP address 532a.

[0023] The attack scenario storage unit 120 stores the attack scenario 300 and network configuration information 500 acquired by the analysis unit 110.

[0024] The attack scenario storage unit 120 may also use the network configuration information 500 to determine whether the device under evaluation 200 can monitor the attack indicated in the attack data 310, etc., and store the result of the determination. Being able to monitor an attack means that the device under evaluation 200 can acquire communication data flowing through the communication network during the attack, or can acquire logs on its own device. For example, if the device under evaluation 200 is a network-type IDS, and the device under evaluation 200 can acquire communication data of the monitored device flowing through the communication network during the attack, then the device under evaluation 200 can monitor the attack.

[0025] The attack trace memory unit 130 stores multiple attack trace data (attack trace data) corresponding to attack identification information 312 to 332. The attack trace data is, for example, data of attacks actually carried out against the communication network in the past. The former attacks may be experimental attacks. They may also be data of hypothetical attacks created to simulate actual attacks. Attacks against the communication network may be successful or unsuccessful, or both. The devices targeted by the attack trace data may or may not currently exist on the communication network.

[0026] Figure 6 shows an example of attack trace data stored in the attack trace storage unit 130. The attack trace storage unit 130 contains multiple attack trace data 610 to 630.

[0027] The attack trace data 610 includes attack identification information 611 and one or more trace information 612 (612a, 612b, ...). The attack trace data 610 corresponds to the attack data 310 in Figure 4, and the attack identification information 611 contains the same value as the attack identification information 312 in the attack data of Figure 4. The attack scenario 300 may be created after the attack trace data 610 in Figure 6 has been acquired, by referring to the attack trace data 610. The trace information 612 is information generated during the attack, such as communication data flowing through the communication network (communication data if the attack is successful or if it fails), or logs remaining on the host. The communication data is acquired by a capture device or capture software installed on the communication network. The data format of the trace information 612 may be arbitrary, for example, PCAP format or TXT format. The trace information 612 (612a, 612b, ...) corresponds to the parameter 313 (313a, 313b, ...) in Figure 4, and pseudo-traces are generated by rewriting the trace information 612a and 612b with the parameters 313a and 313b, as will be described later.

[0028] Other attack trace data 620, 630, etc., have a similar structure to attack trace data 610. That is, attack trace data 620 includes attack identification information 621 and one or more trace information 622 (622a, 622b, ...). Attack trace data 630 includes attack identification information 631 and one or more trace information 632 (632a, 632b, ...).

[0029] Figure 7 shows an example of trace information 612a among multiple trace information 612 (612a, 612b, ...) in the attack trace data 610 of Figure 6. In this example, trace information 612a is communication data that flowed through the communication network during the attack. More specifically, trace information 612a includes multiple packets 710-730.

[0030] Packet 710 includes time information 711, a header 712, and data 713. Header 712 includes multiple pieces of information such as the source IP address 712a and the destination IP address 712b. Data 713 is the main body of the communication data. Other packets 720 and 730 have a similar structure to packet 710. That is, packet 720 includes time information 721, a header 722, and data 723. Header 722 includes multiple pieces of information such as the source IP address 722a and the destination IP address 722b. Packet 730 includes time information 731, a header 732, and data 733. Header 732 includes multiple pieces of information such as the source IP address 732a and the destination IP address 732b.

[0031] The pseudo-trace generation unit 140 sequentially selects attack data in the attack scenario storage unit 120, and in the attack trace storage unit 130 selects attack trace data corresponding to the selected attack data (attack trace data with matching attack identification information), thereby generating pseudo-trace data. More specifically, for example, if attack data 310 and attack trace data 610 are selected, pseudo-trace information is generated by rewriting the trace information 612a in the attack trace data 610 based on the corresponding parameter 313a of the attack data 310. The pseudo-trace information is obtained by rewriting the trace information 612a to simulate an actual attack. For example, assuming an attack is carried out according to parameter 313a, pseudo-trace information is generated by changing the source IP address and destination IP address in the trace information 612a to simulate the communication data flowing in an actual attack. However, if the attack trace data 610 is traces obtained from an attack on the same device as the attack data 310, the source IP address and source IP address in the trace information 612a will match the parameter 313a.

[0032] Figure 8 shows an example of pseudo-trace information 612a” generated from trace information 612a based on parameter 313a. The time information 711~731 in Figure 7 has been changed to the current time and is now 711”~731” respectively. Also, the headers 712~732 in Figure 7 have been changed based on parameter 313a and are now headers 712”~732” respectively. Note that headers 712”~732” contain the source IP address 712a”~732a” and destination IP address 712b”~732b”, port number, etc. In this example, it is assumed that data 713~733 have the same values ​​as the data in Figure 7 (the data is not rewritten), but it is also possible that data 713~733 may be rewritten from the data in Figure 7.

[0033] Figure 9 shows specific examples of trace information 612a and pseudo-traces information 612a''. The upper part of Figure 9 shows a specific example of trace information 612a, and the lower part shows a specific example of pseudo-traces information 612a''. The time information, source IP address, and destination IP address have been rewritten. The payload (data) has not been rewritten. In the example shown in the figure, packets are sent and received between two devices (for example, the attacking device and the attacked device), so the source IP address and destination IP address are swapped between the first and second packets.

[0034] Similarly, pseudo-trace information is generated by modifying the trace information 612b... in the attack trace data 610 based on the corresponding attack data parameters 313b.... In this way, the pseudo-trace generation unit 140 generates a group of pseudo-trace information corresponding to the trace information 612a, 612b....

[0035] As described above, the pseudo-trace generation unit 140 generates a set of pseudo-trace information for each attack data in an attack scenario, using the corresponding attack trace data.

[0036] The pseudo-trace transmission unit 150 transmits a group of pseudo-trace information corresponding to each target device 200 in order according to the attack sequence 311 to 331. When transmitting pseudo-trace information, one piece of trace information may be transmitted at a time, or multiple pieces of trace information may be transmitted at a time. If one piece of trace information is transmitted at a time, the detection result of the target device 200 can be grasped each time, making evaluation easier. On the other hand, if all the trace information groups for each attack data are transmitted at once, the processing of this device 100 can be completed at the time of transmission, eliminating the need for waiting time.

[0037] When transmitting (reading) simulated trace information to the device under evaluation 200, the device under evaluation 200 may be integrated into a communication network (actual environment) or isolated from the communication network. Even when the device under evaluation 200 is integrated into a communication network, only normal communication data flows through the network, and the simulated trace information is transmitted only to the device under evaluation 200, thus avoiding any impact on other devices in the communication network. Therefore, without actually attacking the target device, it is possible to send simulated trace information simulating an attack on the target device to the device under evaluation 200 and evaluate whether the attack can be detected. If the device under evaluation 200 is isolated from the communication network, evaluation is possible without the effort of actually building a simulated environment. In this case as well, it is naturally possible to evaluate whether the device under evaluation 200 can detect an attack without actually attacking the target device.

[0038] [Processing Procedure] Figure 10 is a flowchart showing an example of the processing procedure of the security countermeasure evaluation device 100 according to this embodiment.

[0039] The analysis unit 110 acquires the attack scenario 300 and network configuration information 500 (step S11 in Figure 10). The attack scenario 300 may be in JSON format or XML format, etc. The attack scenario 300 may include attack methods for each order of attack, without including the order of attacks. The network configuration information 500 may be in JSON format or XML format, etc. The network configuration information 500 may be input by the user via an operating device.

[0040] The attack scenario storage unit 120 stores the attack scenario 300 acquired by the analysis unit 110 (step S12 in Figure 10). At this time, it may also store whether or not the attack indicated by each attack data in the attack scenario is an attack that can be monitored by the evaluation target device 200. An example of determining whether or not it is a monitorable attack is as described above.

[0041] The pseudo-trace generation unit 140 uses the attack trace data from the attack trace storage unit 130 to generate pseudo-trace information corresponding to the corresponding attack data of the attack scenario 300 (step S13 in Figure 10). The pseudo-trace information corresponds to the communication data flowing through the communication network or the logs generated on the host, assuming that the attack was carried out according to the parameters of the attack data.

[0042] The pseudo-trace transmission unit 150 transmits the pseudo-trace information generated in step S13 to the device under evaluation 200 in accordance with the attack sequence of the attack data, causing the device under evaluation 200 to perform a detection operation (step S14 in Figure 10). The device under evaluation 200 obtains a detection result indicating whether an attack is detected or not. The detection result can take various forms, such as being stored as a log in the device under evaluation 200 or being output as an alert. The user may read the log of the detection result from the device under evaluation 200 using an operating device and display it on the screen.

[0043] [Method for generating pseudo-trace information when the trace information is communication data] Figure 11 is a flowchart showing an example of a processing procedure in which the pseudo-trace generation unit 140 generates pseudo-trace information based on attack data 310 included in the attack scenario 300, when the trace information is communication data. The processing in Figure 11 may be performed when the attack indicated by each attack data in the attack scenario is an attack that can be monitored by the evaluation target device 200, and the trace information is communication data. Here, an example of a processing procedure for attack data 310 included in the attack scenario 300 is shown, but the same applies to attack data 320 and attack data 330.

[0044] First, the pseudo-trace generation unit 140 acquires the attack method 410 included in the attack data 310 (step S21 in Figure 11).

[0045] Next, the pseudo-trace generation unit 140 acquires communication data, which is trace information corresponding to the attack identification information 312 included in the attack method 410, from the attack trace storage unit 130 (step S22 in Figure 11).

[0046] Next, the pseudo-trace generation unit 140 converts parameters such as the source IP address and destination IP address of packets included in the communication data (trace information) based on the parameter 313 included in the attack method 410 (step S23 in Figure 11). If the communication data (trace information) includes multiple packets, the parameters such as the source IP address and destination IP address of all packets are converted based on the parameter 313. The conversion may be performed, for example, by creating a correspondence table of IP addresses based on the parameter 313 and the packets included in the communication data, and following the correspondence table.

[0047] Next, the pseudo-trace generation unit 140 shifts the time information of the packets included in the communication data (trace information) to the current time (step S24 in Figure 11). If the communication data (trace information) includes multiple packets, the time information of all packets is shifted to the current time.

[0048] Through the above process, it is possible to generate simulated communication data that mimics the communication data that occurs during an attack, as pseudo-traffic information.

[0049] [Method for generating pseudo-trace information when trace information is in the form of logs] Figure 12 is a flowchart showing an example of a processing procedure for generating pseudo-trace information in the pseudo-trace generation unit 140 based on attack data 310 included in the attack scenario 300, when the trace information is a log. The processing in Figure 12 may also be performed when the attacks in each attack data of the attack scenario are not network-monitored attacks by the evaluation device 200, and the trace information is a log. Here, an example of a processing procedure for attack data 310 included in the attack scenario 300 is shown, but the same applies to attack data 320 and attack data 330.

[0050] First, the pseudo-trace generation unit 140 acquires the attack method 410 included in the attack data 310 (step S51 in Figure 12). Next, the pseudo-trace generation unit 140 obtains a log (host log), which is trace information corresponding to the attack identification information 312 included in the attack method 410, from the attack trace storage unit 130 (step S52 in Figure 12). There may be one log or multiple logs. Also, the log may be a log from when the attack attempt was successful, or a log from when the attack attempt failed.

[0051] Next, the pseudo-trace generation unit 140 converts the parameters included in the host log based on the parameters 313 included in the attack method 410 (step S53 in Figure 10). The parameters to be converted may be host-identifying information such as at least one of the IP address and MAC address.

[0052] Next, the pseudo-trace generation unit 140 shifts the time information contained in the log to the current time (step S54 in Figure 10). If there are multiple logs, the time information of all logs is shifted to the current time.

[0053] Through the above process, it is possible to generate host logs that simulate the host logs generated during an attack, serving as pseudo-traffic information.

[0054] [Example of the First Embodiment] The device under evaluation 200 is a network-type IDS (hereinafter referred to as NIDS) or a network-type IPS (hereinafter referred to as NIPS). Based on the communication data in the event of a successful or unsuccessful attack and the attack scenario, communication data is generated as pseudo-trace information and transmitted to the NIDS or NIPS. In other words, the pseudo-trace generation unit 140 generates pseudo-communication data (pseudo-trace information) based on the attack data included in the attack scenario 300. Then, the pseudo-trace transmission unit 150 causes the device under evaluation 200 (NIDS or NIPS) to read the pseudo-communication data (pseudo-trace information). By manually analyzing the logs or alerts generated by the NIDS or NIPS when the pseudo-communication data is read, it is evaluated whether the attack was detected by the NIDS, or whether the attack was detected and blocked by the NIPS. This makes it possible to evaluate whether the installation location and settings of the NIDS or NIPS, or the attack detection method, are appropriate.

[0055] As described above, according to this embodiment, by generating pseudo-traces information that simulates communication data assuming an attack has occurred, it is possible to evaluate whether the settings of the device under evaluation are appropriate without actually attacking the monitored system (communication network). In other words, pseudo-traces information that would occur if an attack were carried out according to an attack scenario is generated based on attack trace data acquired in advance, and this is loaded into the device under evaluation. This makes it possible to evaluate whether the security measures of the monitored system are functioning properly without actually attacking the monitored system.

[0056] (Second Embodiment) Figure 13 is a block diagram showing the functional configuration of the security countermeasure evaluation device 100A according to the second embodiment. Blocks identical to those in the functional block diagram of Figure 1, which shows the configuration of the security countermeasure evaluation device 100 according to the first embodiment, are denoted by the same reference numerals, and detailed explanations are omitted.

[0057] The security countermeasure evaluation device 100A includes an analysis unit 110, an attack scenario storage unit 120, an attack trace storage unit 130, a pseudo-traces generation unit 140, a pseudo-traces transmission unit 150, a function evaluation unit (evaluation unit) 160, an evaluation result storage unit 170, and an evaluation result output unit 180. The function evaluation unit 160, the evaluation result storage unit 170, and the evaluation result output unit 180 are added to the security countermeasure evaluation device of the first embodiment.

[0058] The function evaluation unit 160 receives output information based on the detection operation results of the device under evaluation 200, evaluates whether the attack was detected or blocked based on the output information, and creates evaluation result data 800. The output information may be logs of an intrusion detection (attack detection) application running in the IDS, or logs of an intrusion blocking (attack blocking) application running in the IPS. Alternatively, if the device under evaluation 200 is configured as attack detection tool software installed on the host, the output information may be logs of the operation of the attack detection tool remaining on the host.

[0059] Figure 14 shows an example of evaluation result data 800. Evaluation result data 800 includes multiple evaluation results 810 to 830. Evaluation results 810 to 830 correspond to attack data 310 to 330 of attack scenario 300 in Figure 4.

[0060] The evaluation result 810 includes attack identification information 312 (see Figure 4), output information 811, and analysis results 812.

[0061] Output information 811 is information output from the device under evaluation 200 that has read the set of pseudo-trace information (e.g., pseudo-trace information 612a, 612b, etc.) transmitted in response to the attack data 310. If the device under evaluation 200 is an IDS or IPS, output information 811 may also be a log of the intrusion detection or intrusion blocking application running in the IDS or IPS. If the device under evaluation 200 is an attack detection tool installed on a host, output information 811 may also be a log of the operation of the attack detection tool on the host. If output information 811 cannot be obtained from the device under evaluation 200, output information 811 does not need to be stored in the evaluation result 810.

[0062] The analysis result 812 is the result of evaluating whether the device under evaluation 200 was able to detect or block the attack, based on the output information 811, and is generated by the function evaluation unit 160. The analysis result 812 may also be data such as a string indicating, for example, that the attack was detected or blocked, or that the attack was not detected or blocked. The function evaluation unit 160 analyzes the output information 811 and includes the analysis result in the evaluation result 810, so that the user can easily understand whether the attack was detected or blocked. If the output information 811 is not obtained from the device under evaluation 200, the function evaluation unit 160 may determine that it is not possible to evaluate the relevant attack data in the attack scenario.

[0063] The evaluation result storage unit 170 stores the evaluation result data 800 created by the function evaluation unit 160.

[0064] The evaluation result output unit 180 outputs the evaluation result data 800 stored in the evaluation result storage unit 170. The evaluation result output unit 180 may output only a portion of the evaluation result data 800. For example, it may output only the evaluation result that indicates that the analysis result 812 could not be detected or blocked, out of the multiple evaluation results included in the evaluation result data 800. The evaluation result data 800 may be output to a display device (e.g., a console) or output (written) to a file. The user can check the contents of the evaluation result data 800 by viewing it on the console screen or by opening the file on a PC (personal computer), etc.

[0065] [Processing Procedure] Figure 15 is a flowchart showing an example of the processing procedure of the security countermeasure evaluation device 100A according to the second embodiment. Steps S11 to S14 are the same as the processing procedure of the security countermeasure evaluation device 100 according to the first embodiment, so a detailed explanation is omitted.

[0066] The functional evaluation unit 160 receives output information from the device under evaluation 200 and analyzes it to generate evaluation result data 800, which includes evaluation results regarding the attack data of the attack scenario 300 (step S15 in Figure 15).

[0067] The evaluation result storage unit 170 stores the evaluation result data 800 generated by the function evaluation unit 160 (step S16 in Figure 15).

[0068] The evaluation result output unit 180 reads the evaluation result data 800 from the evaluation result storage unit 170 and outputs it (step S17 in Figure 15).

[0069] [Example 1 of the second embodiment] If the device under evaluation 200 is a network-type IDS (hereinafter referred to as NIDS), pseudo-traces information is generated based on communication data in the event of a successful or unsuccessful attack, and it is evaluated whether the NIDS detects the attack corresponding to the attack data.

[0070] In this embodiment, the pseudo-trace generation unit 140 generates pseudo-communication data (pseudo-trace information) based on the attack data and attack trace data of the attack scenario 300, and the pseudo-trace transmission unit 150 causes the NIDS to read the pseudo-trace information in the order of the attack. The function evaluation unit 160 obtains a log (output information) of the detection operation results from the NIDS.

[0071] Figure 16 is a flowchart showing an example of a processing procedure for generating an evaluation result 810 in the functional evaluation unit 160 according to Embodiment 1 of the second embodiment. Here, we show an example of a processing procedure for generating an evaluation result 810, but the same procedure applies to evaluation results 820 and 830.

[0072] First, the function evaluation unit 160 determines whether the attack based on the attack data 310 is monitorable by NIDS (step S31 in Figure 16). More specifically, it determines whether the attack identified by the attack identification information 312 is monitorable by NIDS. For example, it determines, based on the parameters of the attack data, whether the subnet where NIDS is located is the same as the subnet of the device (monitored device) identified from the parameters of the attack data. If they are different, it may be determined that monitoring is not possible because NIDS cannot receive packets flowing through the monitored subnet. If monitoring is not possible, the analysis result 812 is set to information indicating that monitoring is not possible (e.g., "monitoring not possible") (step S33 in Figure 16). Here, the function evaluation unit 160 determined that monitoring was not possible and generated the analysis result 812, but it is also possible that the device under evaluation 200 understands that it could not monitor from the output information received from the device under evaluation 200 and generates the analysis result 812.

[0073] If monitoring is possible, the function evaluation unit 160 determines whether the attack was detected by NIDS based on the acquired NIDS logs or alerts (step S32 in Figure 16). If it determines that the attack was detected, the function evaluation unit 160 sets the analysis result 812 to information indicating detection (e.g., "detected") (step S34 in Figure 16). For example, the function evaluation unit 160 may determine that the attack was detected if a specific string is included in the log or alert. The function evaluation unit 160 generates an evaluation result 810 including the analysis result 812, attack identification information 312, and output information 811.

[0074] If the Functional Evaluation Unit 160 determines that the above attack was not detected, it sets the analysis result 812 to information indicating that it was not detected (for example, "not detected") (step S35 in Figure 16). For example, the Functional Evaluation Unit 160 may determine that the above attack was not detected if a specific string is included in the log or alert. Alternatively, the Functional Evaluation Unit 160 may determine that the attack was not detected if no log or alert is obtained. The Functional Evaluation Unit 160 generates an evaluation result 810 which includes the analysis result 812, the attack identification information 312, and the output information 811.

[0075] [Example 2 of the second embodiment] In Example 1 of the second embodiment, it was assumed that the device under evaluation 200 is a network-type IDS, but a network-type IDS can only detect attacks and cannot block them. In this embodiment, we will describe the case where the device under evaluation 200 is a network-type IPS (hereinafter referred to as NIPS) that can detect and block attacks.

[0076] Figure 17 is a flowchart showing an example of a processing procedure for obtaining analysis results 812 from output information 811 and generating evaluation results 810 in the functional evaluation unit 160 according to Embodiment 2. Here, we show an example of a processing procedure for generating evaluation results 810, but the same procedure applies to evaluation results 820 and 830.

[0077] First, the function evaluation unit 160 determines whether the attack based on the attack data 310 is monitorable by NIPS (step S41 in Figure 17). More specifically, it determines whether the attack identified by the attack identification information 312 is monitorable by NIPS. Whether or not it is monitorable can be the same as in the first embodiment described above. If it is not monitorable, the analysis result 812 is set to information indicating that it is not monitorable (for example, "not monitorable") (step S43 in Figure 17).

[0078] If the above attack is monitorable, the function evaluation unit 160 determines whether the attack has been detected based on the acquired NIPS logs or alerts (step S42 in Figure 17). If it determines that the attack has not been detected, the function evaluation unit 160 sets the analysis result 812 to information indicating that the attack was not detected (e.g., "not detected") (step S45 in Figure 17). The function evaluation unit 160 generates an evaluation result 810 which includes the analysis result 812, attack identification information 312, and output information 811.

[0079] If the Functional Evaluation Unit 160 determines that the above attack has been detected, it determines whether the attack was blocked or not based on the NIPS log or alert (step S44 in Figure 17). If it determines that the attack was blocked, the Functional Evaluation Unit 160 sets the analysis result 812 to information indicating that the attack was blocked (e.g., "blocked") (step S46 in Figure 17). If it determines that the attack was not blocked, the Functional Evaluation Unit 160 sets the analysis result 812 to information indicating that the attack was not blocked (e.g., "not blocked") (step S47 in Figure 17). For example, the Functional Evaluation Unit 160 may determine whether the attack was blocked or not if a specific string is included in the log or alert. The Functional Evaluation Unit 160 generates an evaluation result 810 including the analysis result 812, attack identification information 312, and output information 811.

[0080] [Example 3 of the second embodiment] In Example 1 and Example 2 of the second embodiment, the device under evaluation 200 was a network-type IDS or network-type IPS, and communication data was used as pseudo-infrared information. In this embodiment, we will describe a case where the device under evaluation 200 is a log-based attack detection tool (log detection tool), and logs are used as pseudo-infrared information. The attack detection tool can be installed on GW11 or device 12 in a network configuration such as that shown in Figure 2.

[0081] In this embodiment, the pseudo-trace generation unit 140 generates a pseudo-host log based on the attack data of the attack scenario 300 and the attack trace data. The pseudo-trace transmission unit 150 transmits the host log to the device under evaluation 200 in the order of the attacks, causing the device under evaluation 200 (attack detection tool) to read the host log. The function evaluation unit 160 obtains the output information of the attack detection tool from the device under evaluation 200.

[0082] Figure 18 is a flowchart showing an example of a processing procedure for obtaining analysis results 812 from output information 811 and generating evaluation results 810 in the functional evaluation unit 160 according to Embodiment 3 of the second embodiment. Here, an example of the processing procedure for evaluation results 810 is shown, but the same procedure applies to evaluation results 820 and 830.

[0083] First, the function evaluation unit 160 determines whether the attack data 310 can be monitored by the attack detection tool (step S61 in Figure 18). For example, if the target to be monitored, as identified by the parameters, is a host on which the attack detection tool is installed (in the case of an offline attack detection tool), it is determined that the attack against the host can be monitored. If the target to be monitored, as identified by the attack identification information or parameters, is a host on which the attack detection tool is not installed, it may be determined that it cannot be monitored. If it cannot be monitored, the analysis result 812 is set to information indicating that the attack cannot be monitored (for example, "cannot be monitored") (step S63 in Figure 18).

[0084] If the above attack can be monitored, the function evaluation unit 160 determines whether or not an attack has been detected based on the output information of the acquired attack detection tool (step S62 in Figure 18). If it determines that an attack has been detected, the function evaluation unit 160 sets the analysis result 812 to information indicating that the above attack has been detected (for example, "detected") (step S64 in Figure 18). The function evaluation unit 160 generates an evaluation result 810 that includes the analysis result 812, the attack identification information 312, and the output information 811.

[0085] If the function evaluation unit 160 determines that the above attack has not been detected, it sets the analysis result 812 to information indicating that the above attack has not been detected (for example, "not detected") (step S65 in Figure 18). For example, the function evaluation unit 160 may determine whether the above attack has been detected or not if a specific string is included in the output information of the attack detection tool. Whether or not an attack has been detected may be determined manually or according to an arbitrary algorithm. The function evaluation unit 160 generates an evaluation result 810 which includes the analysis result 812, the attack identification information 312, and the output information 811.

[0086] In Examples 1 to 3 of the second embodiment, examples of operation were described for the case where the device under evaluation 200 is a network-type IDS, a network-type IPS, and an attack detection tool, respectively, but other examples are also possible. For example, the device under evaluation 200 may be a host-type IDS, a host-type IPS, a firewall, a WAF (Web Application Firewall), or a personal firewall.

[0087] (Hardware configuration) Figure 19 shows the hardware configuration of the information processing device according to each embodiment. The information processing device consists of a computer device 900. The computer device 900 includes a CPU 901, an input interface 902, a display device 903, a communication device 904, a main memory 905, and an external memory device 906, which are interconnected by a bus 907.

[0088] The CPU (Central Processing Unit) 901 executes an information processing program, which is a computer program, on the main memory 905. An information processing program is a program that implements the aforementioned functional configurations of the information processing device. An information processing program may not be a single program, but rather a combination of multiple programs or scripts. The CPU 901 implements each functional configuration by executing the information processing program.

[0089] The input interface 902 is a circuit for inputting operation signals from input devices such as keyboards, mice, and touch panels to the information processing device. The input interface 902 corresponds to the input section of the information processing device according to each embodiment.

[0090] The display device 903 displays data output from the information processing device. The display device 903 is, for example, an LCD (liquid crystal display), an organic electroluminescent display, a CRT (cathode ray tube), or a PDP (plasma display), but is not limited to these. Data output from the computer device 900 can be displayed on this display device 903. The display device 903 corresponds to the output unit of the information processing device according to each embodiment.

[0091] The communication device 904 is a circuit for the information processing device to communicate with an external device wirelessly or via a wired connection. Data can be input from an external device via the communication device 904. The data input from the external device can be stored in the main memory 905 or the external memory 906.

[0092] The main memory 905 stores information processing programs, data necessary for the execution of information processing programs, and data generated by the execution of information processing programs. Information processing programs are deployed and executed on the main memory 905. The main memory 905 is, for example, RAM, DRAM, or SRAM, but is not limited to these. Each storage unit or database of the information processing apparatus according to each embodiment may be built on the main memory 905.

[0093] The external storage device 906 stores information processing programs, data necessary for executing the information processing programs, and data generated by the execution of the information processing programs. These information processing programs and data are read into the main memory 905 when the information processing programs are executed. The external storage device 906 is, for example, a hard disk, optical disk, flash memory, and magnetic tape, but is not limited to these. Each storage unit or database of the information processing device may be built on the external storage device 906.

[0094] The information processing program may be pre-installed on the computer device 900, or it may be stored on a storage medium such as a CD-ROM. Furthermore, the information processing program may be uploaded to the internet.

[0095] Furthermore, the information processing device may consist of a single computer device 900, or it may be configured as a system consisting of multiple interconnected computer devices 900.

[0096] It should be noted that the present invention is not limited to the embodiments described above, and the components can be modified and implemented in practice without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the embodiments described above. For example, a configuration in which some components are removed from all the components shown in each embodiment is also conceivable. Moreover, components described in different embodiments may be appropriately combined.

[0097] This embodiment can also be configured as follows. [Item 1] A pseudo-trace generation unit generates pseudo-trace information of the second attack based on trace information of the first attack acquired during the first attack on the first device in the communication network and information regarding the attack method of the second attack on the second device. A pseudo-trace transmission unit transmits pseudo-trace information of the second attack to an evaluation target device that detects the attack based on the attack trace information, Equipped with an information processing device. [Item 2] The information relating to the attack method of the second attack includes parameters relating to the second attack, The pseudo-trace generation unit generates pseudo-trace information for the second attack by rewriting the trace information for the first attack based on the parameters. The information processing device described in item 1. [Item 3] The aforementioned second device may be a different device or the same device as the aforementioned first device. The information regarding the traces of the first attack includes information regarding the first device, The parameters include information relating to the second device, The pseudo-trace generation unit generates pseudo-trace information of the second attack by rewriting the information about the first device included in the trace information of the first attack with information about the second device based on the parameters. The information processing device described in item 2. [Item 4] The information regarding the traces of the first attack further includes time information of the first attack, The pseudo-trace generation unit further generates pseudo-trace information for the second attack by overwriting the time information of the first attack included in the trace information of the first attack with the time information of the current time. The information processing apparatus according to claim 3. [Item 5] The information relating to the first device includes the address information of the first device, The parameter includes the address information of the second device as information about the second device, The pseudo-trace generation unit overwrites the address information of the first device with the address information of the second device. An information processing device as described in item 3 or 4. [Item 6] The pseudo-trace generation unit selects an attack scenario from a plurality of attack scenarios including information about the attack method and attack identification information identifying the method of the second attack, and selects attack trace data from a plurality of attack trace data including trace information of the first attack and attack identification information identifying the method of the first attack, which includes attack identification information that matches the attack identification information included in the selected attack scenario. The pseudo-trace generation unit generates pseudo-trace information of the second attack based on the trace information of the first attack included in the selected attack trace data and the information of the attack method included in the selected attack scenario. An information processing device as described in any one of items 1 to 5. [Item 7] The device under evaluation is a device that acquires communication data flowing through the communication network during an attack, and detects or blocks the attack based on the acquired communication data. The trace information of the first attack is communication data that flows through the communication network and is acquired from the communication network at the time of the first attack. An information processing device as described in any one of items 1 to 6. [Item 8] The trace information of the first attack is a log of operations acquired by the first device at the time of the first attack. The device under evaluation is a device that detects the attack based on operation logs acquired by the device targeted by the attack at the time of the attack, as trace information of the attack. An information processing device as described in any one of items 1 to 7. [Item 9] An evaluation unit receives output information from the device under evaluation indicating the result of detecting the second attack based on pseudo-traces of the second attack, and generates an analysis result indicating whether or not the second attack was detected by the device under evaluation based on the output information. An information processing device described in any one of items 1 to 8, which is equipped with the features described in item 1 to 8. [Item 10] The pseudo-trace generation unit selects an attack scenario from a plurality of attack scenarios, each including parameters relating to the second attack and attack identification information identifying the method of the second attack; and selects attack trace data from a plurality of attack trace data, each including trace information of the first attack and attack identification information identifying the method of the first attack, each containing attack identification information that matches the attack identification information included in the selected attack scenario. The pseudo-trace generation unit generates pseudo-trace information for the second attack based on the trace information of the first attack included in the selected attack trace data and the parameters included in the attack scenario. The evaluation unit identifies the attack identification information included in the attack scenario that was the source for generating the pseudo-traces information of the second attack, The evaluation unit generates an evaluation result that includes the analysis result, the output information, and the identified attack identification information, and outputs the evaluation result. The information processing device described in item 9. [Item 11] The trace information of the first attack is the trace information obtained when the attack is successful during the first attack. An information processing device described in any one of items 1 to 10. [Item 12] The trace information of the first attack is the trace information obtained when the attack fails during the first attack. An information processing device described in any one of items 1 to 11. [Item 13] Based on the trace information of the first attack obtained during the first attack on the first device in the communication network, and information regarding the attack method of the second attack on the second device, pseudo-traces information of the second attack are generated. The simulated trace information of the second attack is transmitted to the evaluation target device that detects the attack based on the attack trace information. The information processing method performed by computers. [Explanation of Symbols]

[0098] 11: Gateway (GW) 12:Equipment 31: Attacker's device 100: Security measures evaluation device (information processing device) 100A: Security measures evaluation device (information processing device) 110: Analysis Department 120: Attack Scenario Memory Unit 130: Attack Trace Memory Unit 140: Pseudo-trace generation unit 150: Pseudo-traces transmission unit 160: Functional Evaluation Department (Evaluation Department) 170: Evaluation result storage unit 180: Evaluation result output section 200: Device under evaluation 300: Attack Scenario 310: Attack data 311: Attack Order 312: Attack Identification 313: Parameters 313a: Parameters 313b: Parameters 320: Attack data 321: Attack Order 322: Attack Identification 323: Parameters 323a: Parameters 330: Attack data 331: Attack Order 332: Attack Identification 333: Parameters 333a: Parameters 410: Attack Methods 420: Attack Methods 430: Attack Methods 500: Network Configuration Information 510: Host Information 511: Hostname 512: Network Parameters 512a: IP address 520: Host Information 521: Hostname 522: Network Parameters 522a IP address 530: Host Information 531: Hostname 532: Network parameters 532a: IP address 610: Attack trace data 611: Attack Identification 612:Trace information 612a: Trace information 612a”: Pseudo-trace information 612b:Trace information 620: Attack trace data 621: Attack Identification 630: Attack trace data 631: Attack Identification 710, 710”: Packet 711, 711”: Time information 712, 712”: Header 712a, 712a”: Source IP address 712b, 712b”: Destination IP address 713: Data 720, 720”: Packet 721, 721”: Time information 722, 722”: Header 722a, 722a”: Source IP address 722b, 722b”: Destination IP address 723: Data 730, 730”: Packet 731, 731”: Time information 732”: Header 732a, 732a”: Source IP address 732b, 732b”: Destination IP address 733: Data 800: Evaluation result data 810: Evaluation Results 811: Output Information 812:Analysis results 820: Evaluation Results 830: Evaluation Results 900: Computer equipment 902: Input Interface 903:Display device 904: Communication device 905: Main memory 906: External storage device 907: Bus A1: Attack A2: Attack

Claims

1. A pseudo-trace generation unit generates pseudo-trace information of the second attack based on trace information of the first attack acquired during the first attack on the first device in the communication network and information regarding the attack method of the second attack on the second device. A pseudo-trace transmission unit transmits pseudo-trace information of the second attack to an evaluation target device that detects the attack based on the attack trace information, Equipped with an information processing device.

2. The information relating to the attack method of the second attack includes parameters relating to the second attack, The pseudo-trace generation unit generates pseudo-trace information for the second attack by rewriting the trace information for the first attack based on the parameters. The information processing apparatus according to claim 1.

3. The aforementioned second device may be a different device or the same device as the aforementioned first device. The information regarding the traces of the first attack includes information regarding the first device, The aforementioned parameters include information relating to the second device, The pseudo-trace generation unit generates pseudo-trace information of the second attack by rewriting the information about the first device included in the trace information of the first attack with information about the second device based on the parameters. The information processing apparatus according to claim 2.

4. The information regarding the traces of the first attack further includes time information of the first attack, The pseudo-trace generation unit further generates pseudo-trace information for the second attack by overwriting the time information of the first attack included in the trace information of the first attack with the time information of the current time. The information processing apparatus according to claim 3.

5. The information relating to the first device includes the address information of the first device. The aforementioned parameters include address information of the second device as information relating to the second device, The pseudo-trace generation unit overwrites the address information of the first device with the address information of the second device. The information processing apparatus according to claim 3.

6. The pseudo-trace generation unit selects an attack scenario from a plurality of attack scenarios including information about the attack method and attack identification information identifying the second attack method, and selects attack trace data from a plurality of attack trace data including trace information of the first attack and attack identification information identifying the first attack method, the attack trace data including attack identification information that matches the attack identification information included in the selected attack scenario. The pseudo-trace generation unit generates pseudo-trace information of the second attack based on the trace information of the first attack included in the selected attack trace data and the information of the attack method included in the selected attack scenario. The information processing apparatus according to claim 1.

7. The device under evaluation is a device that acquires communication data flowing through the communication network during an attack, and detects or blocks the attack based on the acquired communication data. The trace information of the first attack is communication data that flows through the communication network and is acquired from the communication network at the time of the first attack. The information processing apparatus according to claim 1.

8. The information regarding the traces of the first attack is a log of operations acquired by the first device at the time of the first attack. The device under evaluation is a device that detects the attack based on operation logs acquired by the device targeted by the attack at the time of the attack, as trace information of the attack. The information processing apparatus according to claim 1.

9. An evaluation unit receives output information from the device under evaluation indicating the result of detecting the second attack based on pseudo-traces of the second attack, and generates an analysis result indicating whether or not the second attack was detected by the device under evaluation based on the output information. The information processing apparatus according to claim 1, comprising:

10. The pseudo-trace generation unit selects an attack scenario from a plurality of attack scenarios, each including parameters relating to the second attack and attack identification information identifying the method of the second attack; and selects attack trace data from a plurality of attack trace data, each including trace information of the first attack and attack identification information identifying the method of the first attack, each containing attack identification information that matches the attack identification information included in the selected attack scenario. The pseudo-trace generation unit generates pseudo-trace information for the second attack based on the trace information of the first attack included in the selected attack trace data and the parameters included in the attack scenario. The evaluation unit identifies the attack identification information included in the attack scenario that was the source for generating the pseudo-traces information of the second attack, The evaluation unit generates an evaluation result that includes the analysis result, the output information, and the identified attack identification information, and outputs the evaluation result. The information processing apparatus according to claim 9.

11. The trace information of the first attack is the trace information obtained when the attack is successful during the first attack. The information processing apparatus according to claim 1.

12. The trace information of the first attack is the trace information obtained when the attack fails during the first attack. The information processing apparatus according to claim 1.

13. Based on the trace information of the first attack obtained during the first attack on the first device in the communication network, and information regarding the attack method of the second attack on the second device, pseudo-traces information of the second attack are generated. The simulated trace information of the second attack is transmitted to the evaluation target device that detects the attack based on the attack trace information. The information processing method performed by computers.

Images