Security test design system, security test design method, and program

Abstract

To provide a security test design system which allows a person with no knowledge about security to easily and efficiently design many test items while prioritizing them, and prevents omission of test items related to essential measures, such as security standards.SOLUTION: A security test design system 200 configured to design a security test for testing vulnerability of a system to be tested includes: a keyword extraction unit 232 which extracts technical item keywords indicating techniques to be used in components in the system 260 to be tested; a test item extraction unit 233 which extracts test items based on the technical item keywords extracted in the keyword extraction unit and a vulnerability test-related list including test items of the security test; and a priority evaluation unit 234 which evaluates degrees of priority for the test items extracted in the test item extraction unit.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a security test design system, a security test design method, and a program. [Background technology]

[0002] In the development of systems and software, as their functions and components become more diverse and complex, the communications connecting them also become more diverse and complex, and the importance of testing to ensure that systems, etc. continue to function normally is increasing. In addition, as everything is connected to a network and systems, etc. are easily accessed from outside, the risk of being attacked from outside targeting vulnerabilities in systems, etc. is also increasing. Therefore, it is required to carry out effective and efficient security testing in order to find and fix vulnerabilities that could lead to a decrease in security due to design errors or defects during the development of systems, etc.

[0003] In response to this, for example, Patent Document 1 discloses a test device and the like that aims to reduce the erroneous detection of vulnerabilities that can be excluded based on requirements and design specifications. This test device is configured to be able to access a database that stores security attack types, test procedure information that is information that defines test procedures corresponding to the security attack types, and success condition information that is information that defines test success conditions corresponding to the test procedures, in association with each other. Then, this test device receives a specification of a security attack type and an attack target, refers to the database, identifies test procedure information that corresponds to the received security attack type, executes a test on the received attack target based on the identified test procedure information, and judges the success or failure of the test based on the success condition information that corresponds to the received security attack type. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2011-086201 A Summary of the Invention [Problem to be solved by the invention]

[0005] However, as the components and communication paths within a system become more diverse and complex, the number of security test items increases exponentially, and the above-mentioned conventional technology creates a huge number of test items, making it impossible to perform security tests effectively and efficiently. In addition, creating security tests to check for vulnerabilities during the development of a system requires high security skills, knowledge of ever-changing attack methods, and knowledge of the specific risks of the system being developed, but there are only a limited number of people with such advanced skills. In addition, depending on the area of ​​the system being developed, required test items may be specified depending on the area in which they are used.

[0006] Therefore, the present invention provides a security test design system, a security test design method, and a program that enable even a person with no security knowledge to easily and effectively design test items by prioritizing a large number of test items, and prevent overlooking test items related to essential measures such as security standards. [Means for solving the problem]

[0007] In order to solve the above problems, there is provided a security test design system that designs a security test to test for the presence or absence of vulnerabilities in a system under test, the security test design system including a keyword extraction unit that extracts technology item keywords that indicate technologies used in components within the system under test, a vulnerability test related list including test items for the security test, a test item extraction unit that extracts test items based on the technology item keywords extracted by the keyword extraction unit, and a priority evaluation unit that evaluates the priority of the test items extracted by the test item extraction unit. This makes it possible to provide a security test design system that extracts test items according to the technology used in the system to be tested and evaluates the priority of the test items, allowing even people with no security knowledge to prioritize a large number of test items and design test items simply and effectively.

[0008] Further, the priority assessment unit may be characterized in assessing the priority based on a risk-based priority and / or a cost-based priority. This makes it possible to evaluate the priority of test items from the perspective of risk and cost, and to design test items suited to various situations.

[0009] The system may further include an input receiving unit that receives input from a user, and an output unit that outputs some or all of the test items evaluated by the priority evaluation unit, wherein the input receiving unit receives as input design information of the system under test including technical item keywords extracted by the keyword extraction unit, and risk assessment results including paths that may be subject to attack from the outside in communication paths connecting the components in the system under test and their priorities, and the output unit outputs the test items together with their priorities. This allows prioritized test items to be output based on design information of the system to be tested and risk assessment results including possible routes of attack from outside and their priorities, so that even people with no security knowledge can easily and effectively design test items by prioritizing a large number of test items.

[0010] Furthermore, the system under test may be a railway system, and the priority evaluation unit may evaluate the priority using at least one of priorities according to the security levels of the components contained in the design information or risk assessment results of the system under test, or priorities according to the test items related to the mandatory measures of the railway security standard TS50701. According to this, when the system to be tested is a system that operates in a specific area such as a railway system, there may be security standards that correspond to that specific area. Even in such a case, it is possible to prevent a person without security knowledge from overlooking test items related to essential measures such as security standards.

[0011] In order to solve the above problem, there is provided a security test design method executed by a computer to design a security test that tests for the presence or absence of vulnerabilities in a test target system, the security test design method extracts technology item keywords that indicate technologies used in components in the test target system, extracts test items based on a vulnerability test related list that includes test items for the security test and the extracted technology item keywords, and evaluates the priority of the extracted test items. According to this, by extracting test items according to the technology used in the system to be tested and evaluating the priority of the test items, a security test design method can be provided that enables even a person with no security knowledge to assign priorities to a large number of test items and design test items simply and effectively.

[0012] In order to solve the above problem, there is provided a program for causing a computer to function as a security test design method for designing a security test that tests for the presence or absence of vulnerabilities in a test target system, the program including a keyword extraction means for extracting technology item keywords that indicate technologies used in components in the test target system, a vulnerability test related list including test items for the security test, a test item extraction means for extracting test items based on the extracted technology item keywords, and a priority evaluation means for evaluating the priority of the extracted test items. This allows for the extraction of test items that correspond to the technology used in the system under test and for the evaluation of the priority of the test items, thereby providing a program that enables even a person with no security knowledge to easily and effectively design test items by prioritizing a large number of test items. Effect of the Invention

[0013] As described above, according to the present invention, it is possible to provide a security test design system, a security test design method, and a program that enable even a person with no security knowledge to easily and effectively design test items by prioritizing a large number of test items, and prevent the omission of test items related to essential measures such as security standards. [Brief description of the drawings]

[0014] [Figure 1] FIG. 1 illustrates a computer system for implementing an embodiment of the present invention. [Diagram 2] FIG. 2 shows an example of the configuration of a security test design system according to an embodiment of the present invention. [Diagram 3] FIG. 3 shows an example of functional blocks of a security test design system according to an embodiment of the present invention. [Figure 4] FIG. 4 shows an example of a flowchart of a process executed in the security test design system according to the embodiment of the present invention. [Diagram 5] FIG. 5 shows an example of a technology item list in a vulnerability test related list group used in a security test design system according to an embodiment of the present invention. [Figure 6] FIG. 6 shows an example of a general test item list in a vulnerability test related list group used in a security test design system according to an embodiment of the present invention. [Figure 7] FIG. 7 shows an example of a test tool list in a vulnerability test related list group used in a security test design system according to an embodiment of the present invention. [Figure 8]FIG. 8 shows an example of a security level countermeasure requirement list in a vulnerability test related list group used in a security test design system according to an embodiment of the present invention. [Figure 9] FIG. 9 shows device blocks of a test target system of a security test design system according to an embodiment of the present invention. [Figure 10] FIG. 10 shows an example of input of design information of a test target system to a security test design system according to an embodiment of the present invention. [Figure 11] FIG. 11 shows an example of input of a risk assessment result for a test target system to a security test design system according to an embodiment of the present invention (part 1). [Figure 12] FIG. 12 shows an example (part 2) of input of a risk assessment result for a test target system to a security test design system according to an embodiment of the present invention. [Figure 13] FIG. 13 shows a method for calculating the risk value recorded in the risk assessment result for the system under test. [Figure 14] FIG. 14 shows an example (part 1) of a technology item keyword list output during processing executed in the security test design system according to the embodiment of the present invention. [Figure 15] FIG. 15 shows an example (part 2) of a technology item keyword list output during processing executed in the security test design system according to the embodiment of the present invention. [Figure 16] FIG. 16 shows an example of a method for creating a technology item keyword list that is output during processing executed in the security test design system according to the embodiment of the present invention. [Figure 17] FIG. 17 shows an example (part 1) of a test item list output in the process executed in the security test design system according to the embodiment of the present invention. [Figure 18]FIG. 18 shows an example (part 2) of a test item list output in the process executed in the security test design system according to the embodiment of the present invention. [Figure 19] FIG. 19 shows an example (part 3) of a test item list output in the process executed in the security test design system according to the embodiment of the present invention. [Figure 20] FIG. 20 shows an example (part 4) of a test item list output in the process executed in the security test design system according to the embodiment of the present invention. [Figure 21] FIG. 21 shows an example (part 5) of a test item list output in the process executed in the security test design system according to the embodiment of the present invention. [Figure 22] FIG. 22 shows an example (part 6) of a test item list output in the process executed in the security test design system according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0015] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. First, a computer system 300 for implementing an embodiment of the present disclosure will be described with reference to Fig. 1. The mechanisms and devices of the various embodiments disclosed herein may be applied to any suitable computing system. The main components of the computer system 300 include one or more processors 302, memory 304, a terminal interface unit 312, a storage interface 314, an I / O (input / output) device interface 316, and a network interface 318. These components may be interconnected via a memory bus 306, an I / O bus 308, a bus interface unit 309, and an I / O bus interface unit 310.

[0016] Computer system 300 may include one or more general purpose programmable central processing units (CPUs) 302A and 302B, collectively referred to as processors 302. In some embodiments, computer system 300 may include multiple processors, while in other embodiments, computer system 300 may be a single CPU system. Each processor 302 executes instructions stored in memory 304 and may include an on-board cache.

[0017] In one embodiment, memory 304 may include random access semiconductor memory, storage devices, or storage media (either volatile or non-volatile) for storing data and programs. Memory 304 may store all or a portion of the programs, software modules, and data structures that implement the functions described herein. For example, memory 304 may store security test design application 350. In one embodiment, security test design application 350 may include instructions or descriptions that execute on processor 302 the functions described below.

[0018] In some embodiments, security test design application 350 may be implemented in hardware via semiconductor devices, chips, logic gates, circuits, circuit cards, and / or other physical hardware devices instead of or in addition to a processor-based system. In some embodiments, security test design application 350 may include data other than instructions or descriptions. In some embodiments, cameras, sensors, or other data input devices (not shown) may be provided to communicate directly with bus interface unit 309, processor 302, or other hardware of computer system 300.

[0019] Computer system 300 may include a bus interface unit 309 that provides communication between processor 302, memory 304, display system 324, and I / O bus interface unit 310. I / O bus interface unit 310 may couple to an I / O bus 308 for transferring data to and from various I / O units. I / O bus interface unit 310 may communicate via I / O bus 308 with multiple I / O interface units 312, 314, 316, and 318, also known as I / O processors (IOPs) or I / O adapters (IOAs).

[0020] The display system 324 may include a display controller, a display memory, or both. The display controller may provide video, audio, or both data to the display device 326. The computer system 300 may also include one or more devices, such as sensors, configured to collect data and provide the data to the processor 302. The display system 324 may be connected to a display device 326, such as a stand-alone display screen, a television, a tablet, or a handheld device.

[0021] The I / O interface unit provides the ability to communicate with various storage or I / O devices. For example, the terminal interface unit 312 may be attached to user I / O devices 320, such as user output devices such as a video display, a television with speakers, and user input devices such as a keyboard, a mouse, a keypad, a touchpad, a trackball, buttons, a light pen, or other pointing device. A user may use a user interface to enter input data or instructions to the user I / O devices 320 and the computer system 300, and receive output data from the computer system 300, by manipulating the user input devices. The user interface may be displayed on a display, played through speakers, or printed via a printer, for example, via the user I / O devices 320.

[0022] The storage interface 314 may be attached to one or more disk drives or direct access storage devices 322 (usually magnetic disk drive storage devices, but may also be an array of disk drives or other storage devices configured to appear as a single disk drive). In some embodiments, the direct access storage device 322 may be implemented as any secondary storage device. Contents of the memory 304 may be stored in the direct access storage device 322 and retrieved from the direct access storage device 322 as needed. The I / O device interface 316 may provide an interface to other I / O devices such as printers, fax machines, etc. The network interface 318 may provide a communications path to allow the computer system 300 and other devices to communicate with each other. This communications path may be, for example, a network 330.

[0023] In some embodiments, computer system 300 may be a device that receives requests from other computer systems (clients) without a direct user interface, such as a multi-user mainframe computer system, a single-user system, or a server computer, etc. In other embodiments, computer system 300 may be a desktop computer, a portable computer, a laptop, a tablet computer, a pocket computer, a telephone, a smartphone, or any other suitable electronic device.

[0024] Next, a security test design system 200 according to the present invention will be described with reference to Fig. 2. The security test design system 200 according to the present invention includes a security test design device 210, an operation terminal 250, and a test target system 260, which are connected via a communication network 255. The communication network 255 may include, for example, a local area network (LAN), a wide area network (WAN), a satellite network, a cable network, a Wi-Fi network, or any combination thereof.

[0025] The security test design apparatus 210 is a computing device that executes the method of the present invention for designing a security test for a system under test 260. The security test design apparatus 210 may be, for example, a desktop computer, a server computer, a laptop computer, a tablet computer, a workstation, a mobile terminal, or any other type of computing device, and is not limited to the present invention.

[0026] The security test design device 210 includes a processor 211 for executing instructions stored in a memory 230, an I / O interface 212 for controlling communication between internal and external devices of the security test design device 210, a network interface 213 for controlling communication via a communication network 255, a user I / O interface 214 for receiving input from a user, a storage unit 215 for storing security test design programs, data, various lists related to vulnerability testing, test tools used for security testing, etc., a memory 230 for storing executable programs loaded to execute each function of the security test design method according to an embodiment of the present invention, and a bus 216 for controlling communication between these components.

[0027] The memory 230 of the security test design device 210 includes, for example, an input receiving unit 231 that receives instructions and data from a user as input via the I / O interface 212, a keyword extracting unit 232 that extracts technology item keywords that indicate technologies used in components in the test target system 260 that are received as input or stored in the storage unit 215, a test item extracting unit 233 that extracts test items based on a vulnerability test related list including test items of security tests stored in the storage unit 215 and the extracted technology item keywords, a priority evaluation unit 234 that evaluates the priority of the extracted test items, and an output unit 235 that outputs a part or all of the evaluated test items. The output unit 235 includes outputting the evaluated test items visually / auditorily to the user via the I / O interface 212, outputting the evaluated test items to another computer system via the network interface 213, and outputting the evaluated test items to another area of ​​the memory 230 or the storage unit 215.

[0028] Each function included in the security test design device 210 may be realized by a software module constituting the security test design application 350 in the computer system 300 shown in Fig. 1, or may be an independent dedicated hardware device. The security test design device 210 may be operated using a display device, keyboard, mouse, etc. connected to this device via an I / O interface 212, or may be operated using a remote operation terminal 250 connected via a communication network 255.

[0029] The above functions may be implemented in the same computing environment or in a distributed computing environment. For example, the input receiving unit 231 may exist on the operation terminal 250, the keyword extracting unit 232 and the test item extracting unit 233 may exist on the security test design device 210, and the priority evaluation unit 234 and the output unit 235 may exist on another security test design device (not shown) connected to the communication network 255. Such a security test design system 200 includes an input receiving unit 231 that receives input from a user, a keyword extracting unit 232 that extracts technology item keywords that indicate technologies used in components in a test target system 260, a vulnerability test related list including test items of security tests, a test item extracting unit 233 that extracts test items based on the technology item keywords extracted by the keyword extracting unit 232, a priority evaluation unit 234 that evaluates the priority of the test items extracted by the test item extracting unit 233, and an output unit 235 that outputs a part or all of the test items evaluated by the priority evaluation unit 234. This makes it possible to provide a security test design system 200 that extracts test items according to the technology used in the test target system 260 and evaluates the priority of the test items, allowing even a person with no security knowledge to assign priorities to a large number of test items and design test items simply and effectively.

[0030] The test target system 260 can be accessed via a communication network 255, and is tested using a test tool for security testing stored in the storage unit 215 or the like from the operation terminal 250. The test target system 260 does not need to be connected to the same network as the security test design device 210, and may be present in an independent network. The security test design device 210 is applicable to a computer system for executing various applications as the test target system 260, or an organization or device that uses such a computer system. For example, the test target system 260 may be various systems such as a control system, an IoT system, a factory system, or a railway system. In this specification, for the purpose of explanation, a railway on-board device shown in FIG. 9 to FIG. 13 is used as a specific example.

[0031] The functions included in the security test design device 210 will be described with reference to Fig. 3 to Fig. 22. As described in Fig. 2, each function included in the security test design device 210 has an input receiving unit 231 that receives input from a user, a keyword extracting unit 232 that extracts technology item keywords that indicate technologies used in components in the test target system 260, a test item extracting unit 233 that extracts test items based on a vulnerability test related list including test items of security tests and the extracted technology item keywords, a priority evaluation unit 234 that evaluates the priority of the extracted test items, and an output unit 235 that outputs some or all of the evaluated test items.

[0032] The security test design device 210 receives, in the input receiving section 231, a set of specifications of the test target system 260 to be the target of the security test and a result of a risk assessment previously performed on the test target system 260 from the user. The set of specifications includes elements (devices and networks connecting the devices) constituting the test target system 260, as well as the technologies used in these elements and the required security level, and is described in documents and drawings (for example, a block diagram of the components). The risk assessment result is a result of evaluation of security risks (for example, a path that may be attacked from the outside, elements present on the path, a risk value of the path, etc.) that is created in advance by a person knowledgeable about the test target system 260 and security tests based on the specifications of the test target system 260, and is described in documents and drawings. Specific examples of specifications and risk assessment results will be described later.

[0033] When the input reception unit 231 receives the set of specifications and the risk assessment results, the input reception unit 231 may store the information in the storage unit 215 or may directly pass the information to the keyword extraction unit 232. The keyword extraction unit 232 extracts general-purpose technical items as keywords from the information received from the input reception unit 231 or the storage unit 215. General-purpose technical items are technical terms commonly used in the relevant technical field, such as "UDP" (User Datagram Protocol) and "http" (Hypertext Transfer Protocol). These technical items are stored in advance as a technical item list in the vulnerability test related lists in the storage unit 215. An example of the technical item list is shown in FIG. 5. In the technical item list shown in this figure, a technical item called Serial Interface with a technical item ID of 1, a technical item called CUI with a technical item ID of 2, and a technical item called IP protocol with a technical item ID of 3 are shown.

[0034] When the keyword extraction unit 232 extracts the technology items used in the test target system 260, the list of the technology items is passed to the test item extraction unit 233. When the test item extraction unit 233 receives the extracted technology items, it refers to the technology item list stored in the storage unit 215 and extracts general test items (SID) corresponding to each technology item. For example, in the technology item list shown in Fig. 5, a general test item SID54 is linked to a technology item Serial Interface, so the test item extraction unit 233 extracts SID54 as a general test item corresponding to the technology item Serial Interface.

[0035] The list of general test items is stored in the vulnerability test related lists in the storage unit 215. An example of the general test item list is shown in FIG. 6. In the general test item list shown in this figure, the general test item IDs SID20 and SID54 are exemplified. For example, SID54 is a test of "connecting to the console and obtaining root privileges", and the cost (equipment) required for evaluation is "2. Commercially available (SW tool)", the cost (technology) required for evaluation is "2. Use of existing attack code", and the cost (period) required for evaluation is "120 hours". The general test item list may also include pass / fail criteria. In this way, the general test item list specifies how much cost each test item requires.

[0036] The general test item list specifies equipment parameters, technology parameters, and time parameters as parameters that indicate costs. Equipment parameters include "-1. Free tools, -2. Use of existing attack code, -3. Commercially available (HW tools), -4. Custom-made tools (expensive research equipment)," and the smaller the number, the lower the cost, and the larger the number, the higher the cost. Technology parameters include "-1. Script kiddies, -2. Use of existing attack code, -3. Creation of attack code, -4. Invention of attack method," and the smaller the number, the lower the cost, and the larger the number, the higher the cost. Time parameters include "-1. 8 hours, -2. 120 hours, -3. 240 hours," and the smaller the number, the lower the cost, and the larger the number, the higher the cost.

[0037] Furthermore, when the test item extraction unit 233 receives the extracted technology items, it refers to the technology item list stored in the storage unit 215 and extracts a test tool (TID) corresponding to each technology item. For example, in the technology item list shown in Fig. 5, a test tool T2 is associated with a technology item CUI, so the test item extraction unit 233 extracts T2 as a test tool corresponding to the technology item CUI.

[0038] The list of test tools is stored in the vulnerability test related lists in the storage unit 215. An example of the test tool list is shown in FIG. 7. In the test tool list shown in this figure, the tool IDs T1 to T7 are exemplified. For example, the tool ID T2 has the tool name "Nmap", the tool category "analysis tool", the flag "SW", and the summary "port scanner. Used to investigate open ports and the version of running services. (https: / / nmap.org / )". In this way, the test tool list specifies the test tools to be used for the technology items, as well as their properties and conditions.

[0039] When the test items for testing the test target system 260 are extracted by the test item extraction unit 233, the list of the test items is passed to the priority evaluation unit 234. When the priority evaluation unit 234 receives the extracted test items, it refers to the security level countermeasure requirements (hereinafter referred to as SL countermeasure requirements) stored in the storage unit 215, and extracts the SL countermeasure requirements corresponding to each test item. For example, the SL countermeasure requirements shown in FIG. 8 include information on whether the test is a mandatory item for the test target system 260. In this SL countermeasure requirement list, the countermeasure requirement ID corresponds to 17, and the IEC 62443 countermeasure requirement is "communication integrity", and the TS 50701 Annex B requirement ID is "9. Virtual private networks (VPN)", which is a mandatory test. In this way, the SL countermeasure requirement list specifies what kind of test items are available in a specific area and whether they are mandatory or optional. In addition, the priority evaluation unit 234 evaluates the priority of the test items using not only the SL countermeasure requirements but also various evaluation values ​​such as risks. Details will be described later.

[0040] TS50701 specifies the testing of security functions on the high security level parts of railway systems, such as SG, DMZ, etc. For example, it requires security functions on the periphery of the communication system for each SL1-3 evaluation, and specifies that the priority of security functions on the periphery is to be raised.

[0041] When the priority of the test items is evaluated by the priority evaluation unit 234, the test items with the priorities are passed to the output unit 235. The output unit 235 outputs the received prioritized test items. This output may output all the received test items with priorities, or may output only the test items with high priorities selected by a predetermined threshold without priorities.

[0042] The process of the security test design device 210 will be described in detail with reference to the flowchart in Fig. 4. In Step 1, the input receiving unit 231 receives input of a set of specifications of the test target system 260 and its risk assessment results, and the keyword extracting unit 232 extracts technology item keywords indicating technologies used in the components in the test target system 260, and outputs a technology item keyword list.

[0043] Here, in order to explain a specific example of the set of specifications and the risk assessment results, a railway on-board equipment will be explained as an example of the system under test 260 with reference to Figs. 9 to 13. The railway on-board equipment is a system that receives signals from railway ground equipment and controls calculation of the vehicle speed and position, generation of a brake pattern, brake control when the pattern is exceeded, etc. FIG. 9 is a device block diagram of the on-board equipment. The railway on-board equipment has two paths for receiving signals from the outside: a communication interface from ground equipment (ground communication device, Balise) and a maintenance interface from maintenance equipment. The device directly connected to the communication interface is the BTM, and the devices directly connected to the maintenance interface are the RTM, EVC, and DMI. Therefore, for someone who tries to invade this on-board equipment, the BTM, RTM, EVC, and DMI are the first and most vulnerable to attack.

[0044] Also, for example, since the L2SW and RLU are connected to the BTM, once the BTM is infiltrated, the intrusion may proceed via the routes BTM->L2SW and BTM->RLU, and furthermore, the routes BTM->L2SW->EVC and BTM->RLU->EVC may be routes that may be attacked from the outside. Figure 10 is an example of a specification for on-board equipment, and in this specification, it is clear that the EVC with the device number 1 is an important device that is "the control unit of the on-board equipment. It calculates the verification speed pattern based on information from the wayside equipment and controls the brakes by verifying the speed," and that its security level is SL3. It is also clear that the input / output interfaces of the EVC are the serial interface, SD card, and Ethernet interface, and that in the case of Ethernet, the UDP protocol is the communication protocol.

[0045] An equipment block diagram like Figure 9 shows the type of equipment in the on-board equipment and whether it is connected or not, while a document like Figure 10 shows the type of equipment that makes up the on-board equipment and more detailed information. It is preferable that the specification document includes the type of equipment, target security level, input / output interface, communication protocol, countermeasure technical items, etc., as shown in Figure 10. Note that the countermeasure technical items, in the case of EVC, include "-User authentication (SL3 required), -DoS attack resistance (SL2 required), -Hardware tamper resistance (SL4)," which describe the security measures taken by the designer and are referred to during risk assessment.

[0046] FIG. 11 shows the results of the risk assessment, with the lower part being a block diagram of the on-board device, with additional paths (thick arrows) that may be attacked from the outside in the communication paths connecting the components in the test target system 260, and the upper part being a documented description of these paths. For example, path No. 4 starts at BTM through the communication interface, has an end goal of EVC, and the associated action is "BTM sends data received from Balise to EVC." FIG. 12 also shows the results of the risk assessment, and in addition to the previous figure, includes the details of the threat and the CVSS risk value for each path.

[0047] As shown in Figure 13, the CVSS risk score, which serves as a basic evaluation criterion, is calculated from the impact and ease of attack. The impact is calculated from the perspectives of the impact on confidentiality, integrity, and availability, and each is calculated by quantifying the magnitude of the category of "none, minor, or severe." The ease of attack is calculated from the perspectives of the attack source category, the complexity of the attack conditions, and whether or not authentication is required before the attack, and is calculated by quantifying the magnitude of the category in about three levels. The person who performed the risk assessment evaluates the risk from these perspectives for each path and calculates a risk score. For example, the risk score for path No. 4 is 7.2.

[0048] When the input receiving unit 231 receives the specification of the on-board device including the technical item keywords extracted by the keyword extracting unit 232 shown in Fig. 10, the input receiving unit 231 extracts the device name, function, target security level, input / output interface, communication protocol, and countermeasure technical item for each device number, and stores them as design information in the database of the test target system 260 in the storage unit 215. Also, when the input receiving unit 231 receives the risk assessment result including the route that may be attacked from the outside in the communication route connecting the components in the test target system 260 shown in Fig. 12 and its priority, the input receiving unit 231 extracts the threat, attack starting point, final goal, interface, component device via, and CVSS risk value for each route number, and stores them as risk assessment results in the database of the test target system 260 in the storage unit 215. In this way, by outputting prioritized test items based on the design information of the test target system 260 and the risk assessment result including the route that may be attacked from the outside and its priority, even a person without security knowledge can assign priorities to a large number of test items and design test items simply and effectively.

[0049] The keyword extraction unit 232 refers to the input / output interface column and the communication protocol column of the stored specifications and extracts technical items used in the on-board equipment. For example, in the on-board equipment shown in Fig. 10, the serial interface, SD card, Ethernet interface, GUI, and USB interface are extracted as the input / output interfaces, and in the case of the Ethernet interface, the UDP protocol is extracted as the communication protocol. For each device, for example, in the EVC with device number 1, the serial interface, SD card, and Ethernet interface are extracted as the input / output interfaces, and the UDP protocol is extracted as the communication protocol.

[0050] The keyword extraction unit 232 may also refer to the column of the component equipment passed through in the stored risk assessment result, and based on the equipment on the path that may be attacked from the outside to the on-board equipment, aggregate the technical items used by the equipment for each path, and extract the keywords of the technical items for each path, as shown in Fig. 14. For example, the path No. 2 is a path that passes through the equipment such as RTM → L2SW → EVC, and in this path, when the RTM is the target of attack, Ethernet Interface, Serial (RS-422 / 485), and UDP are extracted as technical item keywords, and when the EVC is the target of attack, Ethernet Interface, Serial (RS-422 / 485), iTron, and UDP are extracted as technical item keywords. As shown in this figure, the keyword extraction unit 232 may include columns of the path No., the component equipment passed through, the equipment to be attacked, the attack interface, the specifications as technical item keywords (interface, protocol, OS (operating system, etc.)), and the security measures in the technical item keywords.

[0051] Furthermore, the keyword extraction unit 232 may extract countermeasure technology items described in the specifications for each device on the path in addition to the keywords of the technology items for each path, as shown in Fig. 15. For example, for path No. 6, Ethernet Interface and Serial Interface are extracted as the used technology items for DMI, and Ethernet Interface, Serial Interface and UDP are extracted as the countermeasure technology items for EVC, and "DoS attack resistance (SL2 required), CRC, Sequence Check, tamper detection (optional)" is extracted as the countermeasure technology items for DMI, and "DoS attack resistance (SL2 required), IDS: Intrusion Detection (SL4)" is extracted as the countermeasure technology items for EVC.

[0052] In this way, when the security test design device 210 is given the on-board equipment specification shown in FIG. 10 and the risk assessment results shown in FIG. 11 and FIG. 12 as input, in Step 1, it outputs a keyword list of technical items used in the on-board equipment shown in FIG. 14 and FIG. 15.

[0053] Next, in Step 2, the test item extraction unit 233 receives the technology item keyword list output by the input reception unit 231, refers to the vulnerability test related lists in the storage unit 215, and outputs a list of test items to be performed on the on-board device of the test target system 260. As shown in the example of FIG. 16, for example, for a technology item Serial Interface in the technology item keyword list, the test item extraction unit 233 searches for Serial Interface from the technology items in the technology item list in the vulnerability test related list. Since the technology item ID for the technology item Serial Interface is 1, it is found that the corresponding general test item (SID) is SID54. Next, for SID54, it searches from the general test item IDs in the general test item list in the vulnerability test related list. Then, it is found that the test SID54 "connect to console and obtain root authority" is performed for the technology item Serial Interface. Similarly, for other technology items in the received technology item keyword list, it extracts the linked general test items in the same manner, and outputs a list of test items to be performed on the test target system 260.

[0054] Next, in Step 3, the priority evaluation unit 234 receives the test item list output by the test item extraction unit 233, and, in some cases, refers to the vulnerability test related lists in the storage unit 215, and outputs a list of test items and test environments that should be performed preferentially on the on-board equipment of the test target system 260. The priority evaluation unit 234 may extract the test tools and test environments required for each general test item (SID) by referring to the test tool list, as shown in Fig. 17. For example, it is found that SID54 requires a console tool such as Tera term.

[0055] Furthermore, the priority evaluation unit 234 may evaluate the priority according to a priority order based on risk. Specifically, the priority evaluation unit 234 may evaluate the priority according to the CVSS risk value for each path obtained from the risk assessment result, as shown in FIG. 18 (the table on the left in this figure). When the priority is evaluated according to the CVSS risk value, the highest priority is given to paths No. 2, 4, and 6, which have the highest CVSS risk value of 7.2 in this table, followed by paths No. 1, 3, and 7 with 4.67, and then path No. 5 with 2.3. Therefore, when outputting only the path with the highest CVSS risk value, the priority evaluation unit 234 may output only the test items related to paths No. 2, 4, and 6.

[0056] Furthermore, in the case of paths with the same priority, the priority evaluation unit 234 may evaluate the priority according to the occurrence frequency (table on the right in this figure). For example, paths No. 2, 4, and 6 have the same CVSS risk value, but the priority may be evaluated according to the occurrence frequency (based on the same idea as the ease of attack in FIG. 13) for each general test item (SID) performed on these paths. For example, in paths No. 2, 4, and 6 with the same CVSS risk value, the occurrence frequency of SID9 is 8 and the occurrence frequency of SID8 is 7, and the priority evaluation unit 234 evaluates the priority of the test item of SID9 as high and the priority of the test item of SID8 as low among paths No. 2, 4, and 6.

[0057] Furthermore, the priority evaluation unit 234 may evaluate the priority of test items according to the security level for each device defined in the specifications, as shown in Fig. 19. For example, when evaluating test items for a device with security level 3 in an on-board device including devices with security levels 0 to 4 in the test target system 260, the test items with security levels 1 to 3 are evaluated as having high priority and the necessity of implementation is increased, while the test items with security level 0 are evaluated as having low priority and the necessity of implementation is decreased, and the test items with security level 4 are evaluated as unnecessary and the necessity of implementation is not determined.

[0058] 20, the priority evaluation unit 234 may evaluate the priority based on the security level obtained from the specifications of the test target system 260 and whether or not it is a mandatory test item obtained from the SL countermeasure requirement list of the vulnerability test related list. For example, if the EVC device has a CRC and Sequence Check function, general test item SID13 corresponds to it, and the test tool related to the general test item SID13 may be linked to output the test item list.

[0059] Furthermore, the priority evaluation unit 234 may output a combination (category allocation) of test items and test tools for each technology item keyword of the path, as shown in Fig. 21. For example, in the case of an EVC device where the technology item keyword is Serial Interface, the category allocation is SID54 for general test items and T30, T31, T32, and T33 for test tools.

[0060] Also, as shown in Fig. 22, the priority evaluation unit 234 may comprehensively evaluate the priority of the test items based on the priority based on the cost, the priority based on the security level, the priority based on whether the test items are essential in a specific area, and the like. For example, for SID8, the evaluation item is "verification of information obtainable from communication" and the test purpose is "verification that IP addresses, processes, and port numbers used in attacks are not output when connecting to the outside", and the cost (equipment) required for evaluation is "1. Free tools", the cost (technology) required for evaluation is "1. Script kiddies", and the cost (period) required for evaluation is "8 hours", so the cost level is determined to be Low. Furthermore, since SID8 is a test item for a test target of security level 2, it is determined to be High in the SL judgment, and is determined to be High in the test items in the specific area of ​​on-board equipment (TS50701 judgment), and a high priority is given overall.

[0061] In addition, for SID54, the evaluation item is "Interface connection" and the test purpose is "Verify that root privileges cannot be obtained after connecting to the console from an external port," with the cost required for the evaluation (equipment) being "2. Free or commercially available (SW tool)," the cost required for the evaluation (technology) being "2. Use of existing attack code," and the cost required for the evaluation (period) being "8 hours." The SL judgment is High, and the TS50701 judgment is also High, but since it is judged to be slightly more costly than SID8, the cost level is Medium and the overall priority is 5, lower than SID8.

[0062] In addition, for SID30, the evaluation item is "Confirm tampering detection" and the test purpose is "Confirm that measures have been taken to prevent tampering of TCP headers and that tampered packets will not be accepted." The cost (equipment) required for the evaluation is "2. Free or commercially available (SW tool)," the cost (technology) required for the evaluation is "2. Use of existing attack code," the cost (period) required for the evaluation is "240 hours," and the SL judgment is Low because it is optional. The TS50701 judgment is High, but the time cost is judged to be much higher than that of SID54, so the cost level is High, and the overall priority is 9, even lower than SID54.

[0063] As described above, the priority evaluation unit 234 may evaluate the priority based on a priority based on risk or a priority based on cost. According to this, the security test design system 200 can evaluate the priority of the test items from the viewpoint of risk and cost, and can design test items according to various situations. In addition, the priority evaluation unit 234 may evaluate the priority based on a priority based on whether or not the test items are mandatory in a specific area. For example, when the test target system 260 is a railway system, the priority evaluation unit 234 may evaluate the priority using at least one of a priority according to the security level of the components included in the design information or risk assessment result of the test target system 260, or a priority according to the test items related to the mandatory measures of the railway security standard TS50701. According to this, when the test target system 260 is a system that operates in a specific area such as a railway system, there may be a security standard corresponding to the specific area, but even in such a case, it is possible to prevent a person without security knowledge from overlooking a test item related to the mandatory measures such as the security standard.

[0064] The above is also a security test design method executed by a computer to design a security test that tests for the presence or absence of vulnerabilities in the test target system 260. This security test design method is a method of extracting technology item keywords indicating technologies used in components in the test target system 260, extracting test items based on a vulnerability test related list including test items of the security test and the extracted technology item keywords, and evaluating the priority of the extracted test items. According to this, by extracting test items according to technologies used in the test target system 260 and evaluating the priority of the test items, it is possible to provide a security test design method that allows even a person without security knowledge to assign priorities to a large number of test items and design test items simply and effectively.

[0065] The above can also be realized by a program for making a computer function as a security test design method for designing a security test for testing the presence or absence of vulnerability in the test target system 260. This program includes a keyword extraction means for extracting technology item keywords indicating technologies used in components in the test target system 260, a test item extraction means for extracting test items based on a vulnerability test related list including test items of a security test and the extracted technology item keywords, and a priority evaluation means for evaluating the priorities of the extracted test items. According to this, by extracting test items according to technologies used in the test target system 260 and evaluating the priorities of the test items, it is possible to provide a program for easily and effectively designing test items by prioritizing a large number of test items, even for a person without security knowledge.

[0066] The present invention is not limited to the illustrated examples, and may be implemented in any configuration that does not deviate from the scope of the claims. That is, the present invention has been particularly illustrated and described mainly with respect to specific embodiments, but those skilled in the art may make various modifications to the above-described embodiments in terms of the number and other details of the configuration without departing from the scope of the technical idea and purpose of the present invention. [Explanation of symbols]

[0067] 200 Security Test Design System 210 Security Test Design Equipment 231 Input reception section 232 Keyword Extraction Unit 233 Test Item Extraction Department 234 Priority Assessment Unit 235 Output section

Claims

1. A security test design system that designs a security test to test for the presence or absence of vulnerabilities in a test target system, a keyword extraction unit for extracting technology item keywords indicating technologies used in components of the system under test; A vulnerability test related list including test items of security tests and a test item extraction unit that extracts test items based on the technology item keywords extracted by the keyword extraction unit; a priority evaluation unit that evaluates the priority of the test items extracted by the test item extraction unit; A security test design system comprising:

2. 2. The security test design system according to claim 1, wherein the priority evaluation unit evaluates the priority based on risk and / or cost.

3. an input receiving unit that receives an input from a user; an output unit that outputs a part or all of the test items evaluated by the priority evaluation unit; Further equipped with The input receiving unit receives as input design information of a test target system including the technology item keywords extracted by the keyword extracting unit, and a risk assessment result including paths that may be attacked from the outside in a communication path connecting components in the test target system and their priorities; 3. The security test design system according to claim 2, wherein the output unit outputs the test items together with their priorities.

4. The system under test is a railway system. The security test design system according to claim 3, characterized in that the priority evaluation unit evaluates priorities using at least one of the following: priorities according to the security levels of the components contained in the design information or risk assessment results of the system to be tested, or priorities according to the test items related to the mandatory measures of the railway security standard TS50701.

5. 1. A computer-implemented security test design method for designing a security test that tests for the presence or absence of vulnerabilities in a system under test, comprising: Extracting technology item keywords that indicate technologies used in components of the system under test; Extract test items based on the vulnerability test related list including security test items and the extracted technical item keywords. Evaluate the priority of the extracted test items, Security test design methodology.

6. A program for causing a computer to function as a security test design method for designing a security test for testing the presence or absence of vulnerability of a test target system, comprising: A keyword extraction means for extracting technology item keywords indicating technologies used in components in the system under test; A vulnerability test related list including security test items and a test item extraction means for extracting test items based on the extracted technical item keywords; A priority evaluation means for evaluating the priority of the extracted test items; A program for causing a computer to function as a security test design method, comprising:

Images