Evaluation support system, evaluation support method, and program

Abstract

This system provides an evaluation support system that can more effectively assist in the evaluation of the equipment being evaluated. [Solution] The evaluation support system 10 is a system that supports the evaluation of the equipment 40 to be evaluated, and comprises an acquisition unit 11 and an evaluation specification generation unit 12. The acquisition unit 11 acquires threat analysis information d21 showing the results of the analysis of information security threats in the equipment 40 to be evaluated, and vulnerability analysis information d22 showing the results of the analysis of information security vulnerabilities in the equipment 40 to be evaluated. The evaluation specification generation unit 12 generates evaluation specification information d13 including multiple evaluation specifications for the equipment 40 to be evaluated based on the threat analysis information d21 and the vulnerability analysis information d22.

Description

Technical Field

[0001] The present disclosure relates to an evaluation support system for supporting the evaluation of a device to be evaluated, etc.

Background Art

[0002] In recent years, the functions of automobiles have been rapidly evolving. Such functions include, for example, external connection, autonomous driving, automatic control, in-vehicle infotainment (IVI), etc. And due to the evolution of those functions, the integration of electronic control units (ECUs), the development of software-defined vehicles (SDVs), etc., dealing with security risks for automobiles has become more important. As a countermeasure against security risks in in-vehicle product development, threat analysis and vulnerability analysis are performed, and security verification and validity confirmation are required. Note that validity confirmation is also called security evaluation, evaluation, or testing. Compliance with in-vehicle regulations regarding security verification and security evaluation is essential, and security verification and security evaluation are also important for managing software and system risks. In addition, security evaluation is a process for evaluating the overall security state and includes fuzzing tests, vulnerability tests, security function tests, penetration tests, etc.

[0003] For example, Patent Document 1 discloses a security design support system as an evaluation support system. This security design support system is a system that provides efficient support to designers who design the security of information systems, and includes a security design support device. This security design support device creates a threat list that lists security threats that the information system under evaluation may be vulnerable to. Furthermore, the security design support device creates a countermeasure list that lists effective countermeasures against the threats. Furthermore, the security design support device creates a test item list that lists test items that should be performed when the countermeasures are implemented in the information system. Then, based on the threat list, countermeasure list and test item list, the security design support device creates a threat-countermeasure-test item list that associates threats, countermeasures, and test items with each other. [Prior art documents] [Patent Documents]

[0004] [Patent Document 1] Japanese Patent Publication No. 2024-58377 [Overview of the Initiative] [Problems that the invention aims to solve]

[0005] However, the security design support system described in Patent Document 1 has the problem that it does not adequately support the evaluation (i.e., testing) of the equipment being evaluated, which is an information system.

[0006] Therefore, this disclosure provides an evaluation support system and the like that can more effectively support the evaluation of the equipment being evaluated. [Means for solving the problem]

[0007] An evaluation support system according to one aspect of this disclosure is an evaluation support system that supports the evaluation of a device under evaluation, comprising: an acquisition unit that acquires threat analysis information showing the results of an analysis of information security threats to the device under evaluation and vulnerability analysis information showing the results of an analysis of information security vulnerabilities to the device under evaluation; and an evaluation specification generation unit that generates evaluation specification information including a plurality of evaluation specifications for the device under evaluation based on the threat analysis information and the vulnerability analysis information.

[0008] Furthermore, the comprehensive or specific embodiments may be implemented as devices, methods, integrated circuits, computer programs, or recording media such as computer-readable CD-ROMs, or as any combination of devices, methods, integrated circuits, computer programs, and recording media. The recording media may also be non-temporary recording media. [Effects of the Invention]

[0009] The evaluation support system disclosed herein can more effectively support the evaluation of the equipment being evaluated.

[0010] Further advantages and effects of one aspect of this disclosure will be made apparent from the specification and drawings. Such advantages and / or effects are provided by the embodiments and configurations described in the specification and drawings, but not all configurations are necessarily required. [Brief explanation of the drawing]

[0011] [Figure 1] Figure 1 shows an example of the configuration of the development system in the embodiment. [Figure 2] Figure 2 is a diagram illustrating a part of the threat analysis performed by the threat analysis unit in the embodiment. [Figure 3] Figure 3 shows a schematic example of threat analysis information generated and output by the threat analysis unit in the embodiment. [Figure 4]Figure 4 shows a schematic example of vulnerability analysis information generated and output by the vulnerability analysis unit in the embodiment. [Figure 5] Figure 5 shows a schematic example of vulnerability specification information stored in the vulnerability specification database in the embodiment. [Figure 6] Figure 6 shows a schematic example of the countermeasure specification information stored in the countermeasure specification database in the embodiment. [Figure 7] Figure 7 shows a schematic example of attack path specification information stored in the attack path specification database in the embodiment. [Figure 8] Figure 8 shows a schematic example of evaluation tool information stored in the evaluation tool database in the embodiment. [Figure 9] Figure 9 is a diagram illustrating an example of the process by which the evaluation specification generation unit in the embodiment searches for an evaluation specification from the attack path specification information. [Figure 10] Figure 10 is a diagram illustrating an example of the process by which the evaluation specification generation unit in the embodiment searches for an evaluation specification from the countermeasure specification information. [Figure 11] Figure 11 is a diagram illustrating an example of the process by which the evaluation specification generation unit in the embodiment searches for an evaluation specification from vulnerability specification information. [Figure 12] Figure 12 illustrates another example of the process by which the evaluation specification generation unit in the embodiment searches for an evaluation specification from vulnerability specification information. [Figure 13] Figure 13 illustrates yet another example of the process by which the evaluation specification generation unit in the embodiment searches for an evaluation specification from vulnerability specification information. [Figure 14] Figure 14 is a diagram illustrating an example of the process by which the evaluation specification generation unit generates detailed procedures in the embodiment. [Figure 15] Figure 15 shows a schematic example of evaluation specification information generated and output by the evaluation specification generation unit in the embodiment. [Figure 16]FIG. 16 is a diagram showing a schematic example of the evaluated specification information fed back from the evaluation database to the threat analysis unit and the vulnerability analysis unit in the embodiment. [Figure 17] FIG. 17 is a sequence diagram showing an example of the processing operation of the development system in the embodiment. [Figure 18] FIG. 18 is a flowchart showing an example of the processing operation of the evaluation support system in the embodiment. [Figure 19] FIG. 19 is a diagram showing a specific example of the threat analysis information in the embodiment. [Figure 20] FIG. 20 is a diagram showing a specific example of the vulnerability analysis information in the embodiment. [Figure 21] FIG. 21 is a diagram showing a specific example of the attack path specification information in the embodiment. [Figure 22] FIG. 22 is a diagram showing a specific example of the countermeasure specification information in the embodiment. [Figure 23] FIG. 23 is a diagram showing a specific example of the vulnerability specification information in the embodiment. [Figure 24] FIG. 24 is a diagram showing a specific example of the evaluation tool information in the embodiment. [Figure 25] FIG. 25 is a diagram showing a specific example of a part of the evaluated specification information in the embodiment. [Figure 26] FIG. 26 is a diagram showing a specific example of the remaining part of the evaluated specification information in the embodiment.

MODE FOR CARRYING OUT THE INVENTION

[0014] Therefore, the evaluation support system according to the first aspect of this disclosure is a system that supports the evaluation of a device under evaluation, comprising: an acquisition unit that acquires threat analysis information showing the results of an analysis of information security threats to the device under evaluation and vulnerability analysis information showing the results of an analysis of information security vulnerabilities to the device under evaluation; and an evaluation specification generation unit that generates evaluation specification information including a plurality of evaluation specifications for the device under evaluation based on the threat analysis information and the vulnerability analysis information.

[0015] This process generates evaluation specification information containing multiple evaluation specifications for the device under evaluation, based on threat analysis information and vulnerability analysis information. Therefore, since evaluation specification information is generated that corresponds not only to the threat analysis results but also to the vulnerability analysis results of the device under evaluation, it is possible to improve the consistency of the process from threat and vulnerability analysis to the generation of evaluation specification information. Furthermore, it is possible to suppress any omissions in necessary evaluations and increase the likelihood of comprehensively evaluating the device under evaluation. In other words, it is possible to reduce the possibility of an incomplete security evaluation of the device under evaluation. In addition, since the evaluation specification information is generated automatically, the man-hours required to generate that evaluation specification information can be reduced. As a result, it is possible to support the evaluation of the device under evaluation more effectively.

[0016] Furthermore, the evaluation support system according to the second embodiment further includes a database storing security specification information that associates evaluation specifications with each of a plurality of security elements, and the evaluation specification generation unit may search the security specification information for evaluation specifications associated with each of the one or more security elements shown as analysis results in the threat analysis information, and search the security specification information for evaluation specifications associated with each of the one or more security elements shown as analysis results in the vulnerability analysis information, and in generating the evaluation specification information, it may generate evaluation specification information that includes the plurality of evaluation specifications found in the security specification information. Note that the second embodiment may be dependent on the first embodiment.

[0017] This allows the system to retrieve evaluation specifications corresponding to the analysis results for threats and the analysis results for vulnerabilities from the security specification information, and generate evaluation specification information that includes these evaluation specifications. As a result, it is easy to include multiple appropriate evaluation specifications in the evaluation specification information.

[0018] Furthermore, in the evaluation support system according to the third embodiment, the database includes a vulnerability specification database that stores vulnerability specification information included in the security specification information, the vulnerability specification information indicates multiple vulnerabilities as multiple security elements, the vulnerability analysis information indicates one or more vulnerabilities included in the equipment to be evaluated as one or more security elements, the evaluation specification generation unit searches the vulnerability specification information for evaluation specifications associated with each of the one or more vulnerabilities, and in generating the evaluation specification information, the evaluation specification information may be generated that includes one or more evaluation specifications searched from the vulnerability specification information. Note that the third embodiment may be dependent on the second embodiment.

[0019] This allows evaluation specifications corresponding to vulnerabilities in the device under evaluation to be searched from the vulnerability specification information, and evaluation specification information including those specifications is generated. As a result, appropriate evaluation specifications for vulnerabilities in the device under evaluation can be easily included in the evaluation specification information.

[0020] Furthermore, in the evaluation support system according to the fourth embodiment, the database includes an attack path specification database that stores attack path specification information included in the security specification information, the attack path specification information indicates multiple attack paths as multiple security elements, the threat analysis information indicates one or more attack paths in the device under evaluation as one or more security elements, the evaluation specification generation unit searches the attack path specification information for evaluation specifications associated with each of the one or more attack paths, and in generating the evaluation specification information, the evaluation specification information may be generated that includes one or more evaluation specifications searched from the attack path specification information. Note that the fourth embodiment may be dependent on the second or third embodiment.

[0021] This allows the evaluation specifications corresponding to the attack paths in the device under evaluation to be searched from the vulnerability specification information, and evaluation specification information including those evaluation specifications is generated, making it easy to include appropriate evaluation specifications for the attack paths in the device under evaluation in the evaluation specification information.

[0022] Furthermore, in the evaluation support system according to the fifth embodiment, the database includes a countermeasure specification database that stores countermeasure specification information included in the security specification information, the countermeasure specification information indicates multiple countermeasures as multiple security elements, the threat analysis information indicates one or more countermeasures for one or more threats to the device under evaluation as one or more security elements, the evaluation specification generation unit searches the countermeasure specification information for evaluation specifications associated with each of the one or more countermeasures, and in generating the evaluation specification information, the evaluation specification information may be generated that includes one or more evaluation specifications searched from the countermeasure specification information. Note that the fifth embodiment may be subordinate to any one of the second to fourth embodiments.

[0023] This allows evaluation specifications corresponding to countermeasures against threats to the equipment under evaluation to be searched from the countermeasure specification information, and evaluation specification information including those evaluation specifications is generated, making it easy to include appropriate evaluation specifications for countermeasures against threats to the equipment under evaluation in the evaluation specification information.

[0024] Furthermore, the evaluation support system according to the sixth embodiment further includes an evaluation tool database that stores evaluation tool information indicating that an evaluation tool is associated with each of a plurality of conditions, and the evaluation specification generation unit may search the evaluation tool information for an evaluation tool associated with the conditions that the analysis results shown in the threat analysis information satisfy, and in generating the evaluation specification information, it may generate evaluation specification information that includes the evaluation specification using the evaluation tool searched from the evaluation tool information. Note that the sixth embodiment may be subordinate to any one of the first to fifth embodiments.

[0025] This generates evaluation specification information that includes evaluation specifications using evaluation tools tailored to the analysis results of the threat, making it easy to include evaluation specifications that enable proper evaluation in the evaluation specification information.

[0026] Furthermore, in the evaluation support system according to the seventh embodiment, the evaluation specification generation unit may, in generating the evaluation specification information, use the functions and attack paths indicated as analysis results in the threat analysis information to generate the evaluation procedure for the equipment to be evaluated, and may include the generated evaluation procedure in at least one of the plurality of evaluation specifications. Note that the seventh embodiment may be subordinate to any one of the first to sixth embodiments.

[0027] This means that the evaluation procedure is included in the evaluation specification, for example, as a detailed procedure. As a result, evaluators can refer to this detailed procedure, increasing the likelihood that they can perform an appropriate evaluation without hesitation. Furthermore, by generating a detailed procedure for each product being evaluated, it becomes possible to perform an appropriate evaluation for each product.

[0028] Furthermore, in the evaluation support system relating to the eighth aspect, each of the multiple evaluation specifications may include criteria for the evaluation results of the equipment to be evaluated. Note that the eighth aspect may be subordinate to any one of the first to seventh aspects.

[0029] As a result, since the evaluation specifications include criteria, for example, as judgment criteria, when an evaluation is performed according to the evaluation specifications, it is easy to determine whether the evaluation result is OK or NG using those judgment criteria.

[0030] Furthermore, in the evaluation support system according to the ninth embodiment, the evaluation specification generation unit may further determine the priority of each of the multiple evaluation specifications retrieved from the security specification information. The ninth embodiment may be dependent on the second embodiment, or on any one of the third to eighth embodiments that are dependent on the second embodiment.

[0031] This allows evaluators to easily determine, by referring to the priority of each of the multiple evaluation specifications, which evaluations based on which specifications should be performed first and which can be postponed. Therefore, it is possible to prevent important evaluations from being performed first and important evaluations from being postponed.

[0032] Furthermore, in the evaluation support system according to the 10th embodiment, the security specification information may indicate the technical level, evaluation time, and impact of each evaluation specification as numerical values, and the evaluation specification generation unit may, in determining the priority, identify the technical level, evaluation time, and impact corresponding to each of the multiple evaluation specifications retrieved from the security specification information, and determine the priority by performing weighted addition on the identified technical level, evaluation time, and impact. Note that the 10th embodiment may be subordinate to the 9th embodiment.

[0033] This allows the priority of an evaluation specification to be determined by weighting its technical level, evaluation time, and impact. For example, increasing the weight of the technical level allows the priority to be determined primarily from the perspective of the technical level, increasing the weight of the evaluation time allows the priority to be determined primarily from the perspective of the evaluation time, or increasing the weight of the impact allows the priority to be determined primarily from the perspective of the impact (e.g., quality). Furthermore, by adjusting the weight for each product being evaluated, an appropriate priority can be determined for each product.

[0034] Furthermore, the evaluation support system according to the 11th embodiment may further include a feedback unit that feeds back the evaluation specification information, which indicates the evaluation results of the equipment to be evaluated obtained by evaluation according to the evaluation specification information, to the threat analysis unit and the vulnerability analysis unit as evaluated specification information. The threat analysis unit generates the threat analysis information by performing a threat analysis on the equipment to be evaluated, and the vulnerability analysis unit generates the vulnerability analysis information by performing a vulnerability analysis on the equipment to be evaluated. The 11th embodiment may be subordinate to any one of the first to tenth embodiments. The evaluation results may be shown as judgment results such as OK or NG.

[0035] This allows evaluated specification information to be fed back to the threat analysis unit and the vulnerability analysis unit. Therefore, if the evaluated specification information shows a good evaluation result for the evaluation specification, the threat analysis unit can guarantee the results of the threat analysis. For example, if the analysis result is a countermeasure against the threat, the effectiveness of that countermeasure can be guaranteed. On the other hand, if the evaluated specification information shows a bad evaluation result for the evaluation specification, the threat analysis unit can improve the threat analysis. For example, if the analysis result is a countermeasure against the threat, the countermeasure can be improved. As a result, the accuracy of countermeasure planning can be increased. Furthermore, the vulnerability analysis unit can improve the accuracy of vulnerability analysis based on the evaluation results shown in the evaluated specification information. In other words, in the 11th embodiment, not only is there a one-way flow from threat analysis and vulnerability analysis to evaluation, but the evaluation results are also fed back to threat analysis and vulnerability analysis, allowing for effective risk management of the equipment under evaluation.

[0036] Furthermore, the evaluation support method according to the first aspect of this disclosure is a method for supporting the evaluation of a device under evaluation, which acquires threat analysis information showing the results of an analysis of information security threats to the device under evaluation and vulnerability analysis information showing the results of an analysis of information security vulnerabilities to the device under evaluation, and generates evaluation specification information including multiple evaluation specifications for the device under evaluation based on the threat analysis information and the vulnerability analysis information.

[0037] This makes it possible to achieve the same effects and benefits as the evaluation support system according to the first embodiment.

[0038] The embodiments will be described in detail below with reference to the drawings.

[0039] The embodiments described below are all comprehensive or specific examples. The numerical values, shapes, materials, components, arrangement and connection configurations of components, steps, and the order of steps shown in the following embodiments are examples only and are not intended to limit this disclosure. Furthermore, among the components in the following embodiments, those not described in the independent claim representing the highest-level concept will be described as optional components.

[0040] Furthermore, each figure is a schematic diagram and not necessarily a strictly accurate representation. Also, the same component is denoted by the same reference numeral in each figure.

[0041] (Embodiment) Figure 1 shows an example of the configuration of the development system in this embodiment.

[0042] The development system 100 in this embodiment is a system that supports the development of evaluation target equipment (e.g., products or goods), such as an ECU (Electronic Control Unit) installed in a vehicle. The development system 100 may also be a system for complying with ISO (International Organization for Standardization) / SAE (Society of Automotive Engineers) 21434 regulations. This development system 100 includes a threat analysis unit 21, a vulnerability analysis unit 22, an evaluation unit 32, a result determination unit 33, a determination processing unit 34, and an evaluation support system 10.

[0043] The threat analysis unit 21 generates threat analysis information d21 by performing a threat analysis on the device under evaluation. This threat analysis information d21 shows the results of the analysis of information security threats to the device under evaluation. For example, the threat analysis unit 21 identifies attack paths and plans countermeasures against the threats through its threat analysis. The threat analysis unit 21 then generates threat analysis information d21 showing the attack paths and countermeasures. The threat analysis unit 21 may also identify and evaluate threats that the device under evaluation may face. These threats include all elements that could impede the security of the device under evaluation, such as malicious attackers and natural disasters. Note that the attack paths and countermeasures mentioned above are just examples of security elements.

[0044] The vulnerability analysis unit 22 generates and outputs vulnerability analysis information d22 by performing a vulnerability analysis on the device under evaluation. This vulnerability analysis information d22 shows the results of the analysis of information security vulnerabilities in the device under evaluation. For example, the vulnerability analysis unit 22 generates vulnerability analysis information d22 showing the result of determining whether or not the device under evaluation has each vulnerability. In other words, vulnerability analysis information d22 shows the vulnerabilities that the device under evaluation has. The vulnerability analysis unit 22 may also identify and evaluate vulnerabilities present in the device under evaluation. A vulnerability can also be described as a weakness that the device under evaluation cannot defend against attacks. Furthermore, the vulnerability analysis unit 22 may also perform vulnerability analysis to determine the severity of the vulnerabilities. This severity can be expressed as, for example, a CVSS (Common Vulnerability Scoring System) value. The vulnerabilities mentioned above are just examples of security elements.

[0045] The evaluation unit 32 acquires the evaluation specification information d13 generated by the evaluation support system 10 and performs an evaluation (i.e., test) on the device under evaluation according to the evaluation specification information d13. The evaluation unit 32 then outputs information indicating the evaluation result as result information d32. The result information d32 shows, for example, the evaluation result for each evaluation item. The evaluation of the device under evaluation is an evaluation of its security status, such as fuzzing tests, vulnerability tests, security function tests, and penetration tests. The evaluation specification in this embodiment is a specification for performing these evaluations.

[0046] The result determination unit 33 acquires the result information d32 output from the evaluation unit 32 and generates judgment information d33 indicating the judgment result by making a judgment on the evaluation result for each evaluation item shown in the result information d32. The judgment information d33 indicates, for example, whether the evaluation result of the evaluation item is OK or NG. OK indicates that the evaluation result is as expected or in line with the specifications. NG indicates that the evaluation result is not OK, or that the evaluation result is not as expected or in line with the specifications. The judgment result such as OK or NG from the judgment information d33 is also called the test result. The result determination unit 33 then outputs the judgment information d33 to the evaluation support system 10 and the judgment processing unit 34.

[0047] The judgment processing unit 34 acquires the judgment information d33 output from the result judgment unit 33 and outputs an NG report regarding the NG evaluation items and evaluation results indicated in the judgment information d33. The judgment processing unit 34 may also propose improvement suggestions for countermeasures for the NG evaluation items.

[0048] In this embodiment, the evaluation support system 10 is an evaluation support system that assists in the evaluation of the equipment to be evaluated, and generates evaluation specification information d13 based on threat analysis information d21 and vulnerability analysis information d22. The evaluation support system 10 then feeds back the evaluated specification information d14, which is configured by adding the aforementioned judgment information d33 to the evaluation specification information d13, to the threat analysis unit 21 and the vulnerability analysis unit 22.

[0049] Such an evaluation support system 10 includes an acquisition unit 11, an evaluation specification generation unit 12, a vulnerability specification database 13, a countermeasure specification database 14, an attack path specification database 15, an evaluation tool database 16, and an evaluation database 17. Note that databases are also referred to as DBs.

[0050] The acquisition unit 11 acquires threat analysis information d21 from the threat analysis unit 21 and outputs the threat analysis information d21 to the evaluation specification generation unit 12. Furthermore, the acquisition unit 11 acquires vulnerability analysis information d22 from the vulnerability analysis unit 22 and outputs the vulnerability analysis information d22 to the evaluation specification generation unit 12. The acquired threat analysis information d21 shows the analysis results for information security threats in the device under evaluation 40. In other words, the threat analysis information d21 shows one or more security elements such as attack paths and countermeasures as analysis results. The acquired vulnerability analysis information d22 shows the analysis results for information security vulnerabilities in the device under evaluation 40. In other words, the threat analysis information d21 shows one or more security elements such as vulnerabilities as analysis results.

[0051] The evaluation specification generation unit 12 acquires threat analysis information d21 and vulnerability analysis information d22 from the acquisition unit 11 and generates evaluation specification information d13 based on the threat analysis information d21 and vulnerability analysis information d22. At this time, the evaluation specification generation unit 12 generates the evaluation specification information d13 by referring to the information stored in the vulnerability specification database 13, the countermeasure specification database 14, the attack path specification database 15, and the evaluation tool database 16, respectively. The evaluation specification generation unit 12 outputs the generated evaluation specification information d13 to the evaluation unit 32 and the evaluation database 17.

[0052] The vulnerability specification database 13 is a recording medium that stores vulnerability specification information, showing how evaluation specifications are associated with each of multiple vulnerabilities. The countermeasure specification database 14 is a recording medium that stores countermeasure specification information, showing how evaluation specifications are associated with each of multiple countermeasures. Each of these countermeasures is a countermeasure against the aforementioned threats. The attack path specification database 15 is a recording medium that stores attack path specification information, showing how evaluation specifications are associated with each of multiple attack paths. The evaluation tool database 16 is a recording medium that stores evaluation tool information, showing how evaluation tools are associated with each of multiple conditions. Each of the multiple conditions is a condition required for the evaluation specification.

[0053] Furthermore, the vulnerability specification database 13, the mitigation specification database 14, and the attack path specification database 15 can be said to constitute a single database. This database stores security specification information that associates evaluation specifications with each of the multiple security elements. These multiple security elements include the multiple vulnerabilities, multiple mitigation measures, and multiple attack paths mentioned above.

[0054] The evaluation database 17 is a recording medium for storing evaluation specification information d13 and the like. The evaluation database 17 also stores evaluated specification information d14. In other words, the result determination unit 33 stores the determination information d33 in the evaluation database 17. At this time, the result determination unit 33 generates evaluated specification information d14 by adding the determination information d33 to the evaluation specification information d13 already stored in the evaluation database 17. As a result, the evaluated specification information d14 is stored in the evaluation database 17. The evaluated specification information d14 stored in the evaluation database 17 is then fed back to the threat analysis unit 21 and the vulnerability analysis unit 22. In other words, the evaluation database 17 in this embodiment is configured as a feedback unit that feeds back the evaluation specification information d13, which shows the evaluation results of the equipment to be evaluated obtained by evaluation according to the evaluation specification information d13, as evaluated specification information d14 to the threat analysis unit 21 and the vulnerability analysis unit 22.

[0055] In this embodiment, the databases mentioned above include hard disk drives, RAM (Random Access Memory), ROM (Read Only Memory), and semiconductor memory. These databases may be volatile or non-volatile.

[0056] Figure 2 is a diagram illustrating a part of the threat analysis performed by the threat analysis unit 21.

[0057] The threat analysis unit 21 may, for example, evaluate the risk values ​​of each of the assets A, B, and C possessed by the device under evaluation 40, as shown in Figure 2(a). Assets A, B, and C are data or functions that should be protected in the device under evaluation 40. These functions are implemented, for example, by programs. The device under evaluation 40 is configured, for example, as an ECU mounted in a vehicle, and communicates with external devices such as a smartphone 91 and a Diag 92 via wireless or wired connections. The smartphone 91 is a smartphone, and the Diag 92 is a diagnostic device that diagnoses defects and malfunctions in the vehicle. The device under evaluation 40 includes the following physical components: a BT interface 41, a USB interface 42, a CAN interface 43, a Main microcontroller 44, and a CAN microcontroller 45. The BT interface 41 is also referred to as BT I / F and is an interface for Bluetooth®. The USB interface 42 is also referred to as USB I / F and is an interface for USB (Universal Serial Bus). The CAN interface 43, also referred to as CAN I / F, is an interface for CAN (Controller Area Network). The Main microcontroller 44 is a microcomputer that controls the device under evaluation 40. The Main microcontroller 44 has the aforementioned assets A, B, and C. The CAN microcontroller 45 is a microcomputer that controls the CAN of the device under evaluation 40. Note that physical components are hardware components.

[0058] Here, there are physical paths between the Main microcontroller 44 and the BT interface 41, USB interface 42, and CAN microcontroller 45, respectively. There is also a physical path between the BT interface 41 and the smartphone 91. Furthermore, there is a physical path between the CAN microcontroller 45 and the CAN interface 43, and a physical path between the CAN interface 43 and the Diag 92. Note that a physical path is a physical connection path.

[0059] Furthermore, physical components may be assigned a level of attack vulnerability. For example, the BT interface 41 is assigned a Medium level of attack vulnerability, while the USB interface 42 is assigned a Very Low level of attack vulnerability.

[0060] Furthermore, each asset has a level of impact that the equipment 40 being evaluated would experience as a result of an attack on that asset. For example, asset A is set to Moderate as the level of impact, asset B is set to Severe as the level of impact, and asset C is set to Major as the level of impact.

[0061] The threat analysis unit 21 determines the attack paths to each of assets A, B, and C. Specifically, for each of assets A, B, and C, the threat analysis unit 21 determines a physical path consisting of one or more physical components from an external device to the asset. If there are multiple attack paths to an asset, the threat analysis unit 21 selects one attack path from those multiple paths. The attack path is also called an attack path.

[0062] For example, if smartphone 91 attacks assets A, B, and C, smartphone 91 may access the Main microcontroller 44 via the BT interface 41, which is assigned an attackability level of "Medium." Alternatively, smartphone 91 may access the Main microcontroller 44 via the USB interface 42, which is assigned an attackability level of "Very Low." In other words, there are attack paths to assets A, B, and C, one via the BT interface 41 and the other via the USB interface 42. In this case, the threat analysis unit 21 determines the attack path via the physical component assigned the highest attackability level. In the example above, the highest attackability level is Medium. Therefore, the attack path determined by the threat analysis unit 21 is the physical path from smartphone 91 to the Main microcontroller 44 via the BT interface 41. Note that the threat analysis unit 21 may determine all attack paths to the assets, not just the attack path via the physical component assigned the highest attackability level.

[0063] The threat analysis unit 21 evaluates the risk value of an asset by referring to the risk matrix table shown in Figure 2(b) and deriving the risk value of that asset using the level of attack possibility "Medium" and the level of impact of the asset.

[0064] The risk matrix table, as shown in Figure 2(b), displays the risk value corresponding to each combination of attack potential level and impact level. Attack potential levels are distinguished as Very Low, Low, Medium, and High, and are arranged in ascending order. Impact levels are distinguished as Severe, Major, Moderate, and Negligible, and are arranged in descending order.

[0065] In the example above, the impact level of asset A is Moderate, the impact level of asset B is Severe, and the impact level of asset C is Major. Furthermore, the attackability level for the attack paths to these assets is Medium. In other words, the highest level of attackability along the attack path, i.e., the physical path to the assets, is Medium. Therefore, the threat analysis unit 21 derives a risk value of "2" for asset A, a risk value of "4" for asset B, and a risk value of "3" for asset C by referring to the risk matrix table. This evaluates the risk values ​​of assets A, B, and C. Note that the impact level, attackability level, and risk matrix table are defined, for example, by ISO 21434.

[0066] Figure 3 shows a schematic example of threat analysis information d21 generated and output by the threat analysis unit 21.

[0067] The threat analysis information d21 generated by the threat analysis unit 21, as shown in Figure 3, shows, for example, the function ID, function name, asset ID, asset name, threat scenario ID, threat scenario, attack path ID, attack path, OS (Operating System), and countermeasures in association. The function ID is information for identifying the function that the device under evaluation 40 has (i.e., identification information), and the function name is the name of that function. The asset ID is information for identifying the asset associated with that function, and the asset name is the name of that asset. The threat scenario is the scenario of the threat to that asset, and the threat scenario ID is information for identifying that threat scenario. The attack path (i.e., the attack path described above) is the attack path determined for that asset, and the attack path ID is information for identifying that attack path. The OS is the software required to execute that function or asset. The countermeasures are the countermeasures against that attack path and threat scenario. Furthermore, multiple pairs of threat scenario IDs and threat scenarios may be associated with a pair of function IDs and function names, or a pair of asset IDs and asset names. Additionally, one pair of attack path IDs and attack paths may be associated with a pair of threat scenario IDs and threat scenarios, and one OS and one countermeasure may be associated with that pair of attack path IDs and attack paths.

[0068] In a specific example, threat analysis information d21 associates the function ID "ID-a2" and function name "a2" with the asset ID "ID-b6" and asset name "b6". Furthermore, threat analysis information d21 associates the asset ID "ID-b6" and asset name "b6" with the pair of threat scenario ID "ID-c6" and threat scenario "c6", and the pair of threat scenario ID "ID-c2" and threat scenario "c2". Furthermore, threat analysis information d21 associates the pair of threat scenario ID "ID-c6" and threat scenario "c6" with the pair of attack path ID "ID-d1" and attack path "d1", OS "f2", and countermeasure "g2". OS "f2" may be Windows®, Linux®, etc. In this embodiment, the strings of characters consisting of at least one of the letters, numbers, and symbols, such as "g2" included in the information schematically shown, (more specifically, strings other than those containing "ID-") actually represent names or sentences.

[0069] Furthermore, threat analysis information d21 may include the aforementioned impact levels and risk values.

[0070] Figure 4 shows a schematic example of vulnerability analysis information d22 generated and output by the vulnerability analysis unit 22.

[0071] The vulnerability analysis information d22 generated by the vulnerability analysis unit 22, as shown in Figure 4, shows, for example, for each vulnerability ID, the vulnerability ID and the determination result of whether or not the vulnerability identified by that vulnerability ID applies to the device under evaluation 40. The vulnerability ID is information for identifying a vulnerability and may be, for example, a CWE (Common Weakness Enumeration) ID. The determination result indicates "Applicable" if the vulnerability is present in the device under evaluation 40, and "Not Applicable" if the vulnerability is not present in the device under evaluation 40. In other words, the vulnerability determination result indicates whether or not the vulnerability is present in the device under evaluation 40. In a specific example, the vulnerability analysis information d22 shows the vulnerability ID "ID-j1" and the determination result "Applicable" for the vulnerability identified by that vulnerability ID "ID-j1" in association. The vulnerability analysis information d22 may also show the content of the vulnerability.

[0072] Figure 5 shows a schematic example of vulnerability specification information stored in the vulnerability specification database 13.

[0073] The vulnerability specification information 13a stored in the vulnerability specification database 13 shows, for each vulnerability ID, the vulnerability ID, the evaluation specification for the vulnerability identified by that vulnerability ID, and the technical level, evaluation time, and impact of that evaluation specification in association with each vulnerability ID. The technical level is the level of skill required of the user to perform the evaluation according to the evaluation specification for the device under evaluation 40, and is expressed by a number (i.e., an integer) from 1 to 5, for example. The user is, for example, the evaluator who performs the evaluation of the device under evaluation 40. The larger the number, the higher the technical level expressed by that number (i.e., the more difficult the evaluation), and the smaller the number, the lower the technical level expressed by that number (i.e., the easier the evaluation). The evaluation time is the time required to perform the evaluation according to the evaluation specification for the device under evaluation 40, and is expressed by a number (i.e., an integer) from 1 to 5, for example. The larger the number, the longer the evaluation time expressed by that number, and the smaller the number, the shorter the evaluation time expressed by that number. The degree of impact represents the magnitude of the influence that an evaluation conducted according to the evaluation specifications for the equipment being evaluated 40 has on that equipment 40, and is expressed by a number (i.e., an integer) from 1 to 5. The larger the number, the greater the degree of impact represented by that number, and the smaller the number, the smaller the degree of impact represented by that number. It should also be said that the degree of impact represents the magnitude of the influence on the quality of the equipment being evaluated 40.

[0074] Furthermore, the evaluation specification includes a test ID, test items, prerequisites, outline procedure, and judgment criteria. Test items are items of the evaluation specification and are also called evaluation items. The test ID is information used to identify the evaluation specification or evaluation item. Prerequisites are the conditions that are assumed for performing the evaluation of the equipment under evaluation 40. Outline procedure is a general procedure for performing the evaluation of the equipment under evaluation 40. Judgment criteria are the criteria for determining whether the evaluation result of the equipment under evaluation 40 is OK or NG. In other words, these judgment criteria are used in the processing by the result determination unit 33.

[0075] Furthermore, if the vulnerability ID shown in vulnerability specification information 13a is a CWE-ID, and that CWE-ID is, for example, "CWE-327", then the vulnerability identified by "CWE-327" means that the encryption algorithm used is weak. In this case, the evaluation specification associated with "CWE-327" in vulnerability specification information 13a can be said to be, for example, the specification for the vulnerability test described later. Also, if the vulnerability ID shown in vulnerability specification information 13a is a CWE-ID, and that CWE-ID is, for example, "CWE-248", then the vulnerability identified by "CWE-248" means an unhandled exception. If the vulnerability ID shown in vulnerability specification information 13a is a CWE-ID, and that CWE-ID is, for example, "CWE-787", then the vulnerability identified by "CWE-787" means a buffer overflow. The evaluation specifications associated with "CWE-248" or "CWE-787" in vulnerability specification information 13a can also be described as, for example, the specifications for the fuzzing test described later.

[0076] In a specific example, vulnerability specification information 13a shows the vulnerability ID "ID-jx," the evaluation specification "Kax," the technical level "5," the evaluation time "1," and the impact level "1" in association with each other. Furthermore, the evaluation specification "Kax" includes the test ID "ID-(Kax)a," the test item "(Kax)a," the prerequisite "(Kax)b," the outline procedure "(Kax)c," and the judgment criterion "(Kax)d." A more specific example of vulnerability specification information 13a is shown in Figure 23.

[0077] Figure 6 shows a schematic example of the countermeasure specification information stored in the countermeasure specification database 14.

[0078] The countermeasure specification information 14a stored in the countermeasure specification database 14 shows, for each countermeasure, the countermeasure itself, the evaluation specification for that countermeasure, and the technical level, evaluation time, and impact of the evaluation specification in relation to it. It should be noted that each evaluation specification shown in the countermeasure specification information 14a can also be considered, for example, the specification for the security function test described later.

[0079] In a specific example, the countermeasure specification information 14a shows the relationship between the countermeasure "gx", the evaluation specification "Kbx", the technical level "2", the evaluation time "5", and the impact level "3". Furthermore, the evaluation specification "Kbx" includes the test ID "ID-(Kbx)a", the test item "(Kbx)a", the prerequisite "(Kbx)b", the outline procedure "(Kbx)c", and the judgment criterion "(Kbx)d". A more specific example of the countermeasure specification information 14a is shown in Figure 22.

[0080] Figure 7 shows a schematic example of attack path specification information stored in the attack path specification database 15.

[0081] The attack path specification information 15a stored in the attack path specification database 15 shows, for each attack path ID and attack path set, the evaluation specification for that set, and the technical level, evaluation time, and impact of that evaluation specification in association with that set. It should be noted that each evaluation specification shown in the attack path specification information 15a can also be considered, for example, the specification for the penetration test described later.

[0082] In a specific example, attack path specification information 15a shows the pair of attack path ID "ID-dx" and attack path "dx" associated with evaluation specification "Kcx", technical level "3", evaluation time "1", and impact level "4". Furthermore, the evaluation specification "Kcx" includes test ID "ID-(Kcx)a", test item "(Kcx)a", prerequisite "(Kcx)b", outline procedure "(Kcx)c", and judgment criterion "(Kcx)d". A more specific example of attack path specification information 15a is shown in Figure 21.

[0083] Figure 8 shows a schematic example of the evaluation tool information stored in the evaluation tool database 16.

[0084] The evaluation tool information 16a stored in the evaluation tool database 16 shows, for each evaluation tool used to evaluate the equipment under evaluation 40, the tool name of the evaluation tool and the set of conditions under which that evaluation tool is used, in association. The set of conditions consists of the first condition, second condition, third condition, fourth condition, fifth condition, and sixth condition.

[0085] The first condition is a condition regarding the OS required to perform the functions of the device under evaluation 40, indicating one or more types of OS, such as Linux®, for which the evaluation tool can be used. The second condition is a condition regarding the functions (i.e., target functions) of the device under evaluation 40, indicating one or more functions, such as CAN, for which the evaluation tool can be used. The third condition is a condition regarding the licensing type of the functions of the device under evaluation 40, indicating one or more licensing types, such as open source, for which the evaluation tool can be used. The fourth condition is a condition regarding the protocol of the functions of the device under evaluation 40, indicating one or more protocols, such as TCP (Transmission Control Protocol), for which the evaluation tool can be used. The fifth condition is a condition regarding the executable format of the functions of the device under evaluation 40, indicating one or more executable formats, such as GUI (Graphical User Interface), for which the evaluation tool can be used. The sixth condition is a condition regarding the output format of the functions of the device under evaluation 40, indicating one or more output formats, such as XML (Extensible Markup Language), for which the evaluation tool can be used.

[0086] In a specific example, evaluation tool information 16a shows the association between the tool name "tn1" and the first condition "f1", the second condition "a1", the third condition "L1", the fourth condition "P1", the fifth condition "Ex1", and the sixth condition "Ut1". A more specific example of evaluation tool information 16a is shown in Figure 24.

[0087] Figure 9 is a diagram illustrating an example of the process by which the evaluation specification generation unit 12 searches for an evaluation specification from the attack path specification information 15a.

[0088] First, the evaluation specification generation unit 12 searches the attack path specification information 15a for the evaluation specification associated with each set of attack path ID and attack path shown in the threat analysis information d21. Then, the evaluation specification generation unit 12 includes detailed procedures and priorities in the retrieved evaluation specification.

[0089] In other words, the evaluation specification generation unit 12 generates a detailed procedure, which is the procedure for evaluating the equipment 40 under evaluation and provides a detailed description of the outline procedure included in the evaluation specification, and includes the generated detailed procedure in the retrieved evaluation specification. The evaluation specification generation unit 12 also determines the priority of the retrieved evaluation specification. At this time, the evaluation specification generation unit 12 identifies the technical level, evaluation time, and impact associated with the above-mentioned set in the attack path specification information 15a. The evaluation specification generation unit 12 then determines the priority by performing weighted addition on the identified technical level, evaluation time, and impact. For example, the technical level, evaluation time, and impact are represented by variables a, b, and c. In this case, the evaluation specification generation unit 12 determines the priority by performing the operation f(a,b,c)=(p×a)+(q×b)+(r×c), which is a weighted addition function. Note that p, q, and r are weights that satisfy p+q+r=1, respectively.

[0090] Specifically, the evaluation specification generation unit 12 searches the attack path specification information 15a for the evaluation specification "Kc1" associated with the attack path ID "ID-d1" and attack path "d1" shown in the threat analysis information d21. The evaluation specification "Kc1" includes the test ID "ID-(Kc1)a", test item "(Kc1)a", prerequisite "(Kc1)b", summary procedure "(Kc1)c", and judgment criterion "(Kc1)d". The evaluation specification generation unit 12 adds to the evaluation specification "Kc1" a detailed procedure that shows the summary procedure "(Kc1)c" in detail, and the priority of the evaluation specification for test ID "ID-(Kc1)a".

[0091] When the evaluation specification generation unit 12 includes the detailed procedure in the evaluation specification, it first includes the blank detailed procedure in the evaluation specification, and then, in a later process described below, inserts the content of the detailed procedure into that blank space.

[0092] Furthermore, when the evaluation specification generation unit 12 includes priority in the evaluation specification, it performs the aforementioned weighted addition to the technical level "3", evaluation time "2", and impact level "4" associated with the attack path ID "ID-d1" and attack path "d1" shown in the threat analysis information d21. In other words, the evaluation specification generation unit 12 performs the calculation (p×3)+(q×2)+(r×4) by substituting "3" for variable a, "2" for variable b, and "4" for variable c in the weighted addition function f(a,b,c). As a result, the evaluation specification generation unit 12 determines the priority of the evaluation specification for test ID "ID-(Kc1)a".

[0093] Here, the weights p, q, and r are set according to the balance of QCD (Quality, Cost, Delivery). For example, if the evaluator primarily wants to determine priority from the perspective of quality or impact, the impact weight r is set to a value greater than the technical level weight p and the evaluation time weight q. Similarly, if the evaluator primarily wants to determine priority from the perspective of evaluation time, the evaluation time weight q is set to a value greater than the technical level weight p and the impact weight r. Furthermore, if the evaluator primarily wants to determine priority from the perspective of technical level, the technical level weight p is set to a value greater than the evaluation time weight q and the impact weight r. This allows for the appropriate determination of the priority of each evaluation specification and facilitates the planning of the order in which evaluations are carried out according to each evaluation specification.

[0094] Figure 10 is a diagram illustrating an example of the process by which the evaluation specification generation unit 12 searches for an evaluation specification from the countermeasure specification information 14a.

[0095] First, the evaluation specification generation unit 12 searches for the evaluation specification associated with each countermeasure shown in the threat analysis information d21 from the countermeasure specification information 14a. Then, the evaluation specification generation unit 12 includes detailed procedures and priorities in the retrieved evaluation specification.

[0096] In other words, the evaluation specification generation unit 12 generates a detailed procedure, which is the procedure for evaluating the equipment 40 to be evaluated and provides a detailed description of the outline procedure included in the evaluation specification, and includes the generated detailed procedure in the retrieved evaluation specification. The evaluation specification generation unit 12 also determines the priority of the retrieved evaluation specification. At this time, the evaluation specification generation unit 12 identifies the technical level, evaluation time, and impact associated with the countermeasures in the countermeasure specification information 14a. The evaluation specification generation unit 12 then determines the priority by performing weighted addition on the identified technical level, evaluation time, and impact. The weighted addition function f(a,b,c) is as described above.

[0097] Specifically, the evaluation specification generation unit 12 searches for the evaluation specification "Kb2" associated with the countermeasure "g2" shown in the threat analysis information d21 from the countermeasure specification information 14a. The evaluation specification "Kb2" includes the test ID "ID-(Kb2)a", the test item "(Kb2)a", the prerequisite "(Kb2)b", the outline procedure "(Kb2)c", and the judgment criterion "(Kb2)d". The evaluation specification generation unit 12 adds to the evaluation specification "Kb2" a detailed procedure that shows the outline procedure "(Kb2)c" in detail, and the priority of the evaluation specification for the test ID "ID-(Kb2)a".

[0098] When the evaluation specification generation unit 12 includes the detailed procedure in the evaluation specification, it first includes the blank detailed procedure in the evaluation specification, and then, in a later process described below, inserts the content of the detailed procedure into that blank space.

[0099] Furthermore, when the evaluation specification generation unit 12 includes priority in the evaluation specification, it performs the aforementioned weighted addition to the technical level "5", evaluation time "4", and impact level "3" associated with the countermeasure "g2" shown in threat analysis information d21. In other words, the evaluation specification generation unit 12 performs the calculation (p × 5) + (q × 4) + (r × 3) by substituting "5" for variable a, "4" for variable b, and "3" for variable c in the weighted addition function f(a, b, c). As a result, the evaluation specification generation unit 12 determines the priority of the evaluation specification for test ID "ID-(Kb2)a".

[0100] Figure 11 is a diagram illustrating an example of the process by which the evaluation specification generation unit 12 searches for an evaluation specification from the vulnerability specification information 13a.

[0101] First, the evaluation specification generation unit 12 searches for the evaluation specification associated with each vulnerability ID associated with the determination result "applicable" in the vulnerability analysis information d22 from the vulnerability specification information 13a. Then, the evaluation specification generation unit 12 includes detailed procedures and priority in the retrieved evaluation specification.

[0102] In other words, the evaluation specification generation unit 12 generates a detailed procedure, which is the procedure for evaluating the equipment 40 under evaluation and provides a detailed description of the outline procedure included in the evaluation specification, and includes the generated detailed procedure in the retrieved evaluation specification. The evaluation specification generation unit 12 also determines the priority of the retrieved evaluation specification. At this time, the evaluation specification generation unit 12 identifies the technical level, evaluation time, and impact associated with the vulnerability ID in the vulnerability specification information 13a. The evaluation specification generation unit 12 then determines the priority by performing weighted addition on the identified technical level, evaluation time, and impact. The weighted addition function f(a,b,c) is as described above.

[0103] Specifically, the evaluation specification generation unit 12 identifies the vulnerability ID "ID-j1" associated with the judgment result "Applicable" in the vulnerability analysis information d22. Then, the evaluation specification generation unit 12 searches for the evaluation specification "Ka1" associated with that vulnerability ID "ID-j1" from the vulnerability specification information 13a. The evaluation specification "Ka1" includes the test ID "ID-(Ka1)a", the test item "(Ka1)a", the prerequisite "(Ka1)b", the outline procedure "(Ka1)c", and the judgment criterion "(Ka1)d". The evaluation specification generation unit 12 adds to the evaluation specification "Ka1" a detailed procedure that shows the outline procedure "(Ka1)c" in detail, and the priority of the evaluation specification for the test ID "ID-(Ka1)a".

[0104] When the evaluation specification generation unit 12 includes the detailed procedure in the evaluation specification, it first includes the blank detailed procedure in the evaluation specification, and then, in a later process described below, inserts the content of the detailed procedure into that blank space.

[0105] Furthermore, when the evaluation specification generation unit 12 includes priority in the evaluation specification, it performs the aforementioned weighted addition to the technical level "1", evaluation time "2", and impact level "3" associated with the vulnerability ID "ID-j1". Note that the vulnerability ID "ID-j1" is the vulnerability ID associated with the judgment result "Applicable" in the vulnerability analysis information d22. In other words, the evaluation specification generation unit 12 performs the calculation (p×1)+(q×2)+(r×3) by substituting "1" for variable a, "2" for variable b, and "3" for variable c in the weighted addition function f(a,b,c). As a result, the evaluation specification generation unit 12 determines the priority of the evaluation specification for test ID "ID-(Ka1)a".

[0106] Figure 12 illustrates another example of the process by which the evaluation specification generation unit 12 searches for an evaluation specification from the vulnerability specification information 13a.

[0107] Vulnerability analysis information d22 may further indicate, for each vulnerability ID, the location where the vulnerability identified by that vulnerability ID is located, as shown in Figure 12. The location may be, for example, the Main microcontroller 44 shown in Figure 2, or the BT interface 41. In this case, vulnerability specification information 13a shows, for each pair of vulnerability ID and location, the evaluation specification for that vulnerability, and the technical level, evaluation time, and impact of the evaluation specification in relation to that pair.

[0108] Then, the evaluation specification generation unit 12 searches the vulnerability specification information 13a for the evaluation specification associated with each pair of vulnerability IDs and locations associated with the determination result "applicable" in the vulnerability analysis information d22. Then, as described above, the evaluation specification generation unit 12 includes detailed procedures and priorities in the retrieved evaluation specification.

[0109] Specifically, the evaluation specification generation unit 12 identifies the pair of vulnerability ID "ID-j1" and location "Pn3" associated with the judgment result "Applicable" in the vulnerability analysis information d22. Then, the evaluation specification generation unit 12 searches for the evaluation specification "Ka3" associated with that pair from the vulnerability specification information 13a. The evaluation specification "Ka3" includes the test ID "ID-(Ka3)a", test item "(Ka3)a", prerequisite "(Ka3)b", overview procedure "(Ka3)c", and judgment criterion "(Ka3)d". The evaluation specification generation unit 12 adds to the evaluation specification "Ka3" a detailed procedure that shows the overview procedure "(Ka3)c" in detail, and the priority of the evaluation specification for test ID "ID-(Ka3)a".

[0110] Figure 13 illustrates yet another example of the process by which the evaluation specification generation unit 12 searches for an evaluation specification from the vulnerability specification information 13a.

[0111] As shown in Figure 13, the vulnerability specification database 13 may store vulnerability function information 13b for each vulnerability ID, indicating the function associated with the vulnerability identified by that vulnerability ID. In this case, the vulnerability specification information 13a shows, for each pair of vulnerability ID and function, the pair, the evaluation specification for the vulnerability in that pair, and the technical level, evaluation time, and impact of the evaluation specification in association with that pair.

[0112] The evaluation specification generation unit 12 then identifies pairs that include the vulnerability ID associated with the judgment result "Applicable" in the vulnerability analysis information d22 and the function associated with that vulnerability ID in the vulnerability function information 13b. Furthermore, for each pair, the evaluation specification generation unit 12 searches for the evaluation specification associated with that pair from the vulnerability specification information 13a. Then, as described above, the evaluation specification generation unit 12 includes detailed procedures and priority in the searched evaluation specification. In other words, in the example shown in Figure 13, the evaluation specification, technical level, evaluation time, and impact corresponding to the vulnerability ID associated with the judgment result "Applicable" in the vulnerability analysis information d22 are narrowed down by function.

[0113] Specifically, the evaluation specification generation unit 12 identifies a pair in the vulnerability analysis information d22 that includes the vulnerability ID "ID-j1" associated with the judgment result "Applicable" and the function "a2" associated with the vulnerability ID "ID-j1" in the vulnerability function information 13b. The evaluation specification generation unit 12 then searches the vulnerability specification information 13a for the evaluation specification "Ka2" associated with that pair. The evaluation specification "Ka2" includes the test ID "ID-(Ka2)a", the test item "(Ka2)a", the prerequisite "(Ka2)b", the outline procedure "(Ka2)c", and the judgment criterion "(Ka2)d". The evaluation specification generation unit 12 then includes in the evaluation specification "Ka2" a detailed procedure that shows the outline procedure "(Ka2)c" in detail, and the priority of the evaluation specification for the test ID "ID-(Ka2)a".

[0114] Figure 14 is a diagram illustrating an example of the process by which the evaluation specification generation unit 12 generates detailed procedures. In other words, Figure 14 is a diagram illustrating the subsequent process described above.

[0115] The evaluation specification generation unit 12 generates detailed procedures included in the evaluation specification retrieved as described above. In other words, the evaluation specification generation unit 12 generates the contents of the detailed procedures that were left blank in that evaluation specification. At this time, the evaluation specification generation unit 12 identifies the function ID, function name, attack path, and OS corresponding to that evaluation specification.

[0116] For example, as shown in Figures 9 and 10, if the evaluation specification was searched based on threat analysis information d21, the evaluation specification generation unit 12 identifies the attack path used to search for the evaluation specification, as well as the OS, function ID, and function name associated with the attack path or countermeasure used to search for the evaluation specification, from the threat analysis information d21.

[0117] Furthermore, the evaluation specification generation unit 12 obtains the license type, protocol, execution format, and output format of the function having its function ID and function name, for example, in response to user input (i.e., user input). The license type may be, for example, open source or commercial. The protocol may be, for example, TCP, UDP (User Datagram Protocol), HTTP (Hypertext Transfer Protocol). The execution format may be GUI, CLI (Command Line Interface), etc. The output format may be XML, HTML (Hyper Text Markup Language), etc. Note that the OS, the function having the function ID and function name, the license type, the protocol, the execution format, and the output format mentioned above are also called evaluation environment elements.

[0118] Next, the evaluation specification generation unit 12 identifies a tool name from the evaluation tool information 16a that corresponds to the identified OS, function ID, and function name, as well as the acquired license type, protocol, execution format, and output format. In other words, the evaluation specification generation unit 12 identifies a tool name from the evaluation tool information 16a that satisfies the first condition for the identified OS, the second condition for the function having the identified function ID and function name, and the third, fourth, fifth, and sixth conditions for the acquired license type, protocol, execution format, and output format, respectively. For example, if the identified OS is shown in the first condition, then that OS satisfies the first condition. Similarly, if a function having the identified function ID and function name is shown in the second condition, then that function satisfies the second condition. Similarly, if the four evaluation environment elements—license type, protocol, executable format, and output format—are specified in conditions 3, 4, 5, and 6, respectively, then those four evaluation environment elements satisfy conditions 3, 4, 5, and 6, respectively.

[0119] As a result, the evaluation specification generation unit 12 determines an evaluation tool having the specified tool name as the tool to be used in the searched evaluation specification. Furthermore, the evaluation specification generation unit 12 determines a function having the specified function ID and function name as described above as the location where the evaluation will be performed according to that evaluation specification. Furthermore, the evaluation specification generation unit 12 determines an entity included in the specified attack path as the entity that will perform the evaluation according to that evaluation specification. This entity is the entity that carries out the attack according to that attack path. Furthermore, the evaluation specification generation unit 12 determines an outline procedure to be evaluated according to that evaluation specification. Thus, the evaluation specification generation unit 12 generates a detailed procedure in which the determined entity executes the determined outline procedure on the determined function using the determined evaluation tool.

[0120] Specifically, as shown in Figure 9, threat analysis information d21 is used to search for the evaluation specification for test ID "ID-(Kc1)a". In this case, the evaluation specification generation unit 12 identifies the attack path "d1" used to search for the evaluation specification, the OS "f2", the function ID "ID-a2", and the function name "a2" associated with that attack path from the threat analysis information d21.

[0121] Furthermore, the evaluation specification generation unit 12 obtains the license type "L2", protocol "P2", execution format "Ex2", and output format "Ut2" of the function having the function ID "ID-a2" and function name "a2", for example, in response to user input operations.

[0122] Next, the evaluation specification generation unit 12 identifies the tool name "tn2" from the evaluation tool information 16a, which corresponds to the identified OS "f2", function ID "ID-a2", and function name "a2" mentioned above, and the acquired license type "L2", protocol "P2", executable format "Ex2", and output format "Ut2" mentioned above. In other words, the evaluation specification generation unit 12 identifies the tool name "tn2" from the evaluation tool information 16a, which is associated with the first condition indicating the identified OS "f2", the second condition indicating the function having the identified function ID "ID-a2" and function name "a2" (i.e., function "a2"), and the third, fourth, fifth, and sixth conditions indicating the acquired license type "L2", protocol "P2", executable format "Ex2", and output format "Ut2" mentioned above.

[0123] As a result, the evaluation specification generation unit 12 determines an evaluation tool with the identified tool name "tn2" (i.e., evaluation tool "tn2") as the tool to be used in the retrieved evaluation specification. Furthermore, the evaluation specification generation unit 12 determines a function with the identified function ID "ID-a2" and function name "a2" (i.e., function [a2]) as the location where the evaluation according to that evaluation specification will be performed. Furthermore, the evaluation specification generation unit 12 determines subject "A" included in the identified attack path "d1" as the subject that will perform the evaluation according to that evaluation specification. Furthermore, the evaluation specification generation unit 12 determines the outline procedure "(Kc1)c" as the item to be evaluated according to that evaluation specification. Thus, the evaluation specification generation unit 12 generates a detailed procedure in which subject "A" executes the outline procedure "(Kc1)c" on function "a2" using the evaluation tool "tn2". A more specific example of the detailed procedure is shown in Figure 26.

[0124] Furthermore, as shown in Figure 13, if the evaluation specification generation unit 12 has searched for an evaluation specification based on vulnerability analysis information d22, it may identify the function associated with the vulnerability ID corresponding to that evaluation specification from vulnerability function information 13b. The evaluation specification generation unit 12 may then identify the function ID and function name having the identified function, and the attack path and OS associated with that function ID and function name, from threat analysis information d21. The evaluation specification generation unit 12 may then generate the detailed procedure included in the evaluation specification, similar to the example shown in Figure 14. In the example in Figure 14, the function ID and function name from threat analysis information d21 are used to generate the detailed procedure, but asset ID and asset name may also be used. In this case, the evaluation tool information 16a may also indicate the target asset (i.e., asset name) as a second condition.

[0125] The evaluation specification generation unit 12 includes detailed procedures and priorities for each evaluation specification found as described above. The evaluation specification generation unit 12 then generates evaluation specification information d13 by combining multiple evaluation specifications, each including detailed procedures and priorities, and outputs the evaluation specification information d13.

[0126] Figure 15 shows a schematic example of evaluation specification information d13 generated and output by the evaluation specification generation unit 12.

[0127] As shown in Figure 15, the evaluation specification information d13 includes multiple evaluation specifications, each assigned a test number (i.e., test No). These multiple evaluation specifications include a test ID, test items, prerequisites, outline procedure, detailed procedure, decision criteria, technical level, evaluation time, impact, and priority. In the example in Figure 15, the evaluation specification generation unit 12 obtains the technical level, evaluation time, and impact from the attack path specification information 15a, the countermeasure specification information 14a, or the vulnerability specification information 13a and includes them in the evaluation specification. However, the evaluation specification does not necessarily have to include the technical level, evaluation time, and impact.

[0128] Figure 16 shows a schematic example of evaluated specification information d14 that is fed back from the evaluation database 17 to the threat analysis unit 21 and the vulnerability analysis unit 22.

[0129] The evaluated specification information d14 includes the evaluation specification information d13 and the judgment information d33, which consists of multiple test results. Each of the multiple test results indicates whether the evaluation result for the device under evaluation 40, according to the evaluation specifications shown in the evaluation specification information d13, is OK or NG. The test results may also indicate that the evaluation result is conditionally OK. Conditional OK indicates that the evaluation result is OK if the predetermined conditions are met. Alternatively, conditional OK indicates that the evaluation result is problematic from a security standpoint but is correct in terms of the specifications. The test results may also indicate that the evaluation result is NT. NT indicates that evaluation according to the evaluation specifications cannot be performed for reasons such as lack of functionality, i.e., it is not subject to testing.

[0130] Figure 17 is a sequence diagram showing an example of the processing operation of the development system 100.

[0131] The threat analysis unit 21 generates threat analysis information d21 and outputs the threat analysis information d21 to the evaluation specification generation unit 12 via the acquisition unit 11 (step S1). The vulnerability analysis unit 22 generates vulnerability analysis information d22 and outputs the vulnerability analysis information d22 to the evaluation specification generation unit 12 via the acquisition unit 11 (step S2).

[0132] The evaluation specification generation unit 12 transmits the attack path ID and attack path pair contained in the threat analysis information d21 to the attack path specification database 15 (step S3). Then, the evaluation specification generation unit 12 retrieves the evaluation specification associated with that pair from the attack path specification database 15 (step S4).

[0133] Next, the evaluation specification generation unit 12 transmits the countermeasures included in the threat analysis information d21 to the countermeasure specification database 14 (step S5). Then, the evaluation specification generation unit 12 retrieves the evaluation specifications associated with those countermeasures from the countermeasure specification database 14 (step S6).

[0134] Next, the evaluation specification generation unit 12 sends the vulnerability ID contained in the vulnerability analysis information d22 to the vulnerability specification database 13 (step S7). Then, the evaluation specification generation unit 12 retrieves the evaluation specification associated with that vulnerability ID from the vulnerability specification database 13 (step S8).

[0135] Next, the evaluation specification generation unit 12 sends information indicating multiple evaluation environment elements to the evaluation tool database 16 (step S9). Then, the evaluation specification generation unit 12 retrieves the tool name associated with that information from the evaluation tool database 16 (step S10). In other words, the evaluation specification generation unit 12 identifies the evaluation tool that has the tool name. The multiple evaluation environment elements are the OS, the function with the function ID and function name, the license type, the protocol, the executable format, and the output format as described above.

[0136] In steps S3 to S10, the evaluation specification generation unit 12 sends multiple pieces of information, such as attack paths and countermeasures, to multiple databases and retrieves multiple evaluation specifications and tool names from those databases. However, the evaluation specification generation unit 12 may also search for evaluation specifications or tool names from the information contained in each database.

[0137] The evaluation specification generation unit 12 then generates evaluation specification information d13 based on the acquired evaluation specifications and tool names and outputs it to the evaluation unit 32 (step S11). The evaluation unit 32 evaluates the equipment to be evaluated 40 according to the evaluation specification information d13 and generates result information d32 showing the evaluation result and outputs it to the result determination unit 33 (step S12). The result determination unit 33 makes a determination, such as whether the evaluation result shown in the result information d32 is OK or NG, based on the judgment criteria included in the evaluation specification information d13, and stores the judgment information d33 showing the judgment result in the evaluation database 17 (step S13). As a result, evaluated specification information d14, which includes the evaluation specification information d13 and the judgment information d33, is generated and stored in the evaluation database 17.

[0138] This evaluated specification information d14 is fed back from the evaluation database 17 to the threat analysis unit 21 and the vulnerability analysis unit 22 (steps S14, S15).

[0139] Figure 18 is a flowchart showing an example of the processing operation of the evaluation support system 10.

[0140] First, the acquisition unit 11 of the evaluation support system 10 acquires threat analysis information d21 from the threat analysis unit 21 (step S21), and then acquires vulnerability analysis information d22 from the vulnerability analysis unit 22 (step S22).

[0141] Next, the evaluation specification generation unit 12 executes a loop process using the threat analysis information d21 and vulnerability analysis information d22 acquired by the acquisition unit 11 (step S23). This loop process includes a first loop process, a second loop process, and a third loop process.

[0142] In the first loop processing, the evaluation specification generation unit 12 searches the attack path specification information 15a for each attack path number, using the attack path ID and attack path pair corresponding to that attack path number as the search key (step S23a). The attack path number is the number assigned to each attack path ID and attack path pair shown in the threat analysis information d21. The evaluation specification generation unit 12 then identifies the evaluation specification associated with the search key in the attack path specification information 15a (step S23b). In other words, in steps S23a and S23b, the evaluation specification generation unit 12 searches the attack path specification information 15a for the evaluation specification associated with the search key.

[0143] In the second loop processing, the evaluation specification generation unit 12 searches the countermeasure specification information 14a for each countermeasure number using the countermeasure number as the search key (step S23a). The countermeasure number is the number assigned to each countermeasure shown in the threat analysis information d21. The evaluation specification generation unit 12 then identifies the evaluation specification associated with the search key in the countermeasure specification information 14a (step S23b). In other words, in steps S23a and S23b, the evaluation specification generation unit 12 searches the countermeasure specification information 14a for the evaluation specification associated with the search key.

[0144] In the third loop processing, the evaluation specification generation unit 12 searches the vulnerability specification information 13a for each vulnerability number, using the vulnerability ID corresponding to that vulnerability number as the search key (step S23a). The vulnerability number is the number assigned to each vulnerability ID associated with the judgment result "applicable" in the vulnerability analysis information d22. Then, the evaluation specification generation unit 12 identifies the evaluation specification associated with that search key in the vulnerability specification information 13a (step S23b). In other words, in steps S23a and S23b, the evaluation specification generation unit 12 searches the vulnerability specification information 13a for the evaluation specification associated with the search key.

[0145] Furthermore, the evaluation specification generation unit 12 includes a priority in each evaluation specification retrieved as described above.

[0146] Next, the evaluation specification generation unit 12 identifies an evaluation tool for each of the multiple evaluation specifications found in the loop processing by referring to the evaluation tool information 16a (step S24). Furthermore, the evaluation specification generation unit 12 generates the evaluation procedure using the identified evaluation tool as a detailed procedure and includes it in the evaluation specification (step S25).

[0147] The evaluation specification generation unit 12 then combines the evaluation specifications, including the detailed procedures, and assigns a test number to each evaluation specification to generate and output evaluation specification information d13 (step S26). This evaluation specification information d13 is output to the evaluation unit 32 and used to evaluate the equipment 40 under evaluation, and is also stored in the evaluation database 17. The evaluation result of the equipment 40 under evaluation is then determined by the result determination unit 33. As a result, the determination information d33 indicating the determination result is stored in the evaluation database 17 by the result determination unit 33.

[0148] When the result determination unit 33 stores the determination information d33, the evaluation database 17 stores the evaluation specification information d13, which reflects the determination information d33, as evaluated specification information d14 (step S27). The evaluation database 17 (i.e., the feedback unit) then feeds back the evaluated specification information d14 to the threat analysis unit 21 (step S28), and further feeds back the evaluated specification information d14 to the vulnerability analysis unit 22 (step S29).

[0149] Figure 19 shows a specific example of threat analysis information d21.

[0150] Threat analysis information d21, as shown in Figure 19, indicates, for example, the function name "Wi-Fi(registered trademark)-HAL" and the asset name "Wi-Fi connection password". Threat analysis information d21 also indicates, for example, the threat scenario: "[Wi-Fi connection password] is leaked, compromising the confidentiality of [Wi-Fi connection password] and resulting in a securely negligible impact." Furthermore, threat analysis information d21 indicates, for example, the attack path: "An attacker eavesdrops on [Wi-Fi connection password] in the 'Wi-Fi-HAL function'." This "attacker" is the subject mentioned above and may be included in the detailed procedure. Alternatively, this "attacker" may be transformed into an "evaluator" and included in the detailed procedure. Threat analysis information d21 also indicates, for example, "Linux(registered trademark)" as the OS. Finally, threat analysis information d21 indicates, for example, TCR, HCR, and SCR as countermeasures. TCR is a technically required countermeasure. TCR may include, for example, "establishing access control and read / write procedures for vehicle files and data." HCR is a hardware-required measure. HCR may include, for example, "implementing secure boot from an immutable MaskROM." SCR is a software-required measure. SCR may include, for example, "strengthening the operating system."

[0151] Figure 20 shows a specific example of vulnerability analysis information d22.

[0152] Vulnerability analysis information d22 shows the CWE-ID, category, CVE (Common Vulnerabilities and Exposures)-ID, title, description, and applicability determination result. The CWE-ID is the identification information for the category of the vulnerability. The CVE-ID is the identification information for the vulnerability belonging to the category identified by the CWE-ID. The title is the subject of the vulnerability, and the description is a description of the vulnerability. In this vulnerability analysis information d22, for each CVE-ID, the applicability determination result for the vulnerability identified by that CVE-ID is shown. Therefore, the vulnerability ID shown in vulnerability specification information 13a in Figure 5 may be a CVE-ID instead of a CWE-ID. When the vulnerability ID is a CVE-ID, the vulnerability specification information 13a in Figure 5 can show the evaluation specification in more detail than when the vulnerability ID is a CWE-ID.

[0153] Figure 21 shows a specific example of attack path specification information 15a. Figure 22 shows a specific example of countermeasure specification information 14a. Figure 23 shows a specific example of vulnerability specification information 13a.

[0154] Figure 24 shows a specific example of evaluation tool information 16a. Note that Windows, Linux®, macOS®, Bluetooth, and Wi-Fi are registered trademarks.

[0155] Figures 25 and 26 illustrate specific examples of evaluated specification information d14. Figure 25 shows a specific example of the test number, test ID, test items (i.e., test requirements and test objectives), prerequisites, and outline procedure included in evaluated specification information d14. Figure 26 shows a specific example of the detailed procedure, decision criteria, technical level, evaluation time, impact, priority, and test results included in evaluated specification information d14.

[0156] The evaluated specification information d14 includes multiple evaluation specifications, each assigned a test number, and test results, which are the evaluation results for each of those multiple evaluation specifications. These multiple evaluation specifications include, for example, evaluation specifications for vulnerability testing, penetration testing, fuzzing testing, and security function testing. Vulnerability testing is an evaluation to detect security holes or defects in software or systems. Penetration testing is a test that attempts to infiltrate a system from the perspective of a malicious attacker to discover security weaknesses. Fuzzing testing is a test that sends a large amount of invalid or unexpected input to software to see if it causes a crash or bug. Security function testing is a test that verifies whether the system's security functions (e.g., authentication, access control, encryption, etc.) function correctly. These tests are evaluations of the equipment under evaluation 40.

[0157] As described above, the evaluation support system 10 in this embodiment is a system that supports the evaluation of the equipment 40 to be evaluated, and comprises an acquisition unit 11 and an evaluation specification generation unit 12. The acquisition unit 11 acquires threat analysis information d21, which shows the results of the analysis of information security threats in the equipment 40 to be evaluated, and vulnerability analysis information d22, which shows the results of the analysis of information security vulnerabilities in the equipment 40 to be evaluated. The evaluation specification generation unit 12 generates evaluation specification information d13, which includes multiple evaluation specifications for the equipment 40 to be evaluated, based on the threat analysis information d21 and the vulnerability analysis information d22.

[0158] This generates evaluation specification information d13, which includes multiple evaluation specifications for the device under evaluation 40, based on threat analysis information d21 and vulnerability analysis information d22. Therefore, since evaluation specification information d13 is generated that corresponds not only to the threat analysis results but also to the vulnerability analysis results of the device under evaluation 40, the consistency of the process from threat and vulnerability analysis to the generation of evaluation specification information d13 can be improved. Furthermore, it is possible to suppress any omissions in necessary evaluations and increase the likelihood of comprehensively evaluating the device under evaluation 40. In other words, the possibility of an incomplete security evaluation of the device under evaluation 40 can be reduced. In addition, since evaluation specification information d13 is generated automatically, the man-hours required to generate it can be reduced. As a result, the evaluation of the device under evaluation 40 can be supported more effectively.

[0159] Furthermore, the evaluation support system 10 in this embodiment includes a database that stores security specification information, which associates evaluation specifications with each of a plurality of security elements. The evaluation specification generation unit 12 searches the security specification information for evaluation specifications associated with each of the one or more security elements shown as analysis results in the threat analysis information d21. In addition, the evaluation specification generation unit 12 searches the security specification information for evaluation specifications associated with each of the one or more security elements shown as analysis results in the vulnerability analysis information d22. Then, in generating the evaluation specification information d13, the evaluation specification generation unit 12 generates evaluation specification information d13 that includes the plurality of evaluation specifications found in the security specification information. The database includes, for example, a vulnerability specification database 13, a countermeasure specification database 14, and an attack path specification database 15. The security specification information includes, for example, vulnerability specification information 13a, countermeasure specification information 14a, and attack path specification information 15a. The security elements are vulnerabilities, countermeasures, attack paths, etc.

[0160] This allows the security specification information to search for evaluation specifications corresponding to the analysis results for threats and the analysis results for vulnerabilities, and generates evaluation specification information d13 containing these evaluation specifications, making it easy to include multiple appropriate evaluation specifications in evaluation specification information d13.

[0161] Furthermore, the aforementioned database includes a vulnerability specification database 13 that stores vulnerability specification information 13a included in the security specification information. Vulnerability specification information 13a indicates multiple vulnerabilities as multiple security elements. Vulnerability analysis information d22 indicates one or more vulnerabilities included in the device under evaluation 40 as one or more security elements. The evaluation specification generation unit 12 then searches the vulnerability specification information 13a for evaluation specifications associated with each of these one or more vulnerabilities, and in generating evaluation specification information d13, it generates evaluation specification information d13 that includes one or more evaluation specifications found in vulnerability specification information 13a. For example, as shown in Figure 11, the evaluation specification generation unit 12 searches for evaluation specifications in vulnerability specification information 13a.

[0162] As a result, evaluation specifications corresponding to vulnerabilities in the device under evaluation 40 are searched from vulnerability specification information 13a, and evaluation specification information d13 containing those evaluation specifications is generated. Therefore, appropriate evaluation specifications for vulnerabilities in the device under evaluation 40 can be easily included in evaluation specification information d13.

[0163] Furthermore, the aforementioned database includes an attack path specification database 15 that stores the attack path specification information 15a included in the security specification information. The attack path specification information 15a indicates multiple attack paths as multiple security elements. The threat analysis information d21 indicates one or more attack paths in the device under evaluation 40 as one or more security elements. The evaluation specification generation unit 12 then searches the attack path specification information 15a for evaluation specifications associated with each of the one or more attack paths, and in generating the evaluation specification information d13, it generates evaluation specification information d13 that includes one or more evaluation specifications found in the attack path specification information 15a. For example, as shown in Figure 9, the evaluation specification generation unit 12 searches the attack path specification information 15a for evaluation specifications.

[0164] As a result, the evaluation specifications corresponding to the attack paths in the device under evaluation 40 are searched from the vulnerability specification information 13a, and evaluation specification information d13 containing those evaluation specifications is generated. Therefore, the appropriate evaluation specifications for the attack paths in the device under evaluation 40 can be easily included in the evaluation specification information d13.

[0165] Furthermore, the aforementioned database includes a countermeasure specification database 14 which stores the countermeasure specification information 14a included in the security specification information. The countermeasure specification information 14a indicates multiple countermeasures as multiple security elements. The threat analysis information d21 indicates one or more countermeasures for one or more threats to the device under evaluation 40 as one or more security elements. The evaluation specification generation unit 12 then searches the countermeasure specification information 14a for evaluation specifications associated with each of these one or more countermeasures, and in generating the evaluation specification information d13, it generates evaluation specification information d13 that includes one or more evaluation specifications found in the countermeasure specification information 14a. For example, as shown in Figure 10, the evaluation specification generation unit 12 searches the countermeasure specification information 14a for evaluation specifications.

[0166] As a result, evaluation specifications corresponding to countermeasures against threats to the device under evaluation 40 are retrieved from the countermeasure specification information 14a, and evaluation specification information d13 containing those evaluation specifications is generated. Therefore, appropriate evaluation specifications for countermeasures against threats to the device under evaluation 40 can be easily included in the evaluation specification information d13.

[0167] Furthermore, the evaluation support system 10 in this embodiment includes an evaluation tool database 16 that stores evaluation tool information 16a, which associates evaluation tools with each of a plurality of conditions. The evaluation specification generation unit 12 searches the evaluation tool information 16a for evaluation tools associated with the conditions that the analysis results shown in the threat analysis information d21 satisfy, and in generating the evaluation specification information d13, it generates evaluation specification information d13 that includes evaluation specifications using the evaluation tools found in the evaluation tool information 16a. For example, as shown in Figure 14, the evaluation specification generation unit 12 searches for evaluation tools in the evaluation tool information 16a. In the example in Figure 14, each of the plurality of conditions consists of six conditions, from the first to the sixth condition, and the evaluation tool is expressed as a tool name. Also, the analysis results shown in the threat analysis information d21 mentioned above include a function with a function ID and function name, and the OS in the example in Figure 14.

[0168] This generates evaluation specification information d13, which includes evaluation specifications using evaluation tools based on the analysis results of the threat. Therefore, evaluation specifications that enable the evaluation to be performed appropriately can be easily included in evaluation specification information d13.

[0169] Furthermore, in this embodiment, the evaluation specification generation unit 12 generates the evaluation procedure for the device under evaluation 40 using the functions and attack paths shown as analysis results in the threat analysis information d21 when generating the evaluation specification information d13. The evaluation specification generation unit 12 then includes the generated evaluation procedure in at least one of the multiple evaluation specifications. The evaluation procedure is generated as a detailed procedure and included in the evaluation specification, for example, as shown in Figure 14.

[0170] This means that the evaluation procedure is included in the evaluation specification, for example, as a detailed procedure. Therefore, evaluators can refer to this detailed procedure, increasing the likelihood that they can perform an appropriate evaluation without hesitation. Furthermore, by generating a detailed procedure for each product, such as the equipment 40 being evaluated, appropriate evaluations can be performed for each product.

[0171] Furthermore, each of the multiple evaluation specifications in this embodiment includes criteria for the evaluation results of the equipment 40 under evaluation. In the example shown in Figure 15, these criteria correspond to the judgment criteria.

[0172] As a result, since the evaluation specifications include criteria, for example, as judgment criteria, when an evaluation is performed according to the evaluation specifications, it is easy to determine whether the evaluation result is OK or NG using those judgment criteria.

[0173] Furthermore, the evaluation specification generation unit 12 in this embodiment determines the priority of each of the multiple evaluation specifications retrieved from the security specification information.

[0174] This allows evaluators to easily determine, by referring to the priority of each of the multiple evaluation specifications, which evaluations based on which specifications should be performed first and which can be postponed. Therefore, it is possible to prevent important evaluations from being performed first and important evaluations from being postponed.

[0175] Furthermore, in this embodiment, the security specification information shows the technical level, evaluation time, and impact of each evaluation specification as numerical values. When determining priority, the evaluation specification generation unit 12 identifies the technical level, evaluation time, and impact corresponding to each of the multiple evaluation specifications retrieved from the security specification information. The evaluation specification generation unit 12 then determines priority by performing weighted addition on the identified technical level, evaluation time, and impact.

[0176] This allows the priority of an evaluation specification to be determined by weighting its technical level, evaluation time, and impact. For example, increasing the weight of the technical level allows the priority to be determined primarily from the perspective of the technical level, increasing the weight of the evaluation time allows the priority to be determined primarily from the perspective of the evaluation time, or increasing the weight of the impact allows the priority to be determined primarily from the perspective of the impact (e.g., quality). Furthermore, by adjusting the weight for each product being evaluated, an appropriate priority can be determined for each product.

[0177] Furthermore, the evaluation support system 10 in this embodiment includes a feedback unit that feeds back the evaluation specification information d13, which shows the evaluation results of the device under evaluation 40 obtained by evaluation according to the evaluation specification information d13, to the threat analysis unit 21 and the vulnerability analysis unit 22 as evaluated specification information d14. The threat analysis unit 21 generates threat analysis information d21 by performing a threat analysis on the device under evaluation 40. The vulnerability analysis unit 22 generates vulnerability analysis information d22 by performing a vulnerability analysis on the device under evaluation 40. In the examples of Figures 16 and 26, the evaluated specification information d14 shows the test results as the evaluation results of the device under evaluation 40. Also, in the example shown in Figure 1, the feedback unit is configured as an evaluation database 17.

[0178] As a result, the evaluated specification information d14 is fed back to the threat analysis unit 21 and the vulnerability analysis unit 22. Therefore, if the evaluated specification information d14 shows a good evaluation result for the evaluation specification, the threat analysis unit 21 can guarantee the results of the threat analysis. For example, if the analysis result is a countermeasure against the threat, the effectiveness of that countermeasure can be guaranteed. On the other hand, if the evaluated specification information d14 shows a bad evaluation result for the evaluation specification, the threat analysis unit 21 can improve the threat analysis. For example, if the analysis result is a countermeasure against the threat, the countermeasure can be improved. As a result, the accuracy of the countermeasure planning can be increased. In addition, the vulnerability analysis unit 22 can improve the accuracy of the vulnerability analysis based on the evaluation results shown in the evaluated specification information d14. In other words, in this embodiment, not only is there a one-way flow from threat analysis and vulnerability analysis to evaluation, but the evaluation results are also fed back to the threat analysis and vulnerability analysis, so the risk management of the evaluation target device 40 can be performed effectively.

[0179] The above description has been based on embodiments of the evaluation support system 10 and evaluation support method relating to one or more embodiments of the present disclosure. However, the present disclosure is not limited to these embodiments. Various modifications to the above embodiments that a person skilled in the art could conceive of may also be included in the present disclosure, as long as they do not depart from the spirit of the present disclosure.

[0180] For example, in the above embodiment, the evaluation support system 10 does not have an input unit that accepts user input operations, but it may have such an input unit. This allows the user to easily input evaluation environment elements such as license type, protocol, executable format, and output format by performing input operations on the input unit. The user may also input any information related to the generation of evaluation specification information d13.

[0181] Furthermore, in the above embodiment, the function ID and function name are shown in the threat analysis information d21, but only one of them may be shown in the threat analysis information d21. Similarly, in the above embodiment, the asset ID and asset name are shown in the threat analysis information d21, but only one of them may be shown in the threat analysis information d21. Similarly, in the above embodiment, the attack path ID and attack path are shown in the threat analysis information d21, but only one of them may be shown in the threat analysis information d21. In this case, the attack path specification information 15a may show only one of the attack path ID and attack path.

[0182] Furthermore, in the above embodiment, the device under evaluation 40 is an ECU, but it may be any other device that performs information processing.

[0183] Furthermore, in the above embodiment, the evaluation database 17 stores the evaluated specification information d14, and this evaluated specification information d14 is fed back. Here, new countermeasures for the evaluation items of the judgment result (specifically NG) included in the evaluated specification information d14 may also be stored in the evaluation database 17 and fed back. These new countermeasures may be stored through human input.

[0184] Furthermore, evaluated specification information d14, etc., may be fed back not only to the threat analysis unit 21 and the vulnerability analysis unit 22, but also to other components. These other components may be the security execution unit, the CS (Cyber ​​Security) compliance unit, etc. The security execution unit is the lowest-level process in the automotive development V-model and performs tasks such as coding the software program for the device under evaluation 40.

[0185] Furthermore, although the evaluation support system 10 in the above embodiment does not include an evaluation unit 32 and a result determination unit 33, it may include these components.

[0186] Furthermore, in the above embodiment, each component included in the evaluation support system 10 may perform the processing corresponding to that component in response to human input operations, or it may be performed automatically without accepting such input operations. Similarly, each component included in the development system 100 may perform the processing corresponding to that component in response to human input operations, or it may be performed automatically without accepting such input operations. In addition, machine learning models that show the correlation between input and output may be used for the automatically performed processing.

[0187] In the above embodiment, each component may be implemented by dedicated hardware or by executing a software program suitable for each component. Each component may also be implemented by a program execution unit such as a CPU (Central Processing Unit) or processor reading and executing a software program recorded on a recording medium such as a hard disk or semiconductor memory. Here, the software that implements the evaluation support system, etc., in the above embodiment is a computer program that causes a computer to execute each step of the flowchart shown in Figure 18.

[0188] The following cases are also included in this disclosure.

[0189] (1) The above-mentioned at least one system or device is specifically a computer system comprising a microprocessor, ROM, RAM, hard disk unit, display unit, keyboard, mouse, etc. A computer program is stored in the RAM or hard disk unit. The above-mentioned at least one device achieves its function by the operation of the microprocessor in accordance with the computer program. Here, the computer program is composed of a combination of multiple instruction codes that indicate instructions to the computer in order to achieve a predetermined function.

[0190] (2) Some or all of the components constituting at least one of the above-described systems or devices may be made up of a single system LSI (Large Scale Integration). A system LSI is a multi-functional LSI manufactured by integrating multiple components onto a single chip, and specifically, is a computer system comprising a microprocessor, ROM, RAM, etc. A computer program is stored in the RAM. The system LSI achieves its function by operating the microprocessor according to the computer program.

[0191] (3) Some or all of the components constituting at least one of the above systems or devices may consist of an IC card or a standalone module that is detachable from the device. The IC card or module is a computer system consisting of a microprocessor, ROM, RAM, etc. The IC card or module may include the above-mentioned multi-functional LSI. The IC card or module achieves its function by the operation of the microprocessor in accordance with a computer program. The IC card or module may be tamper-resistant.

[0192] (4) The disclosure may also be the methods described above. Alternatively, it may be a computer program that implements these methods using a computer, or a digital signal consisting of a computer program.

[0193] Furthermore, this disclosure may also refer to a computer program or digital signal recorded on a computer-readable recording medium, such as a flexible disk, hard disk, CD (Compact Disc)-ROM, DVD, DVD-ROM, DVD-RAM, BD (Blu-ray® Disc), semiconductor memory, etc. Alternatively, it may refer to a digital signal recorded on such a recording medium.

[0194] Furthermore, this disclosure may also include the transmission of computer programs or digital signals via telecommunications lines, wireless or wired communication lines, networks such as the Internet, data broadcasting, etc.

[0195] Alternatively, the program or digital signal may be carried out by another independent computer system by recording and transferring it on a recording medium, or by transferring the program or digital signal via a network or the like. [Industrial applicability]

[0196] The evaluation support system disclosed herein can be applied to devices or systems that support the evaluation of, for example, ECUs incorporated into vehicles. [Explanation of Symbols]

[0197] 10. Evaluation Support System 11 Acquisition Department 12 Evaluation Specification Generation Unit 13. Vulnerability Specification Database 13a Vulnerability Specification Information 13b Vulnerability feature information 14 Countermeasures Specification Database 14a Countermeasures Specification Information 15 Attack Path Specification Database 15a Attack Pass Specifications 16 Evaluation Tool Database 17. Evaluation Database (Feedback Department) 21 Threat Analysis Department 22 Vulnerability Analysis Department 32 Evaluation Department 33 Result judgment section 34. Determination Processing Unit 40 Equipment to be evaluated 41 BT interface 42 USB interfaces 43 CAN interface 44 Main microcontroller 45 CAN microcontroller 91 Smartphone 92 Diag 100 Development Systems d13 Evaluation Specifications d14 Evaluated Specifications d21 Threat Analysis Information d22 vulnerability analysis information d32 result information d33 Judgment Information

Claims

1. An evaluation support system that assists in the evaluation of equipment under evaluation, An acquisition unit that acquires threat analysis information showing the results of an analysis of information security threats in the device under evaluation, and vulnerability analysis information showing the results of an analysis of information security vulnerabilities in the device under evaluation. An evaluation specification generation unit generates evaluation specification information including multiple evaluation specifications for the device under evaluation based on the threat analysis information and the vulnerability analysis information, An evaluation support system equipped with the following features.

2. The aforementioned evaluation support system further, It includes a database that stores security specification information, which associates evaluation specifications with each of multiple security elements. The evaluation specification generation unit, The evaluation specifications associated with each of the one or more security elements shown as the analysis results in the threat analysis information are searched from the security specification information. The evaluation specifications associated with each of the one or more security elements shown as the analysis results in the vulnerability analysis information are searched from the security specification information. In generating the aforementioned evaluation specification information, The system generates evaluation specification information that includes a plurality of evaluation specifications retrieved from the security specification information. The evaluation support system according to claim 1.

3. The database includes a vulnerability specification database that stores vulnerability specification information included in the security specification information, The vulnerability specification information indicates multiple vulnerabilities as multiple security elements, The vulnerability analysis information indicates one or more vulnerabilities included in the device under evaluation as one or more security elements. The evaluation specification generation unit, The evaluation specifications associated with each of the one or more vulnerabilities are searched from the vulnerability specification information. In generating the aforementioned evaluation specification information, The system generates evaluation specification information that includes one or more evaluation specifications retrieved from the vulnerability specification information. The evaluation support system according to claim 2.

4. The database includes an attack path specification database that stores attack path specification information included in the security specification information, The aforementioned attack path specification information indicates multiple attack paths as multiple security elements, The threat analysis information indicates one or more attack paths in the device under evaluation as one or more security elements. The evaluation specification generation unit, The evaluation specifications associated with each of the one or more attack paths are searched from the attack path specification information. In generating the aforementioned evaluation specification information, The system generates evaluation specification information that includes one or more evaluation specifications retrieved from the attack path specification information. The evaluation support system according to claim 2.

5. The database includes a countermeasure specification database that stores the countermeasure specification information included in the security specification information, The aforementioned countermeasure specification information indicates multiple countermeasures as multiple security elements, The threat analysis information indicates one or more countermeasures against one or more threats to the device under evaluation as one or more security elements. The evaluation specification generation unit, The evaluation specifications associated with each of the one or more countermeasures are searched from the countermeasure specification information. In generating the aforementioned evaluation specification information, The system generates evaluation specification information that includes one or more of the evaluation specifications retrieved from the countermeasure specification information. The evaluation support system according to claim 2.

6. The aforementioned evaluation support system further, It includes an evaluation tool database that stores evaluation tool information, which associates and displays evaluation tools with each of multiple conditions. The evaluation specification generation unit, The evaluation tool associated with the conditions satisfied by the analysis results shown in the threat analysis information is searched from the evaluation tool information. In generating the aforementioned evaluation specification information, The system generates evaluation specification information, which includes the evaluation specifications using the evaluation tool retrieved from the evaluation tool information. The evaluation support system according to claim 1.

7. In generating the evaluation specification information, the evaluation specification generation unit... Using the functions and attack paths shown in the threat analysis information as a result of the analysis, the procedure for evaluating the target device is generated. The generated evaluation procedure is included in at least one of the plurality of evaluation specifications. The evaluation support system according to claim 1.

8. Each of the aforementioned evaluation specifications includes a criterion for the evaluation results of the equipment under evaluation. The evaluation support system according to claim 1.

9. The evaluation specification generation unit further, The priority of each of the multiple evaluation specifications retrieved from the security specification information is determined. The evaluation support system according to claim 2.

10. The aforementioned security specification information, for each evaluation specification, shows the technical level, evaluation time, and impact of that evaluation specification as numerical values, respectively. In determining the priority, the evaluation specification generation unit, For each of the multiple evaluation specifications retrieved from the aforementioned security specification information, The technical level, evaluation time, and impact corresponding to the evaluation specification are identified from the security specification information, The priority is determined by weighting the identified technical level, evaluation time, and impact. The evaluation support system according to claim 9.

11. The aforementioned evaluation support system further, The system includes a feedback unit that feeds back the evaluation specification information, which shows the evaluation results of the equipment to be evaluated obtained by evaluation in accordance with the evaluation specification information, to the threat analysis unit and the vulnerability analysis unit as evaluated specification information. The threat analysis unit generates the threat analysis information by performing a threat analysis on the equipment to be evaluated. The vulnerability analysis unit generates the vulnerability analysis information by performing a vulnerability analysis on the equipment to be evaluated. The evaluation support system according to claim 1.

12. An evaluation support method for supporting the evaluation of equipment under evaluation, The system acquires threat analysis information showing the results of the analysis of information security threats in the device under evaluation, and vulnerability analysis information showing the results of the analysis of information security vulnerabilities in the device under evaluation. Based on the threat analysis information and the vulnerability analysis information, evaluation specification information including multiple evaluation specifications for the device under evaluation is generated. Evaluation support methods.

13. A program to support the evaluation of the equipment under evaluation, The system acquires threat analysis information showing the results of the analysis of information security threats in the device under evaluation, and vulnerability analysis information showing the results of the analysis of information security vulnerabilities in the device under evaluation. Based on the threat analysis information and the vulnerability analysis information, evaluation specification information including multiple evaluation specifications for the device under evaluation is generated. A program that causes a computer to perform a task.

Images