Method for applying test cases for security checks

Abstract

The present invention relates to a computer-implemented method and a test system and a computer program product for applying test cases TC1 for security checks of functional systems 105 to be tested of mobility applications by a test system, wherein test cases TC1 used during security checks of functional systems used in mobility applications and / or usable for such security checks are provided to the test system. a step 101 of dividing the test case TC1 into test modules T11, ..., T14; a step 102 of abstracting the test modules T11, ..., T14, in which the information, data and parameters contained in the test modules T11, ..., T14 and specific to the system in which the respective test case TC1 is used are replaced by abstract placeholder variables in the respective test modules T11, ..., T14; a step 103 of creating abstract test scenarios TS1, TS2, TS3 for security checks of the functional system to be tested, in which abstracted test modules a1, ..., a5 are combined into one test scenario TS1, TS2, TS3; - a step 104 of deriving test cases TC1 for the functional system to be tested from the abstract test scenarios TS1, TS2, TS3, in which corresponding system-specific information, data and parameters of the functional system to be tested are assigned to placeholder variables in the abstracted test modules a1, ..., a5 of each test scenario TS1, TS2, TS3; a step 105 of using test cases TC21, TC22, TC23 derived from test scenarios TS1, TS2, TS3 on the functional system to be tested; is executed.

Description

[Technical Field]

[0001] The present invention relates to a computer-implemented method for applying test cases for security checks of functional systems to be tested, in particular for mobility applications from the automotive and / or aviation sector, by a test system. To this end, the test system is provided with test cases that are used for security checks of functional systems that are used in or intended for mobility applications. Furthermore, the present invention relates to an associated test system and an associated computer program product. [Background technology]

[0002] Modern technical devices, in particular vehicles, have, when they reach a certain level of complexity, a large number of hierarchically linked components, each equipped with its own processor or microcontroller. However, the present invention is not limited to vehicles in the proper sense, but can also be used in connection with other technical units and systems that have a large number of linked components and are used in particular in the automotive and / or aviation fields in mobility applications, i.e., for example, in or around vehicles. In the context of the present invention, such vehicles, other technical units and systems are collectively referred to by the term "functional system". This means that such a functional system can, for example, be a vehicle itself (e.g., a car, a truck, an airplane, etc.) or a complex system used in or in connection with a vehicle to realize a mobility application.

[0003] Examples of such functional systems include the vehicle itself, in particular the combination of various external units / systems with vehicle or vehicle internal units and systems, the combination of vehicle internal systems and units, and the combination of several vehicles or units / systems of several vehicles. Thus, for example, a specific task may be performed by an autonomous computing unit (e.g., a tablet, a smartphone, etc.) which communicates via interfaces with units or systems of the (autonomous) vehicle (e.g., infotainment system, GPS, navigation system, etc.).

[0004] In the case of mobility applications, such as automated driving (AD) or so-called V2X communication, vehicles or in-vehicle systems can exchange data or information with infrastructure units (e.g., traffic lights, barriers, etc.), for example wirelessly. Furthermore, in the case of mobility applications, such as driver assistance systems or so-called advanced driver assistance systems (ADAS), for example, within a vehicle, units or systems (e.g., sensors, etc.) can communicate with other units and systems (e.g., braking systems, engines, etc.) via interfaces. On the other hand, multiple vehicles can also communicate with each other wirelessly, for example, to perform cooperative operations, such as coupled virtual towing, or to perform mobility applications, such as V2V communication, for exchanging information and data between vehicles. Furthermore, vehicles, such as special vehicles in the construction industry or agriculture, emergency vehicles, etc., can be connected to one or more on-board devices. Such combined units or systems are considered functional systems within the meaning of the present invention. The unit or system is used in or for realizing a mobility application (e.g., a vehicle, an autonomous driving, an ADAS, a V2I application or a V2V application, etc.).

[0005] Typically, components of such functional systems have their own memory units and communication interfaces and therefore constitute independent computer systems. Such components are also called "embedded systems" (ES) and perform their tasks within the functional system, almost invisible to the naked eye. In complex integrated systems (e.g., vehicles, aircraft, etc.), many other autonomous embedded systems are often networked together. Such components of functional systems can also be called "cyber-physical systems" (CPS). A cyber-physical system refers to a system that combines software-based units with mechanical and electronic elements and communicates via a data infrastructure (e.g., the Internet, bus systems, etc.). In this case, a cyber-physical system can be built by networking multiple embedded systems via wired and / or wireless communication networks, for example, and can be used in mobility applications (e.g., networked security systems, networked driver assistance systems, etc.).

[0006] Therefore, the complexity of such functional systems, especially in the automotive and / or aviation fields, is rapidly increasing to a degree that cannot be ignored. For example, a vehicle typically has dozens of control devices. Software consisting of tens of millions of lines of program code runs simultaneously on these control devices. However, examples of communication interfaces include various wireless link protocols such as mobile communication protocols (e.g., 5G, LTE, etc.), Bluetooth, WLAN, RFID, V2X interfaces, etc. Some of these wireless link protocols are used simultaneously. Due to the complexity and various communication interfaces, such functional systems have a large attack surface for cyber attacks, such as so-called denial-of-service (DoS) attacks / flooding attacks. Furthermore, there is a risk that attacks will be carried out not only via the actual communication interfaces but also via sensors (e.g., LIDAR, radar systems, etc.). In this case, such attacks can be carried out, for example, by so-called false signals. In principle, any action that affects a functional system from the outside or inside with fraudulent intent and / or any action that fraudulently transfers data from a functional system to a third party is considered a cyber-attack.

[0007] Due to the physical characteristics of vehicles (mass and speed, i.e. high kinetic energy, possibility of direct contact with many people), their large numbers (of vehicles) and the development of autonomous vehicles, such attacks can pose an extremely high risk. This applies not only to large numbers of vehicles, such as cars or trucks, but also to rail vehicles and ships, and especially aircraft. It is therefore important to protect functional systems used in mobility applications - particularly in the automotive and aviation sectors - from cyber-attacks, i.e. from unauthorized influences, so that such attacks have as little or no effect on the respective systems as possible.

[0008] To protect functional systems from cyber attacks and increase their resilience, it is beneficial to find vulnerabilities as early as possible and avoid them as far as possible already during the development process. At the planning stage, this can be achieved by appropriate architectural measures, as described for example in Non-Patent Document 1. During the implementation of the plan, this can be achieved by appropriate development processes, such as best practices, source code reviews, etc.

[0009] To ensure that all relevant vulnerabilities—especially those that arise only when components are combined—can be discovered as much as possible, security checks of functional systems are essential after the components are fully assembled and integrated into a system, e.g., an embedded system and / or a cyber-physical system. Such security checks must enable manufacturers (OEMs), government agencies (e.g., licensing authorities), and other stakeholders (e.g., consumer organizations, fleet operators, etc.) to assess the specific risks of cyberattacks on functional systems of mobility applications, e.g., vehicles. This can be applied for tasks such as inspection or certification when improving or retrofitting vehicles or functional systems. In particular, security checks must detect vulnerabilities and potential vulnerabilities to cyberattacks that exist in functional systems (e.g., vehicles, etc.), but which are often initially unknown. To detect vulnerabilities and potential vulnerabilities (i.e., initially unknown paths through which a cyberattack can be implemented on the functional system to be tested), various attacks are defined for the functional system to be tested (e.g., as test vectors) and then used as system-specific test cases on the functional system to be tested during the security check.

[0010] For example, security checks are required during the development of a new vehicle at the OEM and in the case of market launch / standardization / type approval, and subsequently also throughout the entire lifespan of the vehicle or each functional system. Due to changeable configurations, continuous updates of (parts of) software components, changed environmental conditions (e.g., vehicle-to-infrastructure - V2I, vehicle-to-vehicle - V2V: change of target), newly identified test vectors (derived from specific security research), etc., repeated security checks are necessary for the system test layer, i.e., continuous checks of cybersecurity. In this case, such security checks may be performed in addition to the manufacturer, for example, by the (vehicle) management operator, the licensing authority and / or specialized third parties.

[0011] However, due to the highly restrictive protection and confidentiality policies of manufacturers, particularly in the automotive and even aviation fields, and / or due to the unique and / or manufacturer-specific design of functional systems for mobility applications, uniformly standardized hardware and software are generally not used for functional systems. Therefore, it is difficult to create commonly usable test cases for multiple security checks that are suitable for determining the security or cybersecurity of functional systems with identical functions from different manufacturers. Even when components of a functional system are updated or the functional system is updated, new test cases may have to be created for security checks, or existing test cases may have to be at least adapted. That is, functional systems to be checked for security or cybersecurity—even if their functions are identical—are often different and mostly manufacturer-specific. As a result, test cases must be newly created or at least adapted for security checks, and therefore for each functional system to be tested, individually tailored to the system. In some cases, even if the vulnerabilities or potential vulnerabilities may be similar, existing test cases cannot be transferred from one functional system to another functional system due to the separate design of each functional system to be tested. That is, test cases for security checks must be applied individually to each functional system to be tested, and often manually. Therefore, security checks of functional systems used in mobility applications, such as vehicles or vehicle sub-systems or subsystems, are time- and resource-intensive.

[0012] For example, Patent Document 1 discloses a method and system in which a specific automated test script for detecting errors in an application is converted into an abstract test case representation, which is then stored in a database for reuse. For the abstract test case representation, one or more application states, external interaction sequences, and input data are stored, and the abstract test representation can be enriched with information from an application metadata repository so that the abstract test representation can be used in another test environment or another test platform to test the application. Thus, although a test case can be abstracted to test an application, the abstract test case representation can only be used to test the same application in another test environment or another test platform. [Prior art documents] [Patent documents]

[0013] [Patent Document 1] US Patent Application Publication No. 20050160322 [Non-patent literature]

[0014] [Non-Patent Document 1] Papadimitratos, P., Buttyan, L., HOLCZER, T., Schoch, E., Freudiger, J., Raya, M., Ma, Z., Kargl, F., Kung, A., & Hubaux, J.-P., "Secure Vehicular Communication Systems: Design and Architecture" (2008, IEEE Communications Magazine, Volume 46, Issue 11, Pages 100-109) Summary of the Invention [Problem to be solved by the invention]

[0015] Therefore, the object of the present invention is to provide a method by which test cases that have already been used during a security check for testing a functional system and / or that are available for such a security check can be automatically applied and reused in a simple, resource-saving manner for a security check of another functional system to be further tested. [Means for solving the problem]

[0016] This and other objects are achieved by the methods described in the independent claims. Preferred embodiments of the invention are described in the dependent claims.

[0017] According to the present invention, this problem is solved by a computer-implemented method for applying test cases for a security check of a functional system to be tested used in a mobility application by a test system, wherein the test system is provided with test cases that have already been used in the mobility application during the security check and / or that are available for such a security check, in which: dividing the test cases into test modules; - a step of abstracting the test modules, in which information, data and parameters contained in the test modules and specific to the system in which each test case is used are replaced by abstract placeholder variables in the respective test modules; - creating abstract test scenarios for security checks of the functional system to be tested, in which abstracted test modules are combined into one test scenario; - deriving test cases for a functional system to be tested from the abstract test scenarios, in which corresponding system-specific information, data and parameters of the functional system to be tested are assigned to placeholder variables in the abstracted test modules of each test scenario; - applying test cases derived from the test scenarios to the functional system to be tested; is executed.

[0018] The main feature of the proposed solution is that known test cases or test cases already used and / or available for security checks of functional systems in mobility applications can be simply and resource-savingly generalized and made available for security checks of newly to-be-tested functional systems. In this case, application to each newly to-be-tested functional system is performed automatically. According to the method of the present invention, already used test cases are automatically "recycled" for security checks of newly to-be-tested functional systems. This means that new test scenarios and specific test cases do not have to be developed for each newly to-be-tested functional system, but rather test cases already used and / or available for security checks of other functional systems are returned and then automatically applied for security checks of the newly to-be-tested functional systems. This saves resources and time during security checks.

[0019] Furthermore, it is advantageous to generate feedback after using test cases derived from the test scenarios on the functional systems to be tested for each test case. The feedback is then transmitted to the test system, where each test case is associated with the respective test scenario from which it was derived. This makes it possible, for example, to very easily identify test scenarios that perform very well or that perform poorly. Furthermore, the feedback may store for each test scenario which functional systems the test scenario has already been used on and how useful the test scenario was in uncovering, for example, security vulnerabilities. Thus, during subsequent security checks of the same or similar functional systems, functional test scenarios can be quickly found or poorly performing test scenarios can be avoided.

[0020] During the abstraction, the test modules are also usefully searched for necessary parameters to be provided from the functional system to be tested or from at least one of the preceding test modules in order to execute the respective test modules. The necessary parameters are stored in a parameter list in the test system, and each parameter list is associated with the test module abstracted from the respective test module. Furthermore, the relationship between individual parameters may be stored in the parameter list so that the relationship can be utilized during the creation of test scenarios. Furthermore, during the creation of the test system, unimportant, non-functional, and / or unsuitable test scenarios can be quickly identified and eliminated based on the parameter list, for example, if necessary parameters are not obtainable due to the order and / or combination of multiple test modules. Furthermore, to identify the parameters—particularly when using test cases derived from the respective test scenarios—the parameter list can be used to include corresponding other steps in the test scenario. The creation and execution of such steps, such as identifying parameters or parameter values ​​from the functional system to be tested, can be performed automatically, for example.

[0021] Preferably, the created abstract test scenario is searched for placeholder variables for the required parameters, and the required parameters can be provided to the respective test module from the functional system to be tested and / or from at least one of the preceding test modules in order to execute the test cases derived from the test scenario.

[0022] Ideally, the parameter list associated with each abstracted test module of each test scenario is used to find placeholder variables for the required parameters in each test scenario, so that the placeholder variables and, possibly, the relationships between them can be quickly and easily recognized and utilized.

[0023] Furthermore, it is advantageous to store the abstracted test modules and / or the abstract test scenarios and / or parts of the abstract test scenarios in the test system, in particular in a data bank, so that they can be reused for subsequent security checks of the functional system to be tested. In this case, on the one hand, the test cases can be applied more quickly to the newly tested functional system, and on the other hand, this allows the test module data bank and the test scenario database to be built, i.e., the method of the present invention to be further accelerated. In this case, it may also be advantageous to check whether each abstracted test module is already stored in the test system, in particular in a data bank, before storing it. This makes it very easy to avoid storing the abstracted test modules multiple times.

[0024] Ideally, information about which functional systems each test scenario can be used with is stored in the test system for each abstracted test scenario, so that suitable test scenarios can be found quickly and with little effort for security checks of the functional systems to be tested.

[0025] A preferred configuration of the method proposes that the system-specific information, data and parameters of the functional system to be tested, for assignment to placeholder variables in the abstracted test modules of the respective test scenarios, are provided from the test system, in particular from a database and / or from the system manufacturer, in which empirical values ​​from previous security checks of identical or similar functional systems are stored and / or are identified by machine learning from the empirical values ​​of previous security checks. As the system-specific information, data and parameters of the functional system to be tested for assignment to placeholder variables in the abstracted test modules of the respective test scenarios, values ​​identified from the analysis of the functional system to be tested are used.

[0026] The above-mentioned problems are also solved by a test system and a computer program product. The test system has at least one computer unit configured to execute the method of the present invention. Preferably, the test system may also include or be connected to a memory unit, such as a data bank. Ideally, this memory unit is configured to store abstracted test modules, abstract test scenarios and / or parts of abstract test scenarios, as well as empirical values ​​from previous security checks of the same or similar functional systems. The test system may be connected to or include a system for machine learning. Using the system for machine learning, system-specific information, data, and parameters of the functional system to be tested can be identified from empirical values, etc., for assignment to placeholder variables in the abstracted test modules of the respective test scenarios.

[0027] Preferably, the computer program product is loadable into a test system - for example at least one computer unit - and contains instructions that cause the computer program product to carry out the method of the present invention when the computer program product is executed in the test system.

[0028] The invention will now be described in more detail with reference to Figures 1 to 2c, which show exemplary, schematic and non-limiting preferred embodiments of the invention. [Brief explanation of the drawings]

[0029] [Figure 1] 1 illustrates steps of a computer-implemented method for applying test cases for security checks of a functional system to be checked. [Figure 2a] 1 illustrates abstract steps of the method of the present invention according to a specific embodiment. [Figure 2b] 1 illustrates the synthesis steps of the method of the present invention according to a specific embodiment. [Figure 2c] 1 illustrates specific steps of the method of the present invention according to a specific embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0030] 1 illustrates a procedure of a computer-implemented method. By using this procedure, a test case TC1 used in a security check of a functional system (e.g., a mobility application (e.g., an automobile, etc.) already executed by the functional system and / or the functional system is used in a mobility application (e.g., autonomous driving, V2V communication, V2X communication, etc.)) can be automatically applied to a security check of a functional system to be tested (e.g., a vehicle, a vehicle system, etc.). In this case, the functional system to be tested may be a newly developed system, an improved and / or improved version of an existing functional system, or a functional system in which individual components (e.g., hardware units, software units, etc.) have been replaced or updated.

[0031] The method can be executed on an associated test system, which in this case comprises at least one computer unit. The test system is an active system that performs security checks of the system to be tested according to a computer program product. This computer program product can be, for example, loaded into an internal memory of the test system and executed by, for example, at least one computer unit or processor. Thus, the computer program product comprises instructions that cause the computer program product to carry out the steps of the method of the present invention when executed by the test system.

[0032] For example, to discover vulnerabilities in a functional system to be tested, the corresponding test cases TC1, TC21, ..., TC23 are executed by a test system. To this end, the test system can provide, for example, an interface and an environment (e.g., a test bench or test bed for a mobility application) and, together with the functional system to be tested, form a test environment. The method of the present invention is then executed in this test environment. In this test environment or for this test system, test cases TC1 already used during security checks of other functional systems are available. That is, the test system knows test cases TC1 used by functional systems used in mobility applications during previously executed security checks and / or test cases TC1 that are available for such security checks. In this case, test cases TC1, TC21, ..., TC23 are test scenarios TS1, TS2, TS3 applied to each functional system to be tested, and can be executed, for example, on the test system and on the functional system to be tested during the security check. In this case, abstract descriptions of test cases TC1, TC21, ..., TC23 are considered to be test scenarios TS1, TS2, TS3. The abstract description specifies what should be tested and by what means, and how the individual test scenario phases can be described. For example, test scenarios TS1, TS2, TS3 for a functional system can be obtained from a security analysis and / or security requirements.

[0033] To apply a specific test case TC1 that has already been used by a test system on one functional system to another functional system that is actually to be tested, the steps described below and illustrated in Figure 1 are performed by the test system.

[0034] To this end, in a modularization step 101, a test case TC1 that has already been used to test the functional system is analyzed by the test system and divided or partitioned into separate test modules T11, T12, T13, T14. The test modules T11, T12, T13, T14 may be, for example, test scripts that, when executed, perform targeted attacks against the functional system. If desired, the test modules T11, T12, T13, T14 may consist of only one or several instructions.

[0035] In the subsequent abstraction step 102, the recognized test modules T11, T12, T13, and T14 are abstracted from the test case TC1. In this case, the system-specific information, data, and parameters contained in each of the test modules T11, T12, T13, and T14 (of the functional system in which the respective test case TC1 is used) are replaced with abstract placeholder variables. Thus, in the abstraction step 102, abstracted test modules a1, a2, a3, and a4 are generated from the concrete test modules T11, T12, T13, and T14 that are applied specifically to the system for each test case TC1. Thus, each abstracted test module a1, a2, a3, and a4 is a system-independent text block that describes a separate phase of the test scenarios TS1, TS2, and TS3 and is independent of a specific functional system.

[0036] For example, the abstracted test modules a1, a2, a3, a4 can be stored in the test system, e.g., in an associated data bank. To this end, the data bank can be located, for example, in the internal storage of the test system or in a storage connected to the test system. In this case, before storing each abstracted test module a1, a2, a3, a4, it can be checked whether this test module a1, a2, a3, a4 is recorded in the test system or in the associated data bank. To enable the abstracted test modules a1, a2, a3, a4 to be stored in the test system or in the data bank in an appropriate format, a corresponding description language—e.g., a so-called domain-specific language or DSL for short—is used. A domain-specific language is a formal language. This formal language is designed and implemented for a specific problem domain—a so-called domain—for interaction between humans and digitally operating units (e.g., computers).

[0037] Furthermore, in the abstraction step 102, even before abstraction, the test modules T11, T12, T13, and T14 can be searched for necessary parameters. Necessary parameters are, for example, parameters that are essential for the proper execution of the respective test modules T11, T12, T13, and T14. These parameters can be provided, for example, from the functional system to be tested or from one of the preceding test modules T11, T12, T13, and T14 in the respective test case TC1. However, these parameters can also be obtained from external sources, such as data banks, machine-readable specifications, etc., or entered manually. Such parameters can be, for example, the network address, identification information, and / or specific mapping addresses of the test subject (which must be present to execute the test modules T11, T12, T13, and T14 when sending specific commands via or to components of the functional system to be tested), or configuration data. For example, these parameters define the (step-by-step) characteristics of each test module T11, T12, T13, T14 - for example, to which network address or which port number specific information should be sent, or how long this message should be.

[0038] Required parameters may exist for each test module T11, T12, T13, and T14, and may be specific to each test module T11, T12, T13, and T14. Furthermore, required parameters, such as port numbers, may exist that are common to all test modules T11, T12, T13, and T14 of test case TC1, or that are also common to other test cases TC21, TC22, and TC23 for each functional system. Such parameters may be defined once as "global" parameters for all test modules T11, T12, T13, and T14 of test case TC1.

[0039] The identified required parameters for each test module T11, T12, T13, and T14 can then be combined into a parameter list, for example. The parameter list can store the identified required parameters for each test module T11, T12, T13, and T14, as well as associations between the individual required parameters, for example, whether the individual parameters must have the same value or are obtained from a combination of multiple other parameters. The parameter list can then be generated from each test module T11, T12, T13, and T14 and associated with the abstracted test modules a1, a2, a3, and a4. For example, each parameter list can be stored together with the abstracted test modules a1, a2, a3, and a4 in the test system or an associated database.

[0040] Subsequently, in a synthesis step 103, abstract test scenarios TS1, TS2, and TS3 for security checks of the respective functional systems to be tested are created by combining the abstracted test modules a1, a2, a3, and a4. In this case, abstracted test modules a1, a2, a3, and a4 generated from different test cases TC1—e.g., already used and / or available—can be combined into new test scenarios TS1, TS2, and TS3 for security checks of the functional systems to be tested. The created test scenarios TS1, TS2, and TS3, or at least parts of the created test scenarios TS1, TS2, and TS3, can be stored in the test system or an associated database, for example, for reuse in cases of similar or identical test conditions. For this purpose, an appropriate description language (e.g., a domain-specific language) can be used for the test scenarios TS1, TS2, and TS3, just as for the abstracted test modules a1, a2, a3, a4, and a5.

[0041] Furthermore, in the synthesis step 103, each newly created test scenario TS1, TS2, TS3 can be searched for required parameters. The required parameters can be set or provided to a test module from the functional system to be tested and / or from at least one of the preceding test modules T21, T22, T23, T24, T25 during the execution of test cases TC21, TC22, TC23 derived from the test scenario TS1, TS2, TS3. Thus, for example, already in the synthesis step 103, unsuitable, non-functional, and / or unimportant test scenarios TS1, TS2, TS3 (e.g., due to the order of the abstracted test modules a1, a2, a3, a4, a5) can be recognized and eliminated. For example, the parameter list created in the abstraction step 102 can be used to quickly recognize and find the required parameters in each test scenario TS1, TS2, TS3. For this reason, the parameter lists used in the test scenarios TS1, TS2, and TS3 and linked to the abstracted test modules a1, a2, a3, a4, and a5 are used.

[0042] In the next instantiation step 104, concrete test cases TC21, TC22, and TC23 are then derived from the abstract test scenarios TS1, TS2, and TS3 for the security check of the functional system to be tested. For this purpose, information, data, and parameters of the functional system to be tested are assigned to respective placeholder variables in the abstract test modules a1, a2, a3, a4, and a5 of the test scenarios TS1, TS2, and TS3. For this purpose, the system-specific information, data, and parameters can be provided, for example, by the system manufacturer or the security check client. For this purpose, empirical data from previous security checks of identical or similar functional systems can be used instead or in addition. Similarly, such empirical data can be stored, for example, in the test system or an associated database. Alternatively or in addition, at least some of the system-specific information, data, and parameters can originate not from a database but from a machine learning system that generates the information, data, and parameters based on empirical data. Furthermore, system-specific information, data and parameters of the system to be checked (which may be assigned to abstracted placeholder variables in the abstracted test modules a1, a2, a3, a4, a5 of the test scenarios TS1, TS2, TS3) may be identified from an analysis of the actual functional system to be tested (e.g., system scan, port scan, etc.) or from obtainable manufacturer data.

[0043] After the reification step 104, the concrete test cases TC21, TC22, TC23 derived from the abstract test scenarios TS1, TS2, TS3 are applied to the functional system to be tested in an application step 105. In the application step 105, it can be documented, for example, during an ongoing security check, which concrete test cases TC21, TC22, TC23 are very well suited to finding vulnerabilities, for example, which test cases TC21, TC22, TC23 provide the best results, for example, which test cases TC21, TC22, TC23 provide less good results, for example, which test cases TC21, TC22, TC23 do not work or work exclusively abnormally on the functional system to be tested, etc.

[0044] After the application step 105, information about each test case TC21, TC22, and TC23 collected during the execution of the security check can then be transmitted to the test system in a feedback step 106. For example, feedback can be generated for each test case TC21, TC22, and TC23. For example, information for executing each test case TC21, TC22, and TC23 is included in this feedback. The feedback can then be transmitted to the test system, where it can be linked, for example, to the test scenarios TS1, TS2, and TS3 from which the test cases TC21, TC22, and TC23 were derived. The feedback can then identify test cases TC21, TC22, and TC23 that perform very well or that do not work or only work abnormally. Furthermore, for each abstract test scenario TS1, TS2, and TS3, information can be stored about which functional systems the respective test scenarios TS1, TS2, and TS3 have already been used in. Thus, when another security check of the same or a similar functional system is performed, test scenarios TS1, TS2, TS3 that have already been tried on the test system and that have worked successfully or satisfactorily can be identified in the synthesis step 103 and used again to generate test cases TC21, TC22, TC23 in the instantiation step 104. Furthermore, in this way, test scenarios TS1, TS2, TS3 that do not work (e.g., cannot be executed on the functional system due to the order of test modules, required parameters, etc.) or that are not suitable or important for the particular functional system can be very easily eliminated from the test system or the associated database. This allows the test system to be continuously improved and to learn from the security checks that have already been performed, in particular from the instantiation step 104.

[0045] 2a to 2c show in detail the abstraction step 102, the synthesis step 103, and the instantiation step 104 of the method according to an embodiment. In this embodiment, for example, a security check is performed on a real vehicle. For example, after a new development of one or more components or after an update of one or more components, a security check is performed on the real vehicle. In this case, for example, vulnerabilities or weak points of the vehicle to cyber attacks are discovered.

[0046] 2a illustrates the abstraction step 102. In this case, an example real vehicle TC1 is used as a reference, for example, a vehicle with a similar functional system (e.g., a vehicle from another manufacturer) or an identical functional system (e.g., a vehicle before a component replacement or update). The test case TC1 is, for example, a real attack vector used on a functional system that has undergone a security check. This real attack attempts to reach the functional system or the vehicle's engine control unit, for example, by attacking an external interface of the functional system (e.g., Bluetooth, radio, etc.).

[0047] The illustrated test case TC1 may be divided into four test modules T11, T12, T13, and T14 in the modularization step 101. In this case, each test module T11, T12, T13, and T14 may represent an actual attack on a component, a functional system, or a vehicle subsystem being tested. For example, the first test module T11 may be an attack on the vehicle's external interfaces. For example, the attack may discover existing external interfaces (wireless communication interfaces, sensors, etc.). After the first test module T11 has successfully executed its attack, the second test module T12 may attempt to reach the vehicle's infotainment system. For example, if the second test module T12 is successfully executed, the third test module T13 may attempt to attack or exploit the vehicle's in-vehicle network. If the third test module T13 is successful, the fourth test module T4 may attempt to take over the vehicle's engine control unit.

[0048] Next, in an abstraction step 102, concrete test modules T11, T12, T13, and T14 are abstracted, which are divisions of the concrete test case TC1. That is, in the test system, all system-specific information, data, and parameters related to the vehicle tested by the test case TC1 are removed from the test modules T11, T12, T13, and T14 and replaced with abstract placeholder variables. Abstracted test modules a1, a2, a3, and a4 that abstractly describe the respective attacks are thus obtained from the test modules T11, T12, T13, and T14. Thus, for example, the abstracted test module a1 derived from the first test module T11 may include a description of discovering a wireless interface of (any) functional system and establishing a connection with this interface. For example, the abstracted second test module a2 derived from the second test module T12 may include a description of establishing a connection with a component or subsystem of the functional system (reachable through the discovered interface) after successfully connecting with the wireless interface. For example, an abstracted third test module a3 derived from the third test module T13 abstractly describes the use of the internal network of the functional system. For example, a takeover of the control unit of the functional system can be abstractly described by an abstracted fourth test module a4. For example, these abstracted test modules a1, a2, a3, a4 can be stored in the test system or an associated data bank if they are not already stored in the test system or an associated data bank.

[0049] Furthermore, even before abstraction, test modules T11, T12, T13, and T14 can be searched for required parameters and test module-specific parameters can be created accordingly. Each test module-specific parameter list can be associated with each abstracted test module a1, a2, a3, and a4 and stored together with the respective abstracted test module a1, a2, a3, and a4 in the test system or an associated database.

[0050] 2b illustrates the synthesis step 103 of the method, based on an embodiment, such as a security check of a vehicle to be tested. In this case, abstracted test modules a1, a2, a3, a4, and a5 are combined into abstract test scenarios TS1, TS2, and TS3. Each test scenario TS1, TS2, and TS3 abstractly describes a possible attack vector. To create the test scenarios TS1, TS2, and TS3, for example, abstracted test modules a1, a2, a3, and a4 derived from a concrete test case TC1 in the abstraction step 102 and another abstracted test module a5 abstracted from a test case TC1 of another security check that has already been executed can be used. For this purpose, an abstracted test module a5 already stored in the test system or an associated database can be used.

[0051] In the synthesis step 103 shown in FIG. 2b, for example, the abstracted test modules a1, a2, a3, and a4 generated in the abstraction step 102 shown in FIG. 2a are combined into a first test scenario TS1. For example, in another second test scenario TS2, the abstracted second test module a2, the abstracted third test module a3, and the abstracted fourth test module a4 from the concrete test case TC1 are combined with an abstracted fifth test module a5 (e.g., an abstract description of how a connection to a component (e.g., a gateway) within the system will be established and / or how this connection will be used). In this case, the abstracted fifth test module a5 is inserted, for example, between the abstracted third test module a3 and the abstracted fourth test module a4. Similarly, in the third test scenario TS3, the abstracted second test module a2, the abstracted third test module a3, and the abstracted fourth test module a4 are combined with the abstracted fifth test module a5. In this case, in the third test scenario, for example, the abstracted fifth test module a5 is placed between the second test module a2 and the third test module a3.

[0052] Thus, in the synthesis step 103, abstract test scenarios TS1, TS2, and TS3 can be generated by arbitrarily combining the abstracted test modules a1, a2, a3, a4, and a5 of the test system. For example, the generated abstract test scenarios can be searched for required parameters or parameter lists stored with each of the abstracted test modules a1, a2, a3, a4, and a5 can be evaluated to exclude unsuitable or meaningless combinations of the abstracted test modules a1, a2, a3, a4, and a5—such as a combination in which the abstracted fourth test module a4 (e.g., hijacking the control unit of the functional system) is placed before the first test module a1 (e.g., discovering an external interface that is a vulnerability of the functional system). Furthermore, suggestions for combining the abstracted test modules a1, a2, a3, a4, and a5, available in the test system or an associated database for generating the abstracted test systems TS1, TS2, and TS3, can be used to generate the test scenarios TS1, TS2, and TS3. Furthermore, in the synthesis step 103, the created abstracted test scenarios TS1, TS2, TS3 or at least parts of these test scenarios may be stored in the test system or in an associated data bank - for example as proposals for creating test scenarios TS1, TS2, TS3 later in the synthesis step 103.

[0053] FIG. 2c illustrates an instantiation step 104 of a method according to an illustrative embodiment, e.g., a security check of a vehicle to be actually tested. In the instantiation step 104, concrete test cases TS1, TS2, and TS3 for a functional system to be actually tested, e.g., a vehicle with newly developed or updated components, are derived from the abstract test scenarios TS1, TS2, and TS3 created in the synthesis step 103. In this case, information, data, and parameters specific to the functional system or vehicle system to be actually tested are assigned to abstracted placeholder variables a1, a2, a3, a4, and a5 in the abstracted test modules a1, a2, a3, a4, and a5 of the respective test scenarios TS1, TS2, and TS3. Thus, a first concrete test case TC21 for a security check of a vehicle to be actually tested is generated from the first test scenario TS1. Thus, the second test scenario TS2 or the third test scenario TS3 is further made into a second concrete test case TC22 or a third concrete test case TC23. The system-specific information, data and parameters may be derived from the requester of the security check or from the vehicle manufacturer's data, for example, from experience from previous security checks of the vehicle or similar vehicles, or from an analysis of the actual vehicle to be tested, etc.

[0054] The test cases TC21, TC22, TC23 derived in the instantiation step 104 are then used for a security check of the actual functional system or vehicle to be tested in the application step 105. Then, in the feedback step 106, knowledge and experience derived from the use of the test cases TC21, TC22, TC23 can be transmitted to the test system as feedback for the test cases TC21, TC22, TC23. In the test system, the feedback can be linked to the corresponding test scenarios TS1, TS2, TS3 and stored so that the feedback can be used again during a later security check of the test system.

Claims

1. A computer-implemented method for applying test cases (TC1) for security checks of a functional system (105) to be tested of a mobility application by a test system, comprising: providing a test system with test cases (TC1) that were used during a security check of a functional system used in a mobility application and / or that can be used for such a security check; below, a step (101) of dividing a test case (TC1) into test modules (T11, . . . , T14); - abstracting the test modules (T11,...,T14), in which the information, data and parameters contained in the test modules (T11,...,T14) and specific to the system in which each test case (TC1) is used are abstracted into each test module (T11,...,T14). , . . . , T14) with abstract placeholder variables (102); - a step (103) of creating abstract test scenarios (TS1, TS2, TS3) for security checks of the functional system to be tested, in which abstracted test modules (a1, . . ., a5) are combined into one test scenario (TS1, TS2, TS3); - a step (104) of deriving test cases (TC1) for the functional system to be tested from said abstract test scenarios (TS1, TS2, TS3), in which information, data and parameters specific to the corresponding system of the functional system to be tested are assigned to placeholder variables in the abstracted test modules (a1, . . . , a5) of each test scenario (TS1, TS2, TS3); - applying (105) test cases (TC21, TC22, TC23) derived from the test scenarios (TS1, TS2, TS3) to the functional system to be tested; The method in which the above is performed.

2. feedback is generated after using test cases (TC21, TC22, TC23) derived from test scenarios (TS1, TS2, TS3) on the functional system to be tested for each test case (TC21, TC22, TC23); 2. The method of claim 1, wherein the feedback is transmitted to a test system, where each test case (TC21, TC22, TC23) is associated with each derived test scenario (TS1, TS2, TS3) (106).

3. During this abstraction, the test modules (T11, . . . T14) are searched for necessary parameters to be provided from the functional system to be tested (105) or from at least one of the preceding test modules (T11, . . . T14) in order to execute each test module (T11, . . . T14), 3. The method according to claim 1, wherein the required parameters are stored in parameter lists in the test system, and each parameter list is associated with a test module (a1, . . . , a5) abstracted from each test module (T11, . . . , T14).

4. The created abstract test scenarios (TS1, TS2, TS3) are searched for placeholder variables for the required parameters (103); To execute the test cases (TC21, TC22, TC23) derived from the test scenarios (TS1, TS2, TS3), the necessary parameters are transferred to the respective test modules (T21, TS2, TS3) from the functional system to be tested and / or from at least one of the preceding test modules (T11, . . . T14).

4. The method according to claim 1, wherein the method can be provided on a computer-readable medium (C25).

5. 5. The method according to claim 4, characterized in that a parameter list associated with each abstracted test module (a1, . . . , a5) of each test scenario (TS1, TS2, TS3) is used (103) to find placeholder variables for required parameters in each test scenario (TS1, TS2, TS3).

6. 6. The method according to claim 1, wherein abstracted test modules (a1, . . . , a5) and / or abstract test scenarios (TS1, TS2, TS3) and / or parts of the abstract test scenarios (TS1, TS2, TS3) are stored (102, 103) in a test system, in particular in a data bank.

7. 7. The method according to claim 6, characterized in that before storing (102) each abstracted test module (a1, . . . , a5), it is checked whether each abstracted test module (a1, . . . , a5) is already stored in the test system, in particular in a data bank.

8. 8. The method according to claim 1, wherein information about which functional system each test scenario (TS1, TS2, TS3) was used in is stored (103) in the test system for each abstracted test scenario (TS1, TS2, TS3).

9. 9. The method according to claim 1, wherein the system-specific information, data and parameters of the functional system to be tested are provided by a test system, in particular a database and / or by the system manufacturer, in which empirical data from previous security checks of identical or similar functional systems is stored, for assignment to placeholder variables in the abstracted test modules (a1, . . . , a5) of the respective test scenarios (TS1, TS2, TS3), and / or are determined (104) from the empirical data of previous security checks by machine learning.

10. 10. The method according to claim 1, wherein values ​​determined from an analysis of the functional system to be tested are used (104) as system-specific information, data and parameters of the functional system to be tested for assignment to placeholder variables in the abstracted test modules (a1, . . . , a5) of the respective test scenarios (TS1, TS2, TS3).

11. A test system comprising at least one computer unit adapted to carry out the method according to any one of claims 1 to 10.

12. The test system includes and / or is connected to at least one memory unit, which is an abstracted test module (a1 12. The test system according to claim 11, configured to store abstract test scenarios (TS1, TS2, TS3) and / or parts of abstract test scenarios (TS1, TS2, TS3), as well as experience points from previous security checks of the same or similar functional systems.

13. A test system according to claim 11 or 12, A computer program product comprising instructions that when executed by a test system cause the computer program product to perform the method of any one of claims 1 to 10.

Images