Computer-implementing method for scenario-based test of at least partly autonomous travel function that should be tested according to key performance indicators (KPI) and / or for homologation

Abstract

To provide a computer-implementing method for evaluating a scenario-based test of an at least partly autonomous travel function that should be tested according to key performance indicators, and / or simulation and / or a test case for homologation, and to provide a test unit and a computer readable data carrier.SOLUTION: KPIs are mapped according to plugins, and the KPI plugins are selected dynamically and to be reusable for simulation and / or a test case. Further, at least one KPI plugin is selected using a KPI plugin mechanism in simulating and / or defining a test, and automatically executed according to the KPI plugin mechanism during execution.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present invention relates to a computer-implemented method for scenario-based testing and / or homologation of at least partially autonomous driving functions to be tested according to key performance indicators (KPIs).

[0002] The present invention further relates to a test unit for scenario-based testing and / or homologation of at least partially autonomous driving functions to be tested according to key performance indicators (KPIs). The present invention furthermore relates to a computer program and a computer-readable data carrier.

Background Art

[0003] For example, driving assistance systems such as adaptive cruise control and / or functions for highly automated or autonomous driving can be verified or validated using various inspection methods. Here, in particular, the hardware-in-the-loop method, the software-in-the-loop method, simulation and / or test drives can be used.

[0004] The effort, in particular the time cost and / or expense, for testing this type of vehicle function using the above inspection methods is, in this case typically, very high because it is necessary to test a large number of driving situations that may potentially occur.

[0005] Testing at least partially autonomous means of transportation exclusively on roads with driving sections of several billions of kilometers is impossible in terms of time and cost. Furthermore, even if many redundant test kilometers occur, critical and abnormal situations related to the capabilities of at least partially autonomous vehicles will not occur.

[0006] This requires considerable effort, not only for test drives but also for simulations. German Patent Application Publication No. 102017200180 describes a method for verifying and / or validating vehicle functions aimed at autonomously guiding a vehicle longitudinally and / or laterally.

[0007] This method includes the step of calculating test control commands for vehicle functions to the vehicle's actuators based on ambient environment data relating to the vehicle's surrounding environment, wherein the test control commands are not executed by the actuators.

[0008] This method further includes the step of simulating a hypothetical traffic situation that would exist if a test control command were implemented, using a road user model for at least one road user in the vehicle's surrounding environment, based on surrounding environment data.

[0009] This method further includes a step of providing test data related to hypothetical traffic conditions. Vehicle functions, in this case, operate passively within the vehicle to calculate test control commands.

[0010] A drawback of this method is that actual vehicle operation is required to determine the necessary data, verify and / or validate vehicle functions, and calculate the required data.

[0011] Manufacturers of at least partially autonomous modes of transport require parallelized, simulation-based solutions. This involves a testing methodology where "scenario-based testing is performed." For this purpose, intelligent test-follow-the-moment control is needed to evaluate simulation results and, if necessary, modify parameterizations.

[0012] Therefore, the object of the present invention is to present a method, test unit, computer program, and computer-readable data carrier that demonstrates how to evaluate simulations and / or test cases in scenario-based testing and / or homologation of at least partially autonomous driving functions to be tested using key performance indicators (KPIs). In a preferred embodiment of the present invention, the KPIs are mapped by KPI plugins, which are dynamically and reusably selected for simulations and / or test cases. At least one KPI plugin is selected by the KPI plugin mechanism during simulation and / or test definition and is automatically executed by the KPI plugin mechanism during execution. [Overview of the project] [Means for solving the problem]

[0013] This problem is solved by the present invention with a computer implementation method for evaluating simulations and / or test cases in scenario-based testing and / or homologation of at least partially autonomous driving functions to be tested by key performance indicators (KPIs) according to claim 1, a test unit according to claim 12, a computer program according to claim 14, and a computer-readable data carrier according to claim 15.

[0014] Autonomous vehicles include multiple control devices. Each individual control device and its combination must be extensively tested during development and homologation. Scenario-based testing is conducted to ensure the error-free functional characteristics of the control devices under all traffic conditions. In scenario-based testing, the vehicle's driving characteristics are analyzed in traffic conditions as close to reality as possible. The type of traffic condition to be analyzed and its evaluation depend on the system being tested. Therefore, when conducting scenario-based testing of systems and system components for autonomous driving of vehicles, scenarios are defined that can abstractly represent traffic conditions. This allows test cases to be run again for each scenario. A logical scenario, in this case, defines an abstraction of traffic conditions, including the road, driving style, and surrounding traffic, without specific parameter values. By selecting specific parameter values, a logical scenario becomes a specific scenario. Each such specific scenario corresponds to an individual traffic condition.

[0015] For the basic distinction of traffic scenarios or scenarios according to the present invention in scenario-based testing, not only static parameters such as the surrounding environment, buildings, or lane width (but not limited to these) are used, but also, in particular, the driving style of individual road users. The motion of road users and the accompanying driving style are described by a trajectory. The trajectory describes not only the path in the spatial direction but also the path in the temporal direction. For example, the motion of road users can be distinguished by parameters such as speed.

[0016] Autonomous driving capabilities are realized by systems such as control devices. Conventionally, these control devices are tested using hardware-in-the-loop testing in actual vehicles under real traffic conditions, or alternatively, validated through complete virtual testing. Therefore, simulation can also be used.

[0017] The method according to the present invention allows, for example, a so-called cut-in scenario to be distinguished from other scenarios. This cut-in scenario can be described as a traffic situation in which a highly automated or autonomous vehicle is traveling in a predetermined lane, and another vehicle traveling at a slower speed than the vehicle enters the vehicle's lane from a further lane at a predetermined distance. In this case, the vehicle being tested (SUT) represents the vehicle to be tested.

[0018] The speeds of your vehicle and any other vehicles (hereinafter referred to as "other vehicles") are constant in this scenario. Since your vehicle is faster than the other vehicles, you need to brake to avoid a collision between the two vehicles.

[0019] However, cut-in scenarios may exist in various forms of expression, such as differences in the speeds of road users.

[0020] Furthermore, a passing scenario is also possible, where the vehicle is the one being overtaken. In this scenario, the vehicle travels in a predetermined lane and overtakes other vehicles that are traveling at a slower speed. At that time, the vehicle changes lanes to another lane and overtakes the other vehicle at a faster speed. The speed of the vehicle does not need to be constant in this scenario. After the vehicle has overtaken the other vehicle, the vehicle returns to its original lane.

[0021] Such scenarios can be simulated with different parameter values. To use simulation and computation time as efficiently as possible, high coverage, i.e., test-following control to achieve homologation, is necessary. This requires not only a proper evaluation of the simulation results but, in some cases, a proper evaluation of each simulation step.

[0022] The term Key Performance Indicator (KPI) in the method according to the present invention refers to a KPI value that can be measured and / or calculated after or during a simulation of at least partially autonomous vehicles, representing progress or satisfaction with important objectives or critical success factors. The KPI and / or KPI value can be used to evaluate the simulation and / or simulation steps for test follow-up control, thereby enabling testing of desired resource efficiency and time efficiency.

[0023] To this end, according to the present invention, the KPI is implemented as a KPI plugin. The KPI plugin includes an executable script or a KPI script and a configuration description. The executable script records how the KPI value is calculated, and the configuration description indicates at least what input data the script requires and how it must be executed, and possibly additional metadata describing which scenarios the KPI plugin is applicable to. The metadata indicating the applicability of the KPI plugin in a scenario may be, for example, scenario parameters such as a specific lane width, or conditions on driving condition parameters such as the number of road users or speed limits.

[0024] As a result, the KPI plugin becomes reusable in various scenarios and / or tests. In contrast, there is no need to adapt the scripts and / or configuration files to be executed for use in different scenarios. Therefore, the KPI plugin can be dynamically selected and integrated into tests and / or simulations, such as during test creation and / or simulation execution, and thus can be used in different simulations and / or tests for different scenarios. This selection can be made before and / or after test and / or simulation creation according to the present invention. The selection of the plugin can be changed according to embodiments of the present invention. In a preferred embodiment, the selection of the KPI plugin is possible by the test / simulation creator in the user interface. In a further embodiment, programmable incorporation of the KPI plugin is also possible. A further embodiment is intended to be included together with the computer-implemented method according to the present invention. According to the computer-implemented method according to the present invention, the execution time of the simulation, including simulation creation, is shortened.

[0025] A further problem of the present invention is to automatically execute the execution of at least one selected KPI plugin. For this purpose, a KPI plugin mechanism is provided. Together with test definition and / or simulation creation, the KPI plugin mechanism receives the KPI plugin. Subsequently, the KPI plugin mechanism holds the KPI plugin and executes it at the assumed location. The execution of the KPI plugin supplies a KPI value, and this KPI value is stored in the test result and / or simulation result by the KPI plugin mechanism after execution.

[0026] The KPI plugin mechanism is provided to execute the KPI plugin in two different deformed forms. This deformed embodiment is defined in the configuration file.

[0027] The first variant embodiment is performed online and thus simultaneously with the simulation. In this case, the evaluation is performed at each simulation step and the KPI values are available immediately after the simulation. This evaluation is based on simulation parameters that are available during execution, such as scenario parameters and / or operating condition parameters.

[0028] The second variant embodiment is executed downstream of the current simulation, i.e., offline. Thus, the KPI values are determined only after the simulation, rather than real-time parameters being considered and utilized for calculating the KPI values during the simulation. For the determination of the KPI values, the simulation results are passed to a KPI script.

[0029] Basically, the KPI plug-in mechanism can hold at least one KPI plug-in, but it is also possible to hold multiple KPI plug-ins. All KPI plug-ins recorded in the KPI plug-in mechanism are executed for test execution and / or simulation.

[0030] To improve reusability, all KPI plug-ins are managed in a KPI plug-in pool.

[0031] In contrast to known methods of integrating evaluation into scenario-based testing, the KPI plug-in mechanism manages the KPI plug-ins and their execution. Thereby, the KPI plug-ins are not fixedly coupled to the simulation and do not need to be redefined for each scenario or SUT. In known methods, a coupling of the evaluation and the scenario is required. In addition to reusability, the automatic execution of the KPI plug-ins is prominent in the method according to the invention. At runtime, the KPI plug-in mechanism automatically loads, inspects, and executes the selected KPI plug-ins.

[0032] The KPI plugin mechanism requires standardized KPI settings and a standardized interface for KPI scripts. Predefined criteria allow for the separation of KPI development and scenario development.

[0033] Further embodiments of the present invention are subject to further dependent claims and the following description with reference to the drawings.

[0034] The test unit includes means for evaluating simulations and / or test cases in scenario-based testing and / or homologation of at least partially autonomous driving functions to be tested according to key performance indicators (KPIs).

[0035] A further aspect of the present invention is envisioned in which a computer program is included that contains program code for carrying out the method according to the present invention when the computer program is executed on a computer. A further aspect of the present invention is envisioned in which a data carrier is included that contains program code for a computer program for carrying out the method according to the present invention when the computer program is executed on a computer.

[0036] The computer implementation methods described herein can be used to evaluate simulations and / or test cases in scenario-based testing and / or homologation of at least partially autonomous driving functions to be tested using key performance indicators (KPIs). Similarly, the test units according to the present invention are suitable for performing test follow-up control of multiple different devices or control equipment in, for example, automobiles, commercial vehicles and / or commercial vehicles, ships or aircraft, by evaluating simulation results and / or simulation steps for different scenarios or driving conditions.

[0037] Next, in order to better understand the present invention and its advantages, the following description will be referenced in conjunction with the accompanying drawings. The present invention will be described in more detail below based on exemplary embodiments shown in the schematic drawings. [Brief explanation of the drawing]

[0038] [Figure 1] This is a schematic diagram for distinguishing the scenarios according to the present invention. [Figure 2] This is a schematic diagram illustrating the boundary between critical and non-critical test results. [Figure 3] This figure illustrates the KPI plugin according to the present invention. [Figure 4] This is a schematic diagram illustrating the KPI plugin mechanism and KPI plugin pool according to the present invention. [Figure 5] This is a schematic diagram illustrating the use of the KPI plugin according to the present invention. [Figure 6] This is a schematic diagram illustrating the use of the KPI plugin according to the present invention. [Figure 7] This is a schematic representation of how to determine KPI values ​​according to the present invention. [Figure 8] This diagram shows the offline usage flow of the KPI plugin mechanism according to the present invention. [Figure 9] This diagram illustrates the online usage flow of the KPI plugin mechanism according to the present invention. [Modes for carrying out the invention]

[0039] Figure 1 illustrates two different scenarios, S1 and S2, each showing a single intersection area. Both scenarios, S1 and S2, depict the vehicle itself (Ego). In S1, the vehicle (Ego) performs a turning maneuver. This vehicle is also referred to as the "Subject Under Test" (SUT). Four other vehicles (F1-F4) are also involved. In S2, the vehicle follows a straight course without the involvement of other vehicles. Therefore, there are differences in both the scenario parameters and the driving condition parameters. The purpose of these scenarios may, for example, be to test and simulate adaptive cruise control. When defining a test, the KPI plugin mechanism receives a KPI plugin for evaluating adaptive cruise control, consisting of a script and a configuration description. During and / or after the simulation, the defined KPI script is executed, and the resulting KPI values ​​are stored. A common KPI plugin can be used for both scenarios.

[0040] In further embodiments, adaptive cruise control can be tested in the vehicle itself (Ego) in highway and / or urban scenarios. In particular, the entry process (cut-in scenario) may be considered. Therefore, in this case, these scenarios differ, especially in terms of scenario parameters.

[0041] First, a highway scenario is selected in the test setup. In this scenario, the important question is whether or not a collision occurred for the vehicle (Ego). Additionally, the severity of the collision should be determined in terms of the collision speed. If no collision occurs, the minimum distance between the vehicle (Ego) and an approaching vehicle detected by adaptive cruise control is considered. For this setup, a KPI plugin with a specific value range can be defined. The concrete implementation of the KPI plugin is done in the KPI script. The script contains a setting description that describes the evaluation and lists the necessary simulation information. This KPI plugin can be implemented as an online or offline KPI plugin. When defining the test, the KPI plugin mechanism receives the KPI plugin consisting of the script and the setting description. During and / or after the simulation, the defined KPI script is executed, and the resulting KPI values ​​are stored. After running the highway scenario test, the vehicle (Ego) can also be tested in a second scenario, for example, in an urban area. In this case, the necessary simulation information exists in the KPI plugin, and therefore, this KPI plugin can be selected again when defining the test. This reusability is provided by the use of the KPI plug-in mechanism. In addition to the KPI plug-in written for testing adaptive cruise control, further KPI plug-ins are available, for example, to evaluate vehicle comfort or fuel consumption during test drives and / or simulations.

[0042] Figure 2 shows a function that indicates the boundary between critical and non-critical test results. The point cloud in the figure represents simulated test results. Alternatively, they may be approximate test results.

[0043] The illustrated function is the safety objective function, which in a preferred embodiment has the following numerical value: that is, the safe distance between the vehicle itself (Ego) and another vehicle (another automobile) is ≥ V FELLOWThe minimum value occurs when the multiplier is 0.55, the maximum value occurs when a collision occurs between the vehicle (Ego) and another vehicle, and the safe distance between the vehicle and the other vehicle is ≤ V FELLOW This refers to a value greater than the minimum value when multiplied by 0.55. Such a safety objective function may be part of the KPI script and can be used to determine KPI values.

[0044] As an alternative to the safety objective function, a comfort or efficiency objective function can be simulated and / or approximated by having a numerical value that has a minimum value when the vehicle's acceleration is constant, a maximum value when a collision occurs between the vehicle (Ego) and another vehicle, and a value between the minimum and maximum depending on the change in acceleration when the vehicle's (Ego) acceleration changes. Multiple driving condition parameters, in particular the speed V of the vehicle (Ego), can be used. EGO and further automobiles, i.e., the speed V of other vehicles FELLOW It is generated, for example, by simulation, within a predetermined defined range.

[0045] For evaluation purposes, such objective functions can be integrated into the KPI plugin and selected in the test and / or simulation definitions. This KPI plugin can then be run directly in the simulation, thereby determining the KPI values.

[0046] Figure 3 shows a depiction of the KPI plugin (KPI-PI) according to the present invention. The KPI plugin (KPI-PI) has an interface to the KPI plugin mechanism. Furthermore, the KPI plugin (KPI-PI) consists of a configuration file (C) and an executable script (S).

[0047] The configuration file (C) contains information about usage, specifically, information about the types of scenarios in which evaluation is possible. This information may include scenario parameters and / or driving condition parameters, in particular, lane width and / or number of road users and / or speed limits in the scenario. This enumeration is not a conclusion, but merely to clarify the type of information. Additionally, the configuration file (C) determines whether the KPI plugin (KPI-PI) can be used online, i.e., during the simulation, or offline, i.e., after the simulation.

[0048] The execution script (S) contains information for determining the KPI values ​​used to evaluate the simulation and / or simulation steps.

[0049] Figure 4 shows a schematic diagram illustrating the KPI Plugin Mechanism (KPI-PIM) and KPI Plugin Pool (KPI-PIP) according to the present invention.

[0050] The KPI Plugin Pool (KPI-PIP) collects and manages all available KPI plugins (KPI-PIs). This includes both currently used and unused KPI plugins (KPI-PIs). All known KPI plugins (KPI-PIs) can be viewed through the KPI Plugin Pool (KPI-PIP).

[0051] The KPI Plugin Mechanism (KPI-PIM) controls the automated execution of selected KPI plugins (KPI-PIs). In contrast to the common method of integrating evaluation into scenario-based testing, thereby binding it to the scenario itself, the KPI Plugin Mechanism (KPI-PIM) manages KPI plugins (KPI-PIs) and their execution. This means that KPIs are not fixedly tied to the simulation and do not need to be redefined for each scenario or SUT; rather, they are induced as KPI plugins (KPI-PIs), managed in the KPI Plugin Pool (KPI-PIP), and executed by the KPI Plugin Mechanism (KPI-PIM). This ensures the reusability of KPI plugins (KPI-PIs), and the automated execution of KPI plugins (KPI-PIs) results in significant time and resource savings. During execution, the KPI Plugin Mechanism (KPI-PIM) automatically loads, inspects, and executes the selected KPI plugins.

[0052] Figure 5 shows a schematic diagram illustrating the use of the KPI Plugin (KPI-PI) according to the present invention in a specific scenario, and therefore in a defined simulation / defined test.

[0053] Therefore, Figure 5 shows a KPI Plugin Mechanism (KPI-PIM) with two KPI Plugins (KPI-PI) KPI-P-I1 and KPI-P-I2. This KPI Plugin Mechanism (KPI-PIM) is shown in relation to Test T1 in Scenario S1.

[0054] Here, the KPI plugins (KPI-PI) KPI-P-I1 and KPI-P-I2 are not directly linked to the scenario. Due to their generic and dynamic definitions, the KPI plugins (KPI-PI) can be reused for various scenarios and tests. Therefore, a new KPI plugin mechanism (KPI-PIM) or a new KPI plugin (KPI-PI) is not created for each new scenario.

[0055] Figure 6 also shows a schematic diagram illustrating the use of the KPI plugin (KPI-PI) according to the present invention in a specific scenario, and therefore a defined simulation / test.

[0056] Here, the KPI Plugin Mechanism (KPI-PIM) is shown in relation to test T2 in scenario S2. Figure 6 shows the KPI Plugin Mechanism (KPI-PIM) with KPI plugins (KPI-PI) KPI-P-I1 and KPI-P-I3. These KPI plugins (KPI-PI) are not directly coupled to the scenario or test. This makes it clear that the KPI plugin (KPI-PI) KPI-P-I1 can be used in both S1, as shown in Figure 5, and S2, as shown in Figure 6. This reuse is achieved without adapting the corresponding KPI plugin and the script to which the KPI plugin belongs.

[0057] Figure 7 shows a schematic diagram for determining KPI values ​​according to the present invention. When a collision (VC) is associated with a particular SUT in a scenario, it is necessary to select a KPI plugin (KPI-PI), such as a plugin for evaluating the safety of the driving conditions (safety KPI). This KPI plugin (KPI-PI) should, in particular, be able to determine the collision speed (IV) when a collision (VC) occurs, or, if no collision (VC) occurs, the minimum distance (Min D) between the vehicle (Ego) and another vehicle should be indicated. These results are particularly relevant to evaluating the scenario in cut-in scenarios where, for example, adaptive cruise control should be tested. The calculated KPI values ​​can be used as a basis for further parameterizing the simulation.

[0058] Figure 8 shows the offline usage flow of the KPI Plugin Mechanism (KPI-PIM) according to the present invention.

[0059] KPIs in the form of KPI plugins (KPI-PI) can be executed online or offline by the KPI Plugin Mechanism (KPI-PIM). In the offline case, KPI execution, i.e., KPI plugin (KPI-PI) execution, can be performed immediately after simulation by the KPI Plugin Mechanism (KPI-PIM). KPI execution at subsequent points in time is also possible.

[0060] First, the job or test case is configured (CJ). This configuration is used in the test execution (EJ) and as input for the simulation (Sim). The simulation (Sim) enables the KPI plug-in mechanism (KPI-PIM) and generates test results (TR) that are used when executing the selected KPI plug-in (KPI-PI).

[0061] The results of the KPI Plugin Mechanism (KPI-PIM) in the form of KPI values ​​can, in a preferred embodiment, be passed back to the Test Configuration (CJ) and linked to an optimized test configuration. It is also possible to use the KPI values ​​for better parameterization, thereby achieving increased test coverage.

[0062] Figure 9 shows the flow of online use of the KPI Plugin Mechanism (KPI-PIM) according to the present invention.

[0063] Therefore, in a preferred embodiment, information from the simulation (Sim) is passed to a KPI plug-in mechanism (KPI-PIM) after each simulation step. The KPI plug-in mechanism (KPI-PIM) executes the selected KPI plug-in (KPI-PI), and the KPI values ​​are passed directly back to the simulation (Sim). The results (TR), including the KPI values, are transmitted together to the test creation and configuration (CJ), which can preferably be used to generate optimized test cases and / or optimized parameter settings.

[0064] The KPI plugin (KPI-PI) can additionally be used as an interruption condition for the simulation (Sim). In a further preferred embodiment, if the KPI value generated by the KPI plugin mechanism (KPI-PIM) should exceed a threshold defined in the configuration, the simulation interruption can be triggered by the transmission of the KPI value in the next simulation step. The threshold may be included in the test definition and / or in the KPI plugin (KPI-PI) configuration file (C).

Claims

1. 1. A computer-implemented method for evaluating simulations and / or test cases in a scenario-based testing and / or homologation of an at least partially autonomous driving function to be tested by key performance indicators (KPIs), comprising: KPIs are mapped by KPI plugins, which are dynamically and reusably selected for a simulation and / or test case; At least one KPI plugin is selected by the KPI plugin mechanism during simulation and / or test definition and is automatically executed by said KPI plugin mechanism during execution. Computer-implemented method.

2. At least one scenario includes at least one test of an apparatus for at least partially autonomous guidance of vehicles and / or road users, 2. The computer-implemented method of claim 1.

3. The at least one test is determined by at least one parameter interrogated during the simulation run and / or at the end of the simulation, said parameter being in particular - scenario parameters including at least one of the following characteristics: number of lanes and / or width and / or curvature and / or road regulations and / or ambient temperature; - driving situation parameters describing the number and characteristics of the moving objects in the scenario, including at least one of the following characteristics: the number of road users and / or the number of lane changes in the traffic situation and / or the speed of the road users and / or the means of transport; Including, 3. The computer-implemented method of claim 2.

4. The means of transport and / or road users include at least an ego vehicle and / or other vehicles, the ego vehicle being the vehicle having the system to be tested (SUT) and the other vehicles being any further vehicles in the scenario, 3. The computer-implemented method of claim 2.

5. A KPI plugin consists of a configuration description and a script that can be executed by a computer.

2. The computer-implemented method of claim 1.

6. The KPI plugin can be run online during a simulation and / or offline after the simulation; A computer-implemented method according to any one of claims 1 to 5.

7. For KPI plugins executed online, the executable script is started automatically at simulation run time and an evaluation of at least one parameter is performed at each simulation step, whereby the evaluation is based on simulation parameters available at run time and a KPI value is determined therefrom.

7. The computer-implemented method of claim 6.

8. For KPI plugins that are executed offline, the executable script is started automatically after a simulation is completed, whereby the KPI values ​​are determined based on the simulation results.

7. The computer-implemented method of claim 6.

9. the currently determined KPI values ​​are indicative of an evaluation of a current simulation and / or at least one simulation step; A computer-implemented method according to any one of claims 1 to 5.

10. The KPI plugin mechanism includes at least one KPI plugin, and all KPI plugins selected in the KPI plugin mechanism are automatically executed for simulation execution and / or test execution. A computer-implemented method according to any one of claims 1 to 5.

11. KPI plugins are managed in the KPI plugin pool. A computer-implemented method according to any one of claims 1 to 5.

12. 1. A test unit of a computer-implemented method for evaluating simulations and / or test cases in a scenario-based test and / or homologation of an at least partially autonomous driving function to be tested according to key performance indicators (KPIs), comprising: KPIs are mapped by KPI plugins, which are dynamically and reusably selected for a simulation and / or test case; At least one KPI plugin is selected by the KPI plugin mechanism during simulation and / or test definition and is automatically executed by said KPI plugin mechanism during execution. Test unit.

13. the test unit being constituted by a control device, in which at least one scenario for a virtual test and / or a real test is determined; 13. The test unit according to claim 12.

14. A computer program comprising: The computer program comprises a program code for carrying out the method according to any one of claims 1 to 5 when the computer program is run on a computer. Computer program.

15. A computer readable data carrier comprising: A computer program comprising a program code for carrying out the method according to any one of claims 1 to 5 when the computer program is run on a computer. A computer readable data carrier.

Images