Method, computer program and device for testing an assistance system

Abstract

The invention relates to a method for testing an assistance system for at least partially automated driving functions of a vehicle by means of a scenario-based test simulation, wherein the scenario-based test simulation simulates an ego vehicle equipped with the assistance system to be tested and at least one surrounding vehicle according to a scenario, taking into account outputs of the assistance system to be tested of the ego vehicle.

Description

[0001] The invention relates to a method for testing an assistance system for at least partially automated driving functions of a vehicle by means of a scenario-based test simulation, wherein the scenario-based test simulation simulates an ego vehicle equipped with the assistance system to be tested and at least one surrounding vehicle according to a scenario, taking into account outputs of the assistance system to be tested of the ego vehicle.

[0002] The invention also relates to a computer program for performing the test simulation for this purpose.

[0003] The invention also relates to a device with a data processing system for performing the test simulation for this purpose.

[0004] With the ongoing development of driver assistance systems, or assistance systems for short, it becomes possible to further automate driving tasks, particularly with regard to longitudinal and lateral control, and thus relieve the driver. One goal is to make traffic safer and more comfortable overall through the development of autonomous or semi-autonomous vehicles.

[0005] For this purpose, one or more driver assistance systems are installed in the vehicles, designed to automate longitudinal and / or lateral control tasks. These systems can, for example, automatically maintain a safe distance from the vehicle ahead, drive autonomously in a designated lane, initiate overtaking maneuvers independently, or brake automatically if the assistance system detects a dangerous situation that could lead to an accident. Such assistance systems are therefore not only systems that can take over driver tasks, but also systems that perform safety functions and tasks to prevent accidents and the associated damage to property and persons.

[0006] Such driver assistance systems typically receive a wealth of data and information to develop a kind of artificial situational awareness for the current traffic situation. The assistance system is continuously supplied with sensor data from the vehicle's sensors, which is used to build this artificial situational awareness. This sensor data can include, for example, radar data, lidar data, image data, and similar information used to digitally capture the vehicle's surroundings. Based on this environmental data, the assistance system then performs its intended control tasks.The output of the assistance system varies depending on the type of assistance system and the level of automation, ranging from purely informative data that can be presented to the driver inside the vehicle (auditory, visual and / or haptic) to control signals for intervention in the vehicle control system, for example to effect steering intervention or acceleration or deceleration of the vehicle.

[0007] Scenario-based methods are often used for the development, testing, and, if necessary, certification of driver assistance systems that take over at least partially automated driving functions or tasks of a vehicle. Due to the large number of specific test scenarios required and the complex test setup, it is desirable to test the assistance system using a simulation rather than with the vehicle itself. In this simulation, a specific scenario is simulated, and corresponding inputs are generated for the assistance system, to which the system responds with an output. In such a scenario, the vehicle maneuvers are precisely defined, and the behavior of the assistance system is analyzed during the test and, if necessary, evaluated based on predefined parameters (e.g., criticality metrics).The insights gained in this way are then used to improve, develop and / or approve the assistance system.

[0008] From DE 10 20 2020 005 507 A1, a method for testing an automated driving function of a vehicle is known, in which a confidence level for existing field tests and a confidence measure for the driving function to be tested are determined over an entire effective range of an operational design domain, wherein measurement data from country-specific field tests are recorded and stored centrally in a database, subsequently knowledge is extracted from this database by means of an event-driven time series analysis with clustering, wherein extracted clusters and their parameter space define a probability of occurrence of each logical scenario and a probability distribution of associated parameters, a sensitivity and reliability analysis identifies a failure region in the parameter space in order to predict a probability of failure for each logical scenario using sampling procedures.where traffic hotspot categories of false positive and false negative events from various information sources are transformed in cluster-analytically characterized driving situations using a database of scenarios to be tested.

[0009] From DE 10 2022 129 009 A1, it is known to create a model that can be used to test different scenarios. The model is designed to include commonalities among the scenarios to be tested. These can be individual trajectories of participating road users or specific characteristics of the environment. Instead of creating a model for the relationship between input parameters and key indicators, a new model is created that is capable of predicting the trajectory of an ego-vehicle independently of the input parameters of the given scenario. This model is trained using machine learning methods.

[0010] German patent DE 10 2023 100 858 A1 discloses an improved evaluation of the performance and capabilities of at least one driving function of an Advanced Driver Assistance System (ADAS) and / or an Automated Driving System (ADS) for a defined driving task in at least one scenario for testing, applying, and validating the ADAS / ADS system. This involves adapting a quality loss function using indirect KPIs and a criticality index for a test case, so that simulated measurements for critical test cases can receive a good rating.

[0011] The variety of scenarios to be considered, particularly for driver assistance systems in highly automated driving functions, is very large and becomes increasingly complex with increasing levels of automation. Therefore, so-called abstract scenarios are used, especially in the development of highly automated driving functions. These describe test scenarios at a high level of abstraction. Concrete scenarios can then be generated from these as needed, assigning a position to each vehicle in the scenario at any given time and thus precisely describing the maneuvers performed.

[0012] The creation of such concrete scenarios from abstract scenarios is known, for example, from Jan Steffen Becker et al: “Simulation of abstract scenarios: Towards automated tooling in criticality analysis”, 2022, and from Jan Steffen Becker: “Safe linear encoding of vehicle dynamics for the instantiation of abstract scenarios”, Int. Conf. on Formal Methods for Industrial Critical Systems (FMICS), 2024.

[0013] Scenarios generated in this way from abstract scenarios or extracted from recorded traffic data make implicit assumptions about the exact behavior of the SuT (System under Test), i.e., the driver assistance system. Even small deviations of the assistance system regarding selected speeds during the test can fundamentally lead to different traffic situations, for example, by changing important distances relative to other road users or infrastructure. For the actual execution of the scenario, the simulation must therefore react to the output of the assistance system (SuT), thus forming a closed simulation loop. This is potentially possible with logical scenarios where the actions can be parameterized and assigned probability distributions.However, in concrete scenarios, which can be generated from abstract scenarios and thus have a higher level of abstraction than logical scenarios, such a closed control loop reaches its limits, since the concrete scenarios describe the behavior of the environment of the SuT absolutely and therefore no reaction to the behavior of the SuT is possible.

[0014] This very adaptation is necessary, however, to ensure that the concrete scenario derived from the abstract scenario adheres to the abstract scenario's intent and the purpose of the simulation. For example, if the simulation's reaction to a suddenly crossing road user is to be tested, the simulation must ensure that the other road user enters the intersection at the correct time (e.g., when the ego vehicle reaches a predetermined distance from the intersection). On the other hand, this adaptation to the ego vehicle's behavior must be selective. If, for example, the simulation does not react to the simulated road user with braking as intended, this must not lead to an evasive maneuver by the simulated road user, thereby unintentionally defusing the situation and rendering the reaction unusable for the test evaluation.

[0015] It is therefore an object of the present invention to provide an improved method for testing assistance systems for at least partially automated driving functions, with which scenarios formed from abstract scenarios can be used in particular.

[0016] The problem is solved according to the invention using the method according to claim 1. Advantageous embodiments of the invention are described in the corresponding dependent claims.

[0017] According to claim 1, a method for testing an assistance system for at least partially automated driving functions of a vehicle is proposed using a scenario-based test simulation, wherein the scenario-based test simulation simulates an ego vehicle equipped with the assistance system to be tested and at least one surrounding vehicle according to a scenario, taking into account outputs of the assistance system to be tested of the ego vehicle.

[0018] Using scenario-based test simulation, preferably executed as a computer program on a data processing system, the assistance system under test is tested in a preferably concrete scenario that is not a logical scenario and is preferably derived from an abstract scenario. The concrete scenario is an instance of the abstract scenario, such that, for example, each scene corresponds to one of the situations in the abstract scenario, while maintaining the sequence of situations and their temporal properties. The concrete scenario is characterized by the fact that it uniquely defines the state of each road user in the scenario at every point in time.

[0019] The responses of the assistance system being tested are fed back into the test simulation, so that after the scenario is completed, the responses of the assistance system are evaluated and incorporated into the development, improvement, and / or approval / certification of the assistance system. The assistance system can be in the form of hardware or a computer program.

[0020] According to the invention, the method comprises the following steps, which are carried out on a data processing system by means of the test simulation: - Obtaining a first control vector to control at least one surrounding vehicle, - Obtaining a second control vector for controlling the ego vehicle from an output of the assistance system under test, - Determining a current simulation state for a simulation time step by the test simulation as a function of the first control vector and the second control vector, wherein the simulation state includes the states of the at least one surrounding vehicle and the ego vehicle, - Providing the simulation state at an assistance system interface of the test simulation in order to input the current, simulated simulation state into the assistance system under test as input data, and - Determining a next first control vector for determining a next simulation state in a next simulation time step such that a deviation between the current simulation state and a reference scenario is minimized.

[0021] The test simulation has a simulator that calculates a simulation state in predefined (and potentially configurable) simulation time steps. To calculate the current simulation state, the simulator receives a first control vector, which defines the control of at least one surrounding vehicle, and a second control vector, which defines the control of the ego vehicle as output from the assistance system under test. The assistance system under test is connected to the test simulation simulator via a corresponding assistance interface, so that the output of the assistance system is taken into account when calculating the current simulation state.

[0022] The calculated simulation state, and thus also the state of the ego vehicle, is transferred as input to the assistance system in order to generate a reaction from the assistance system and to consider the reaction as output when calculating the next simulation state.

[0023] The simulation state includes the states of at least one surrounding vehicle and the ego vehicle, such as (absolute) position, speed, position change, and possibly also virtual sensor data for the assistance system, which would be generated virtually by the ego vehicle based on the current simulation progress of the scenario.

[0024] To calculate the next simulation state, which is to follow the current simulation state in the next simulation time step, at least the first control vector for the surrounding vehicles must be recalculated. This is necessary to react to the output of the assistance system, which was considered in the current simulation state by the second control vector, and, if necessary, to adjust the movement of the surrounding vehicles accordingly. However, this must not be done in such a way that the actual test objective or purpose is lost.

[0025] According to the invention, it is proposed that the next first control vector for determining the next simulation state in the next simulation time step is determined in such a way that any deviation between the current simulation state calculated for the current simulation time step and a reference scenario is minimized. The next simulation time step follows the current simulation time step for which the current simulation state was calculated. Thus, in the next simulation time step, the next simulation state following the current simulation state is calculated.

[0026] The specific scenario that forms the basis for the test simulation is therefore a reference scenario that forms the basis of a (numerical) optimization procedure, so that for the calculation of the next simulation time step those specific controls of the surrounding vehicles are calculated that lead to a deviation (distance) between the current simulation state and the reference scenario being minimized.

[0027] This makes it possible to control the simulation, or more precisely, the vehicles in the simulation, so that the simulation's progression corresponds to the abstract scenario. For this purpose, the concrete scenario, which is an instance of the abstract scenario, is used as a reference scenario.

[0028] According to one embodiment, the steps are repeated multiple times until the end of the reference scenario is reached.

[0029] The test simulation can begin, for example, with predefined control vectors for both the surrounding vehicles and the player vehicle. These can also be randomly generated within a predefined area, which could be part of the abstract or concrete scenario.

[0030] Once started, the simulation cycle generates a current simulation state for each simulation time step and, by determining the first control vector for controlling the surrounding vehicles, adapts the next simulation state to the abstract scenario by minimizing the deviation. In the reference scenario, time is also advanced depending on a simulation time step (which may vary over the entire simulation), so that the current simulation time step is also compared with the corresponding scene within the reference scenario.

[0031] Once the end of the reference scenario is reached, the simulation cycle ends after the corresponding x simulation time steps.

[0032] According to one embodiment, the next first control vector is determined in such a way as to minimize any deviation in the positions of the surrounding vehicles between the current simulation state and a reference scenario.

[0033] The first control vector for controlling the surrounding vehicles is therefore calculated in such a way that the movement of the surrounding vehicles leads to a position that corresponds to the positions specified in the reference scenario and any deviation from this is minimized by solving the optimization procedure.

[0034] According to one embodiment, it is provided that a time progression for the reference scenario is determined in such a way that the deviation between the current simulation state and a reference scenario is minimized.

[0035] This can lead to a discrepancy between the time units in the simulation and the time units in the reference scenario, so that the reference scenario can be accelerated or slowed down.

[0036] According to one embodiment, the deviation is determined by a distance measure of a pseudo-metric.

[0037] The pseudo-metric thus defines a distance measure for the deviation with regard to the positions of the surrounding vehicles (and thus the relative position to the ego vehicle) and / or with regard to the time progress in the reference scenario.

[0038] According to one embodiment, it is provided that a next second control vector for determining the next simulation state in the next simulation time step is determined from an output of the assistance system to be tested, which results from the previous input of the current simulation state into the assistance system to be tested and is provided at the assistance system interface of the test simulation.

[0039] The simulation state, which can contain the state of the ego-vehicle with regard to its position, speed, and / or virtual sensor data, is used as input for the assistance system to generate a response. This response is provided by the assistance system as output at the assistance system interface, so that this output, which is a reaction to a previous input, is taken into account when calculating the next simulation state.

[0040] According to one embodiment, the simulation state is simulated in such a way that the state of the at least one surrounding vehicle includes a position, a change in position and / or a speed.

[0041] According to one embodiment, the simulation state is simulated in such a way that the state of the ego vehicle includes a position, a change in position, a speed and / or virtual sensor data for the assistance system to be tested.

[0042] According to one embodiment, the reference scenario is generated from an abstract scenario as a concrete scenario of the abstract scenario.

[0043] The problem is also solved according to the invention with the computer program according to claim 10. The computer program has program code means that are configured to carry out the method described above when the computer program is executed on a data processing system.

[0044] The problem is also solved according to the invention with the data stream according to claim 11, which represents the computer program as described above.

[0045] The problem is also solved according to the invention with the computer-readable storage medium according to claim 12, which comprises program code means which, when executed by a computer, cause it to execute the method described above.

[0046] The problem is also solved according to the invention with the device for testing an assistance system for at least partially automated driving functions of a vehicle by means of a scenario-based test simulation according to claim 13, wherein the scenario-based test simulation simulates an ego vehicle equipped with the assistance system to be tested and at least one surrounding vehicle according to a scenario, taking into account outputs of the assistance system to be tested of the ego vehicle, wherein the device for carrying out the above-described method is set up on a data processing system of the device.

[0047] The invention is explained in more detail using the attached figures as examples. They show: Fig. 1. Schematic representation of the test simulation; Fig. 2. Schematic representation of a test scenario; Fig. 3 Position diagram of ego vehicle and environment vehicle with respect to the test scenario from Fig. 2; Fig. 4 parameters for the distance metric; Fig. 5. Scenario time progression and metrics during the scenario.

[0048] Fig. Figure 1 shows a schematically simplified representation of the test simulation 10. The test simulation 10 has a simulator 11 which, step by step according to the step size δ, generates a current simulation state S in the next simulation time step. This current simulation state S serves as input for both the assistance system 12 under test and a simulation controller 13, which generates a corresponding first control vector I for controlling the surrounding vehicles 14 in the next simulation state S'. The simulation controller 13 solves a corresponding optimization problem using an optimization procedure to calculate the optimal control for the surrounding vehicles and the optimal time progression for the reference scenario 15. The reference scenario 15 is a concrete scenario derived from an abstract scenario 16 and is therefore an instance of this abstract scenario 16.

[0049] This makes it possible to adapt specific scenarios during a simulation to the actual behavior of the assistance system while simultaneously maintaining the essential aspects of the scenario, so that, for example, criticality metrics can be used to evaluate the assistance system. In this way, the simulation can serve for the approval and improvement of the assistance system. The specific scenario clearly defines the state of each road user within the scenario at every point in time.

[0050] The concrete scenario is an instance of a known abstract scenario. An abstract scenario is defined by the fact that it describes the relationships between road users instead of concrete scenes. An abstract scenario is understood here as a sequence of situations. Furthermore, the abstract scenario determines the temporal properties of this sequence. A concrete scenario is an instance of the abstract scenario if each scene corresponds to one of the situations in the abstract scenario, and the sequence of situations and their temporal properties are maintained.

[0051] The object of the invention is to control the simulation (more precisely, the surrounding vehicles in the simulation) in such a way that the course of the simulation corresponds to the abstract scenario. For this purpose, the concrete scenario, which is an instance of the abstract scenario, is used as a reference scenario. The scenario can be adapted to the behavior of the assistance system in two different ways: Firstly, the playback speed of the reference scenario can be changed, for example, to adapt it to a different speed of the assistance system, or if the assistance system reaches a certain state faster or slower. Secondly, the simulated scene does not have to correspond exactly to the current scene from the reference scenario, as long as the current traffic situation is maintained. The invention uses a distance measure d for this purpose, which is adapted to the abstract scenario.The smaller the distance between two scenes, the more similar the scenes are to each other in relation to the abstract scenario. In mathematical terms, the distance measure d is a pseudo-metric.

[0052] The simulator operates with a fixed (configurable) time step size δ. In each step, it receives a first control vector I and a second control vector J. From this, it generates the next simulation state S, which includes, among other things, the states of the vehicles in the environment and the ego vehicle, as well as, for example, synthetic sensor data for the assistance system.

[0053] The first control vector I contains, among other things, commands for controlling the vehicles in the vicinity of the assistance system and is generated by the simulation controller 13. In each simulation step, the simulation controller 13 calculates an optimal control vector I and an optimal time progression Δ for the reference scenario, such that the difference between the simulation and the reference scenario is minimized in the next step. By the next step, δ time units will have elapsed in the simulation, while Δ time units will have elapsed in the reference scenario, thus accelerating or decelerating the reference scenario by a factor of Δ / δ.

[0054] Technically, this is implemented as follows: A specific scenario is understood below as a function σ:T → Σ that defines a closed time interval T ⊂ ℝ ≥0maps to scenes. The scene σ(t) describes the state of each object at time t. For example, if we take the SuT and a road user f as objects, then σ(t)(v) SuT ) ∈ ℝ the velocity of the SuT and σ(t)(pos r ) ∈ ℝ 2 the position of r at time t.

[0055] The simulation control system uses a simple variant of model predictive control to determine the time progression Δ and the input I. It employs a function F for this purpose. δ(S, I), which predicts the next scene from the current simulation state S, resulting from the simulation with the input I for the environment. This function and its partial derivatives should be in closed form so that common optimization methods can be applied. The function F takes into account both the expected behavior of the SuT (this can be abstracted to any degree, e.g., as a linear extrapolation of Ego's position and velocity) and the behavior of the surrounding vehicle controls.

[0056] The simulation controller takes as input the reference scenario σ: [0,·] → Σ, a suitable (pseudo-)metric d, and an interval B containing the permissible playback speed. A pseudo-metric can be interpreted as a function d that satisfies for all x, y, z: d(x,x) = 0 ∧ d(x,y) = d(y,x) ∧ d(x,z) ≤ d(x,y) + d(y,z). The simulation controller also stores the current scenario time t, which is updated in each step. Initially, t = 0. In each step, the simulation controller calculates an environment input I and a time increment Δ > 0 such that d(Fδ(S,I),σ(t+Δ))minimal is and Δδ∈B, where S is the current simulation state.

[0057] The optimal input I is sent to the surrounding vehicle controls and t is updated to t ← t + δ.

[0058] The outlined algorithm can be further extended to apply different pseudo-metrics during a scenario. Then, d is replaced by a time-dependent pseudo-metric d. t This replaces the current scenario time t. Simultaneously, the playback speed interval can be replaced by an interval B that depends on t. t can be replaced. This allows, for example, adaptation to the behavior of the SuT only during a specific setup phase of the test scenario and deterministic control of the environment afterwards, when the response of the SuT is tested.

[0059] The different time progression δ of the simulation and 4 of the reference scenario results in a reparameterization β of the time axis. A reparameterization is a monotonic and bijective function β:T' → T between two time intervals. Applying a reparameterization β to a scenario σ leads to a scenario σ · β with (σ · β)(t) = σ(β(t)). If k denotes the current simulation step, t k the scenario time at the beginning of step k and Δ k The time progress calculated in step k results in the reparameterization as a piecewise linear function with β(t)=tk+(t−kδ)Δδ for t ∈ [kδ, (k + 1)δ]. Pseudo-metrics d t and playback speed intervals B t are therefore suitable for the reference scenario σ:T → Σ if they satisfy the following for all possible simulation runs σ': T' → Σ and reparameterizations β : T → T': If for all t ∈ T d t (σ'(β'(t)), σ(t)) = 0 and β̇(t) ∈ Bt , then σ' is also an instance of the abstract scenario. This property is always satisfied if d t = d is a metric in the mathematical sense, i.e., d satisfies the property d(x,y) = 0 ⇒ x = y, and B t = B = [1,1]. In this case, σ' = σ is implied, i.e., an exact reproduction of the reference scenario is enforced.

[0060] The following outlines an exemplary procedure for creating a time-dependent pseudo-metric d t a reference scenario can be generated based on the abstract scenario. It is assumed that a scene is represented as a vector x ∈ ℝ. k can be represented, where each entry in the vector stands for a relevant attribute of an object in the simulation. Furthermore, it is assumed that each situation (denoted by index i) in the abstract scenario is described by a linear system of inequalities A. i x ≤ b ican be formulated, i.e., the described situation exists when A i x ≤ b i This applies, for example, to many scenarios that are formalized using traffic sequence charts (Damm et al. 2018). Each row in the inequality system formulates a relation between the road users. The pseudo-metric d t This then follows from dt(x, x') = || A i (x - x')||2 if the situation at time t has index i.

[0061] A concrete reference scenario is used, characterized by the fact that it includes the specific state (i.e., attribute values ​​including position) for each road user at any given time. Furthermore, the reference scenario is an instance of an abstract scenario. The use of a pseudometric in conjunction with a numerical optimization procedure makes it possible to react to the behavior of the assistance system at execution time. Here, the pseudometric is used to optimally select the time progression of the reference scenario and to optimally adapt the current scene from the reference scenario to the actual state of the assistance system. A common optimization goal can be used for this purpose.

[0062] The pseudometric can be chosen depending on the progress of time in the reference scenario and is therefore variable during the test procedure.

[0063] In the Fig. 2, Fig. 3, Fig. 4 to Fig. Section 5 presents a corresponding embodiment with a concrete scenario. The method was implemented using a simple model for controlling an automated train. The abstract scenario used is described in Fig. Figure 2 is presented as a Traffic Sequence Chart (TSC). The scenario is formulated as a sequence of traffic situations, each depicted visually (in the form of a so-called Spatial View, or SV). The example scenario is a railroad crossing scenario in which a road user (black car) is stopped at an unprotected railroad crossing while the ego approaches. The Spatial Views describe the following situations from right to left: 1. The ego is located 200 m and the road user 100 m from the level crossing, both facing towards the level crossing. 2. The ego is located between 30 m and 70 m from the railway crossing, the path user is located on the railway crossing. 3. The road user is on the level crossing. 4. The road user is located on or behind the level crossing. 5. The road user has left the level crossing.

[0064] The traffic situations transition seamlessly into one another, with the exception of SV 1 and SV 2, which are represented by a hatched rectangle. The situation depicted in SV 3 persists for a period of at least 15 seconds. When selecting this scenario, it is important to note that the ego is only represented in the first two spatial views. For TSCs, this means that the ego's behavior in the remaining spatial views is not specified. Thus, the road user's behavior during the first spatial views is adapted to the ego's behavior, and subsequently, the reference scenario is implemented independently of the ego. This ensures that the test evaluation is not negatively influenced by the simulation control.

[0065] In this application example, the simulator used does not provide a physics simulation for road users, and this is not relevant for the test procedure. Instead, the simulation controller directly determines the road user's position, and the controls for the surrounding vehicle are omitted. • The simulation state is a 4-tuple S = (x e ,v e ,x w ,y w ) consisting of x-position x e and speed v e of the ego (since it is rail-bound, the ego can only move in the x-direction) as well as position (x w , y w ) of the road user. • Each scene is a vector x = (x e , x w , y w ) T ∈ ℝ 3 from the perspective of the ego and the path user. • The simulation input for controlling the path user is I = (x' w ,y' w ) consisting of the position of the road user in the next simulation step.

[0066] In Fig. Figure 3 shows the relevant positions over time in the scenario for the road user (dashed line) and the train as an ego vehicle (solid line). The specific parameters of the distance metric are then described in Fig. 4 are then in Fig. 4 shown.

[0067] For the prediction function, the new position of the ego is extrapolated based on its speed, and the position of the path user results directly from the input I, i.e.: Fδ((xe,ve,xw,yw),(x'w,y'w))=(xe+δve,x'e,y'e)T

[0068] The pseudo-metric d t and the simulation velocity interval B t are chosen depending on the current spatial view. As a pseudo-metric d t A simple quadratic semi-norm serves this purpose. dt(x,x')=(atT⋅(x−x'))2 where the weights a it ∈ ℝ 3depend on the spatial view at time t. The time axis of the reference scenario is divided into six intervals J1, J2, ..., J6, and for each interval J i (with i ∈ {1,2, ...,6}) an interval B i and weights a i The influence of the ego position on the pseudo-metric was chosen. The weights for the metric and the playback speed intervals then result from this. at=(ai,1,1)T and B t = B i for t ∈ J iThe selected parameters are listed in Table 1. When choosing parameters, it is important to note that Ego is not part of Spatial Views 3, 4, and 5 and is therefore not included in the corresponding metrics. This means that the user's control is independent of Ego from t ≥ 450 s onwards. This allows for meaningful test evaluation even if the user's behavior deviates from the expected behavior and, for example, does not stop before the level crossing. Furthermore, the playback speed is 1 during SV 3, which is why the duration of SV corresponds to the duration in the reference scenario. This ensures that the minimum duration is met.

[0069] Fig. Figure 5 (above) shows the course of the scenario time t during the simulation. The course suggests a different acceleration behavior of the Ego compared to the reference scenario, which is due to the fact that a simplified dynamics model was assumed when generating the reference scenario. From the course of the metric evaluation ( Fig. 5 below) during the execution of the scenario, it is evident that the deviating acceleration behavior was successfully compensated for by the simulation control and the abstract scenario is fulfilled (the spike after 5 s is to be considered a numerical artifact). Reference symbol list 10 Test simulation 11 Simulator 12 Assistance systems 13 Simulation control 14 Surrounding vehicle 15 Reference scenario 16 abstract scenario I first control vector J second control vector S Simulation state QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 20 2020 005 507 A1

[0008] DE 10 2022 129 009 A1

[0009] DE 10 2023 100 858 A1

[0010] Cited non-patent literature

[0000] Jan Steffen Becker et al: “Simulation of abstract scenarios: Towards automated tooling in criticality analysis, 2022

[0012] Jan Steffen Becker: „Safe linear encoding of vehicle dynamics for the instantiation of abstract scenarios“ Int. Conf. on Formal Methods for Industrial Critical Systems (FMICS), 2024

[0012]

Claims

Method for testing an assistance system (12) for at least partially automated driving functions of a vehicle using a scenario-based test simulation (10), wherein the scenario-based test simulation (10) simulates an ego vehicle equipped with the assistance system (12) to be tested and at least one environment vehicle (14) according to a scenario (16) taking into account outputs of the assistance system (12) to be tested of the ego vehicle, wherein the method comprises the following steps performed by the test simulation (10) on a data processing system: - obtaining a first control vector (I) for controlling the at least one environment vehicle (14), - obtaining a second control vector (J) for controlling the ego vehicle from an output of the assistance system (12) to be tested,- Determining a current simulation state (S) for a simulation time step by the test simulation (10) as a function of the first control vector (I) and the second control vector (J), wherein the simulation state (S) includes the states of the at least one surrounding vehicle (14) and the ego vehicle, - Providing the simulation state (S) at an assistance system interface of the test simulation (10) in order to input the current, simulated simulation state (S) into the assistance system (12) under test as input data, and - Determining a next first control vector (I) for determining a next simulation state (S) in a next simulation time step such that a deviation between the current simulation state (S) and a reference scenario (15) is minimized. Method according to claim 1, characterized in that the steps are repeated multiple times until an end of the reference scenario (15) is reached. Method according to claim 1 or 2, characterized in that the next first control vector (I) is determined in such a way that a deviation with respect to the positions of the surrounding vehicles (14) between the current simulation state (S) and a reference scenario (15) is minimized. Method according to one of the preceding claims, characterized in that a time progression for the reference scenario (15) is determined such that the deviation between the current simulation state (S) and a reference scenario (15) is minimized. Method according to one of the preceding claims, characterized in that the deviation is determined by a distance measure of a pseudo-metric. Method according to one of the preceding claims, characterized in that a next second control vector (J) for determining the next simulation state (S) in the next simulation time step is determined from an output of the assistance system (12) to be tested, which results from the previous input of the current simulation state (S) into the assistance system (12) to be tested and is provided at the assistance system interface of the test simulation (10). Method according to one of the preceding claims, characterized in that the simulation state (S) is simulated such that the state of the at least one surrounding vehicle includes a position, a change in position and / or a speed. Method according to one of the preceding claims, characterized in that the simulation state (S) is simulated such that the state of the ego vehicle contains a position, a change in position, a speed and / or virtual sensor data for the assistance system (12) to be tested. Method according to one of the preceding claims, characterized in that the reference scenario (15) is generated from an abstract scenario (16) as a concrete scenario of the abstract scenario (16). Computer program equipped with program code means for carrying out the method according to any one of claims 1 to 9, when the computer program is executed on a data processing system. Data stream representing the computer program according to claim 10. A computer-readable storage medium comprising program code means which, when executed by a computer, cause the computer to execute the method according to any one of claims 1 to 9. Device for testing an assistance system (12) for at least partially automated driving functions of a vehicle by means of a scenario-based test simulation (10), wherein the scenario-based test simulation (10) simulates an ego vehicle equipped with the assistance system (12) to be tested and at least one surrounding vehicle according to a scenario (16) taking into account outputs of the assistance system (12) to be tested of the ego vehicle, wherein the device for carrying out the method according to one of claims 1 to 9 is set up on a data processing system of the device.

Images