Computer-implemented method for controlling a simulation device by means of alteration suggestions

Abstract

The invention relates to a computer-implemented method for generating an alteration suggestion (10) for at least one simulation run (12) of a simulation (14) having multiple simulation runs (12), comprising the steps of - receiving one or more simulation run results (16) of one or more completed simulation runs (12) of the simulation (14), - predicting at least one simulation run result (20) of a simulation run (12) of the simulation (14) that has not yet been completed, taking into consideration the received simulation run results (16), and - generating the alteration suggestion (10) for simulation runs (12) of the simulation (14) that are yet to be completed, taking into consideration the at least one predicted simulation run result (20). The invention also relates to a device for data processing, comprising means for executing the above method, to a computer program product and to a corresponding computer-readable data carrier.

Description

[0001] COMPUTER-IMPLEMENTED METHOD FOR CONTROLLING A SIMULATION DEVICE BY MEANS OF CHANGE PROPOSALS

[0002] The invention relates to a computer-implemented method for generating a change proposal for at least one simulation run of a simulation comprising a plurality of simulation runs.

[0003] Furthermore, the invention relates to a device for data processing comprising means for carrying out the above method.

[0004] Furthermore, the invention relates to a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the above method.

[0005] Furthermore, the invention relates to a computer-readable data carrier on which the above computer program product is stored.

[0006] Driver assistance systems such as adaptive cruise control and / or functions for highly automated or autonomous driving can be verified or validated using various testing methods. For example, functional tests of electronic control unit (ECU) software can generally be performed using hardware in the form of the ECU itself or a prototype ECU, and the control software can be tested in real-world driving conditions. However, such tests are problematic, not only because of the high costs and considerable time required, but also because of the lack of reproducibility due to the complex environmental conditions.

[0007] Another option is hardware-in-the-loop (HiL) testing, where the ECU is connected to a HiL simulator via inputs and outputs. The HiL simulator simulates the real-world environment. The HiL simulator emulates the electrical signals from sensors and actuators that are read by the ECU. For example, in a HiL test of an internal combustion engine, the signals from the crankshaft and camshaft sensors are generated by an angular processing unit (APU), which is part of the HiL simulator's hardware. The ECU, in turn, uses the control software to generate sensor and / or actuator control signals based on these signals. These signals, in turn, lead to a change in the electrical signals in the HiL simulator. The route that would be driven on as a test track in real driving tests and thus generate different electrical signals for the control unit is represented as a route model within the HiL simulation.However, in order to perform a HiL test, the development of an ECU must already be relatively advanced, since the ECU is connected to the HiL simulator as hardware.

[0008] To enable earlier testing of the control software, for example, at a development stage when no hardware is yet available, hardware-independent software tests are required. Virtual control units, also known as V-ECUs, can be used for this purpose. A virtual control unit is used to emulate a real control unit in a simulation scenario. To create virtual control units that are as realistic as possible, a virtual control unit can be created based on the control software of the real control unit. Naturally, some components of the control software of the real control unit must be adapted to the virtual environment of the simulation platform. This approach generally aims to simulate a real control unit as accurately as possible.

[0009] However, shifting virtual testing to ever earlier development phases is leading to changing requirements, where the focus is no longer on the best possible imitation of the behavior of a specific control unit, but rather on testing a wide variety of applications in a flexible and highly modular environment. This is typically achieved through reproducible, deterministic test drives with a virtual vehicle in a virtual environment. The vehicle, the environment, and the tests—i.e., the traffic scenarios—can all be freely defined by the user. In this way, new algorithms for vehicle control in virtualized, autonomous vehicles can be tested, among other things.

[0010] One possible traffic scenario to consider is a so-called cut-in scenario. A cut-in scenario describes a traffic situation in which a highly automated or autonomous vehicle—hereinafter referred to as the ego vehicle—is driving in a specified lane. Another vehicle, at a certain distance from the ego vehicle and at a lower speed than the ego vehicle, cuts into the lane of the ego vehicle from another lane.

[0011] In the traffic scenario, the speed of the ego vehicle and the other vehicle is usually initially constant. However, since the speed of the ego vehicle is higher than that of the other vehicle, the ego vehicle must be decelerated to avoid a collision with the other vehicle.

[0012] The traffic scenario can be simulated with different parameter values, for example, different distances between the ego vehicle and the other vehicle when merging, or different speed differences between the ego vehicle and the other vehicle. To gain the greatest possible insight from the simulation, many different simulation runs are performed within the possible parameter space. Complex traffic scenarios have many parameters and thus a very large parameter space to cover, so that typically over one hundred thousand simulation runs must be performed to cover the test space. Accordingly, such simulations are resource- and / or time-intensive.

[0013] Based on this, the object of the invention is to simulate complex traffic scenarios in a resource-efficient and / or short-term manner. Preferably, the object of the invention is to limit the parameter space during the simulation.

[0014] This problem is solved by the features of the independent patent claims. Preferred developments can be found in the subclaims.

[0015] According to the invention, a computer-implemented method for generating a change proposal for at least one simulation run of a simulation comprising a plurality of simulation runs is provided, comprising the steps

[0016] - receiving one or more simulation run results of one or more completed simulation runs of the simulation, - predicting at least one simulation run result of a not yet completed simulation run of the simulation taking into account the received simulation run results, and

[0017] - Generating the change proposal for simulation runs that have not yet been completed, taking into account at least one predicted simulation run result.

[0018] One aspect of the invention is that, based on the received simulation run results from previously performed simulation runs, a prediction is made regarding the simulation run results of simulation runs that have not yet been completed, and the change proposal is generated taking these predicted simulation run results into account. Accordingly, the prediction allows the results to be evaluated early—i.e., preferably before all simulation runs of the simulation are executed and completed—which conserves resources and leads to cost savings.In other words, the process for generating the change proposal for at least one simulation run is carried out in parallel to the simulation to be carried out with the large number of simulation runs in order to be able to make early predictions about the simulation run results and to be able to implement corresponding change proposals for the simulation runs that have not yet been completed, preferably before all simulation runs of the simulation have been completed.

[0019] The simulation comprises a plurality of simulation runs, typically on the order of 100,000 simulation runs. When a simulation run is completed, the simulation run result of the corresponding simulation run is also available. Preferably, the plurality of simulation runs have different parameters from one another. In this way, the simulation runs cover the possible parameter space of the simulation. Simulation runs that have not yet been completed are simulation runs that have not yet been started or simulation runs that have already been started but have not yet been completed. For simulation runs that have not yet been completed, no simulation run result is therefore available from the simulation. According to a preferred development of the invention, the method comprises the step of modifying one or more simulation runs of the simulation that have not yet been completed, taking into account the generated change proposal.Preferably, changes to the not-yet-completed simulation runs are automatically implemented based on the change proposal. This can save costs and / or conserve resources without user interaction.

[0020] In this context, according to a further preferred development of the invention, the generated change proposal comprises aborting simulation runs that have not yet been completed, not executing simulation runs that have not yet been started, and / or adjusting simulation parameters for simulation runs that have not yet been started. By aborting simulation runs that have not yet been completed and not executing simulation runs that have not yet been started, the number of simulation runs is reduced, which saves resources, time, and costs. Thus, simulation runs that are not expected to yield any new insights based on the predicted simulation run results are not carried out at all. By adjusting simulation parameters for simulation runs that have not yet been started, the simulation can be limited to the parameter space that has proven to be useful based on the predicted simulation run results.Accordingly, the knowledge gained from the simulations carried out increases.

[0021] According to a further preferred development of the invention, the method comprises the additional step of generating a simulation run result of the simulation by executing and terminating a simulation run of the simulation. The simulation run results received in the first step of the method preferably originate from precisely the simulation to which the method for generating the change proposal is being executed in parallel. This leads to particularly good predictions.

[0022] According to a further preferred development of the method, the method includes the step of training a substitute model of the simulation using the received simulation run results. In other words, the predictions are generated by a substitute model that is trained using the received simulation run results. This advantageously leads to the predictions of the substitute model continuously improving through training with the received simulation run results.

[0023] In this context, according to a preferred development of the invention, the method further comprises the step of determining a quality and / or an error of the surrogate model of the simulation by comparing predicted simulation run results of the surrogate model with the received simulation run results. This allows the prediction accuracy of the surrogate model to be continuously assessed and, preferably only when a predefined quality is exceeded and / or when a predefined error of the surrogate model is undershot, to generate the proposed changes for simulation runs that have not yet been completed and / or to implement the generated proposed changes by preferably automatically modifying simulation runs that have not yet been completed.

[0024] According to a further development of the method, the step of predicting at least one simulation run result of a not yet completed simulation run of the simulation is preferably carried out using the surrogate model of the simulation, taking into account the received simulation run results. The surrogate model has a reduced resource requirement and / or a shorter runtime compared to a simulation model of the simulation. In other words, the surrogate model is more resource-efficient and / or efficient than the simulation model, whereby predicting the simulation run results using the surrogate model is faster and / or less resource-intensive than determining the simulation run results using the simulation model of the simulation.

[0025] With regard to the substitute model, according to a further preferred development, it is provided that the substitute model comprises a neural network and / or a radial basis function. In particular, neural networks and / or linear combinations of radial basis functions are particularly well suited for approximating functions—in the present case, therefore, simulation using the simulation model. According to a further development of the invention, the predicted simulation run result is preferably selected from the group comprising KPI values, observer bonuses, and verdicts. In other words, KPI values, observer bonuses, and / or verdicts are preferably predicted as simulation run results using the substitute model. In the present case, KPI values ​​are understood to mean key performance indicators, i.e., essential performance indicators of the simulation.For example, a KPI could be the distance traveled by a vehicle in a simulation or the number of vehicle collisions relative to the distance traveled. Observer overrides provide information about the dynamic response of the simulation model and enable the detection of a situation defined by an observer, such as the violation of a minimum distance between two vehicles. A verdict, in this case, is a variable that evaluates the simulation and can assume values ​​such as "faulty," "inconclusive," "passed," or "failed."

[0026] According to a further preferred development of the invention, the method comprises the step of evaluating multiple predicted simulation run results, and wherein the evaluation comprises performing a threshold value analysis, determining a trend, and / or determining a passed / failed rate. Based on these evaluations, it is particularly easy to generate suggested changes and / or, taking the generated suggested changes into account, implement changes to one or more not yet completed simulation runs of the simulation. Furthermore, the method can comprise the step of evaluating multiple received simulation run results and multiple predicted simulation run results. In other words, the received simulation run results can be included in the evaluation—that is, in determining the trend and / or the passed / failed rate.

[0027] Further technical features and advantages will become apparent to the person skilled in the art from the following description of a device for data processing, a computer program product and / or a computer-readable data carrier, as well as from the exemplary embodiment.

[0028] The object is also achieved by a data processing device comprising means for executing the method described above. The data processing device is preferably a server-based device. This allows the user to not have to maintain hardware resources to carry out the method for generating the change proposal for at least one simulation run of the simulation comprising a plurality of simulation runs, but rather the method is made available to the user on a server.

[0029] Furthermore, the invention relates to a computer program product comprising instructions which, when the program is executed by a computer, cause the computer to carry out the above method.

[0030] The invention also relates to a computer-readable data carrier on which the above computer program product is stored. The instructions are preferably embedded on the computer-readable data carrier, and when executed by a processor of the computer, the instructions cause the processor to execute the method for generating the change proposal for at least one simulation run of the simulation comprising a plurality of simulation runs.

[0031] The technical advantages and effects of the data processing device, the computer program product and the computer-readable data carrier will become apparent to the person skilled in the art from the description of the method for generating the change proposal and from the exemplary embodiment described below.

[0032] The invention is explained in more detail below with reference to the drawing. The illustrated embodiment is highly schematic, ie distances, dimensions and measurements are not to scale and, unless otherwise stated, do not have any deducible geometric relationships to one another. The drawing shows

[0033] Fig. 1 schematically shows a diagram illustrating a method for

[0034] Generating a change proposal according to a preferred embodiment of the invention.

[0035] Figure 1 schematically shows a diagram to illustrate a method for generating a change proposal 10 for at least one simulation run 12 of a simulation 14 comprising a plurality of simulation runs 12, according to a preferred embodiment of the invention.

[0036] In this exemplary embodiment, a simulation 14 with 100,000 simulation runs 12 is to be performed. After a user has specified simulation parameters for the simulation runs 12, the simulation 14 is started, and simulation run results 16 of the individual simulation runs 12 are determined using a simulation model of the simulation 14. Figure 1 illustrates the situation at a time when the first simulation runs 12 with the simulation model have been performed and completed, and corresponding simulation run results 16 are available. Specifically, 20,000 simulation runs 12 of the simulation 14 were performed, and accordingly, 20,000 simulation run results 16 are available.

[0037] In the method for generating the change proposal 10 for at least one simulation run 12 of the plurality of simulation runs 12, the simulation run results 16 of the completed simulation runs 12 of the simulation 14 are received in a first step. Using these received simulation run results 16, a substitute model 18 is then trained. The substitute model 18 serves the purpose of predicting at least one simulation run result 16. In this case, the substitute model 18 is a neural network. Using the substitute model 18, simulation run results 20 of not yet completed simulation runs 12 of the simulation 14 are subsequently predicted, taking into account the received simulation run results 16. In this exemplary embodiment, the predicted simulation run results 20 are KPI values, observer bonuses, and verdicts.To evaluate the quality of the substitute model 18, the error of the substitute model 18 is also continuously determined. If a predefined threshold for the error of the substitute model 18 is undershot, a change suggestion 10 for not yet completed simulation runs 12 of the simulation 14 is generated, taking into account the previously predicted simulation run results 20. For this purpose, the predicted simulation run results 20 and the received simulation run results 16 are evaluated. In this exemplary embodiment, a threshold analysis, a trend observation, and an examination of the passed / failed rate are performed as part of the evaluation.

[0038] In this exemplary embodiment, the change suggestion 10 comprises aborting simulation runs 12 that have not yet been completed and not executing simulation runs 12 that have not yet been started. Furthermore, in this exemplary embodiment, the change suggestions 10 are implemented automatically. Accordingly, the method presently also comprises modifying several simulation runs 12 of the simulation 14 that have not yet been completed, taking into account the generated change suggestion 10. In this exemplary embodiment, due to the predicted simulation run results 20, this leads to the aborting of all simulation runs 12 that have not yet been completed and the non-execution of all simulation runs 12 that have not yet been started. Thus, 80,000 fewer simulation runs 12 are performed than originally planned, resulting in a corresponding 80% cost saving.

[0039] List of reference symbols

[0040] 10 Change proposal 12 Simulation run

[0041] 14 Simulation

[0042] 16 Simulation run result

[0043] 18 Substitute model 0 predicted simulation run result

Claims

Patent claims 1. Computer-implemented method for generating a change proposal (10) for at least one simulation run (12) of a simulation (14) comprising a plurality of simulation runs (12), comprising the steps - receiving one or more simulation run results (16) of one or more completed simulation runs (12) of the simulation (14), - predicting at least one simulation run result (20) of a not yet completed simulation run (12) of the simulation (14) taking into account the received simulation run results (16), and - generating the change proposal (10) for simulation runs (12) of the simulation (14) that have not yet been completed, taking into account the at least one predicted simulation run result (20).

2. The method according to claim 1, wherein the method comprises the step of modifying one or more simulation runs (12) of the simulation (14) which have not yet been completed, taking into account the generated modification proposal (10).

3. Method according to one of the preceding claims, wherein the generated Change proposal (10) a cancellation of not yet completed Simulation runs (12), failure to carry out not yet started simulation runs (12), and / or adjusting simulation parameters for simulation runs (12) that have not yet been started.

4. The method according to any one of the preceding claims, wherein the method comprises the additional step of generating a simulation run result (16) of the simulation (14) by performing and terminating a simulation run (12) of the simulation (14).

5. The method according to any one of the preceding claims, wherein the method comprises the step of training a replacement model (18) of the simulation (14) using the received simulation run results (16).

6. The method according to the preceding claim, wherein the method comprises the step of determining a quality and / or an error of the substitute model (18) of the simulation (14) by comparing predicted simulation run results (20) of the substitute model (18) with the received simulation run results (16).

7. The method according to any one of the preceding claims, wherein the step of predicting the at least one simulation run result (20) of a not yet completed simulation run (12) of the simulation (14) is carried out taking into account the received simulation run results (16) by means of a substitute model (18) of the simulation (14), and wherein the substitute model (18) has a reduced resource requirement and / or a shortened runtime compared to a simulation model of the simulation (14).

8. The method according to any one of claims 5 to 7, wherein the replacement model (18) comprises a neural network and / or a radial basis function.

9. Method according to one of the preceding claims, wherein the predicted simulation run result (20) is selected from the group comprising KPI values, observer surcharges, verdicts.

10. The method according to any one of the preceding claims, wherein the method comprises the step of evaluating a plurality of predicted simulation run results (20), and wherein the evaluation comprises performing a threshold analysis, determining a trend and / or determining a passed / failed rate.

11. A data processing device comprising means for carrying out the method according to one of claims 1 to 10.

12. A computer program product comprising instructions which, when executed by a computer, cause the computer to carry out the method according to any one of claims 1 to 10.

13. A computer-readable data carrier on which the computer program product according to the preceding claim is stored.

Images