Computer-implemented method for configuring a virtual test system and training method

Abstract

The invention relates to a computer-implemented method for configuring a virtual test system for testing a vehicle function of a motor vehicle, wherein for each of a plurality of input ports of an artifact to be tested, assigning an output port of at least one further artifact to be tested having a highest confidence value is implemented according to a first condition; creating a list of output ports having a highest confidence value is implemented according to a second condition or not assigning an output port is implemented according to a third condition for configuring a connection of an input port of an artifact to be tested with an adapted output port of the at least one further artifact to be tested. The invention furthermore relates to a computer-implemented method for providing a trained machine learning algorithm for configuring a virtual test system for testing a vehicle function of a motor vehicle. The invention also relates to a computer program and a computer-readable data carrier.

Description

TECHNICAL FIELD

[0001] The present invention relates to a computer-implemented method for configuring a virtual test system for testing a vehicle function of a motor vehicle.

[0002] The present invention furthermore relates to a computer-implemented method for providing a trained machine learning algorithm for configuring a virtual test system for testing a vehicle function of a motor vehicle. BACKGROUND

[0003] In PC-based simulation platforms for ensuring the safety of the software of electronic control units (ECU) - which enable the simulation of a plurality of different modules (from functional models to networks of virtual control units to bus systems and vehicle models) - the number of artefacts simulated and thus the effort for configuring the simulation system is constantly increasing.

[0004] In addition to bus communication, such PC-based simulation platforms likewise provide data exchange between artefacts via ports. For this purpose, output ports must be connected with input ports between the artefacts. The order of magnitude of such ports lies between 100 and 10,000 ports per artefact.

[0005] Here, in principle, there are two approaches for configuring the connections. On the one hand, via a graphical user interface of the simulation platform (for example via "Drag & Drop") and on the other hand via automation by means of machine-readable lists.

[0006] Automation, for example via the names of the ports, is often error-prone, since the names are often slightly different. Thus, in both cases, the integration of control units into the simulation system leads to significant effort.

[0007] There is therefore a need to improve existing methods and systems for configuring a virtual test system for testing a vehicle function of a motor vehicle, such that the integration of control units into the simulation system can be achieved automatically and user-friendly, which furthermore leads to a reduction in the time effort by the user and a lower consumption of computing resources. SUMMARY

[0008] It is therefore the task of the present invention to provide a computer-implemented method for configuring a virtual test system for testing a vehicle function of a motor vehicle, which enables the automatic and user-friendly integration of control units into the simulation system, while enabling a reduction in the time effort by the user and a lower consumption of computing resources.

[0009] The task is solved by a computer-implemented method for configuring a virtual test system for testing a vehicle function of a motor vehicle according to the application.

[0010] The method comprises providing a data record comprising first data of a plurality of input ports of an artificial object to be tested and second data of a plurality of output ports of at least another artificial object to be tested.

[0011] Further, the method comprises applying a machine learning algorithm to the data record, wherein for each input port of the plurality of input ports of the artificial object to be tested a confidence value for each output port of the plurality of output ports of the at least another artificial object to be tested is determined.

[0012] The method further comprises for each input port of the plurality of input ports of the artificial object to be tested, assigning the output port of the at least another artificial object to be tested having the highest confidence value according to a first condition, creating a list of output ports having the highest confidence value according to a second condition or not assigning an output port according to a third condition for configuring a connection of the input port of the artificial object to be tested with an adapted output port of the at least another artificial object to be tested.

[0013] The application further relates to a computer-implemented method for providing a trained machine learning algorithm for configuring a virtual test system for testing a vehicle function of a motor vehicle.

[0014] The method comprises providing a first training data record comprising first data of a plurality of input ports of an artificial object to be tested and second data of a plurality of output ports of at least another artificial object to be tested.

[0015] Further, the method comprises providing a second training data record comprising for each input port of the plurality of input ports of the artificial object to be tested a confidence value for each output port of the plurality of output ports of the at least another artificial object to be tested.

[0016] Further, the method comprises training the machine learning algorithm by an optimization algorithm, in particular calculating an extreme value of a loss function for configuring the virtual test system in case of using a stochastic learning.

[0017] The application further relates to a computer program having a program code for implementing at least one of the methods according to the application when the computer program is executed on a computer.

[0018] The present invention further relates to a computer-readable data carrier having program code of a computer program so as to implement at least one of the methods according to the present invention when the computer program is executed on a computer.

[0019] Machine learning algorithms are based on training a data processing device using statistical methods to perform a specific task without requiring explicit initial programming. The goal of machine learning is to construct algorithms that can learn from data and make predictions. These algorithms create mathematical models that can, for example, classify data.

[0020] For example, such machine learning algorithms are used if a physical model, i.e., a model based on physical conditions and / or dependencies, reaches its boundary (e.g., due to increased resource consumption).

[0021] The concept of this invention is to support or automate the creation of connections between simulated artifacts by employing machine learning algorithms.

[0022] Here, machine learning algorithms, especially neural networks, are trained to identify which output port belongs to which input port.

[0023] To this end, the algorithm uses not only names but also metadata (such as data type, unit, etc.). For each input, a list of suitable output ports is created, and for each entry in the list, a confidence value is calculated, which represents the estimated probability of wanting to connect to the correct port.

[0024] Furthermore, multiple conditions, in the form of thresholds, are applied to the confidence value, from which different actions are taken. If the confidence value is, for example, higher than a first configurable threshold, the algorithm concatenates the corresponding output with the corresponding input and records the action in a list for the user.

[0025] If the confidence value is less than the first threshold but greater than the second threshold, the algorithm automatically suggests the corresponding connection, which the user must accept. All other suggestions can be displayed in a list and either accepted or discarded by the user.

[0026] Therefore, the solution according to the present invention advantageously simplifies the configuration of the simulation system and reduces time consumption. Similarly, it avoids the error-proneness inherent in manually creating port connections by the user through a graphical user interface.

[0027] Applying machine learning algorithms to data records, wherein for each of the plurality of input ports of the artifact to be tested, a confidence value is determined for each of the plurality of output ports of at least another artifact to be tested, the above process represents classifying the output ports according to the confidence values ​​assigned to the output ports respectively.

[0028] Here, the confidence value associated with the classification gives the probability that each output port belongs to a specific input port, wherein if a first condition is met, the category with the highest probability is selected, and the first condition must at least reach a threshold, thereby implementing automatic allocation of the output port to the specific input port.

[0029] Other embodiments of the present invention are described below with reference to the accompanying drawings.

[0030] According to a preferred further extension, the method further includes: a first condition is given as follows, that is, if the confidence value is greater than or equal to a predetermined first threshold, particularly greater than or equal to 99%, then the output port of the at least one other artifact to be tested with the highest confidence value is automatically assigned.

[0031] Therefore, a confidence value greater than or equal to 99% means that the remaining classification results, i.e., the remaining output ports obtained, total at most 1%. Thus, it can be assumed that the output ports involved, marked with a confidence value greater than or equal to 99%, are correct, and the algorithm can automatically assign the output ports to the corresponding input ports.

[0032] According to another preferred further extension, the second condition is given as follows: if the confidence value is less than a predetermined first threshold, in particular less than 99%, and greater than or equal to a predetermined second threshold, in particular greater than or equal to 80%, then a list of output ports of a predetermined size with the highest confidence value is created.

[0033] The advantage of creating a short list is that the output ports included in the short list have relatively high confidence values, thus allowing one to assume that one of the output ports is correct.

[0034] According to another preferred further extension, the method includes: based on a list of predetermined sizes, in particular, allowing the user to manually select the appropriate output ports.

[0035] This offers the user the advantage that they do not need to analyze or compare multiple possible output ports, but can simply select the correct output port from a short list of output ports based on a pre-selection.

[0036] According to another preferred further extension, the method further includes: sorting the confidence values ​​included in the created list in ascending or descending order; and wherein the list includes confidence values ​​that are less than a predetermined first threshold, in particular less than 99%, and greater than or equal to a predetermined second threshold, in particular greater than or equal to 80%.

[0037] Therefore, the ranking of confidence values ​​also simplifies the selection process for the user in a favorable way, since it is possible that the output port with the highest confidence value is correct. Furthermore, the user has the possibility of not selecting any of the suggested output ports and rejecting the created list.

[0038] According to another preferred further extension, the method includes: a third condition is given as follows, that is, if the confidence value is less than a predetermined second threshold, particularly less than 80%, then no output port allocation is performed.

[0039] Therefore, a value below the predetermined threshold of confidence level results in the classification being discarded because it is not possible to determine with sufficient probability which output ports to consider.

[0040] Therefore, the error rate in selecting output ports can be reduced in an advantageous way because unlikely results are discarded in advance.

[0041] According to another preferred further extension, the method includes: third data recording having a plurality of output ports of the artifact to be tested and fourth data recording a plurality of input ports of at least another artifact to be tested. Therefore, the machine learning algorithm is capable of linking the output ports of the plurality of artifacts to the input ports.

[0042] According to another preferred further extension, the method includes: data recording of all input ports and all output ports of the artifact to be tested and at least another artifact to be tested, or selection of the input ports and output ports of the artifact to be tested and the at least another artifact to be tested by a user or automatically suggested.

[0043] Therefore, the data contained in the data record enables machine learning algorithms to learn a function for assigning the output port to each input port in the corresponding artifact's input ports.

[0044] According to another preferred further extension, the method includes: recording each implemented allocation from the output port of the at least one other artifact to the input port of the artifact to be tested in a log file.

[0045] The advantage of this is that the user can subsequently understand what they themselves have chosen and what the algorithm has chosen. The log file also contains information on which ports the algorithm has interconnected. Therefore, potential errors can be identified and subsequently manually corrected.

[0046] According to another preferred further extension, the method includes: the artifact to be tested and the at least one other artifact to be tested being a virtual control unit and / or an environmental model of a motor vehicle.

[0047] Therefore, after interconnecting the input ports with the corresponding output ports through the algorithm, simulation of the control unit and / or environment model (especially software-based vehicle functions such as adaptive cruise control) can be implemented.

[0048] According to another preferred further extension, the method includes: an optimization algorithm calculating the allocation of the output ports of the at least one other artifact to be tested using the names (in particular alphanumeric names) and / or metadata (in particular data types) of the input ports of the artifact to be tested.

[0049] By using a variety of data types and information sources, prediction accuracy can be improved in a favorable manner when the input port is interconnected with the corresponding output port.

[0050] According to another preferred further extension, the method includes: the confidence value of the second training data record is less than a predetermined first threshold, in particular less than 99%, and greater than or equal to a predetermined second threshold, in particular greater than or equal to 80%.

[0051] Therefore, the algorithm is trained in an advantageous manner using data that gives the correct classification result for identifying the corresponding output port with a high probability.

[0052] According to another preferred further extension, the method includes: training the machine learning algorithm based on user-implemented selection of or non-application of an output port with the highest confidence value assigned to the at least one other artifact to be tested, or no assignment of an output port.

[0053] Therefore, the algorithm can be further developed through continuous user feedback, which can improve the accuracy or increase the hit rate of the algorithm.

[0054] The features described herein can also be applied to other virtual environments, such as the configuration of other environment models in different environments. Attached Figure Description

[0055] To better understand the invention and its advantages, the following description is now taken in conjunction with the accompanying drawings.

[0056] The invention is further illustrated below with reference to exemplary embodiments, which are given in the schematic diagrams of the accompanying drawings.

[0057] In the picture:

[0058] Figure 1 A flowchart illustrating a computer-implemented method for configuring a virtual testing system according to a preferred embodiment of the present invention, the virtual testing system being used to test the vehicle functions of a motor vehicle;

[0059] Figure 2 A flowchart illustrating a computer-implemented method for providing a trained machine learning algorithm according to a preferred embodiment of the present invention, the machine learning algorithm being used to configure a virtual testing system for testing vehicle functions of a motor vehicle; and

[0060] Figure 3 A schematic diagram of a system for configuring a virtual testing system according to a preferred embodiment of the present invention is shown, the virtual testing system being used to test the vehicle functions of a motor vehicle.

[0061] Unless otherwise stated, the same reference numerals denote the same elements in the figures. Detailed Implementation

[0062] Figure 1 The method shown includes providing an S1 data record DS, which includes first data D1 from a plurality of input ports 10 of the artifact 12 to be tested and second data D2 from a plurality of output ports 14 of at least another artifact 16 to be tested.

[0063] Furthermore, the method includes applying the S2 machine learning algorithm A1 to the data record DS, wherein a confidence value K is determined for each of the plurality of input ports 10 of the artifact 12 to be tested, for each of the plurality of output ports 14 of at least another artifact 16 to be tested.

[0064] Furthermore, the method includes, for each of the plurality of input ports 10 of the artifact 12 to be tested, assigning, according to a first condition B1, at least one output port 14 of another artifact 16 to be tested with the highest confidence value K; and, according to a second condition B2, creating, a list of output ports 14 with the highest confidence value K, or, according to a third condition B3, not assigning, output ports 14, for configuring the connection between the input ports 10 of the artifact 12 to be tested and the adapted output ports 14 of at least one other artifact 16 to be tested.

[0065] The first condition B1 is given as follows: if the confidence value K is greater than or equal to a predetermined first threshold SW1, in particular greater than or equal to 99%, then the output port 14 of at least another artifact 16 to be tested with the highest confidence value K is automatically assigned.

[0066] Furthermore, the second condition B2 is given as follows: if the confidence value K is less than a predetermined first threshold SW1, specifically less than 99%, and greater than or equal to a predetermined second threshold SW2, specifically greater than or equal to 80%, then a list of output ports 14 of a predetermined size with the highest confidence value K is created.

[0067] Based on the created list of predetermined sizes, the user can then make a manual selection of the appropriate output port 14.

[0068] Here, in particular, the confidence values ​​K included in the created list are sorted in ascending or descending order. This list also includes confidence values ​​K that are less than a predetermined first threshold SW1, specifically less than 99%, and greater than or equal to a predetermined second threshold SW2, specifically greater than or equal to 80%.

[0069] The third condition B3 is given as follows: if the confidence value K is less than the predetermined second threshold SW2, especially less than 80%, then the allocation S3c of output port 14 will not be implemented.

[0070] The data record DS also has third data D3 from multiple output ports 14 of the artifact 12 to be tested and fourth data D4 from multiple input ports 10 of at least another artifact 16 to be tested.

[0071] In addition, the data record DS has all the input ports 10 and all the output ports 14 of the artifact 12 to be tested and at least another artifact 16 to be tested, or the input ports 10 and output ports 14 of the artifact 12 to be tested and at least another artifact 16 to be tested, selected by the user or automatically suggested.

[0072] The implemented assignments from at least one output port 14 of the other man-made object 16 to each input port 10 of the man-made object 12 to be tested are recorded in the log file. The man-made object 12 and at least one other man-made object 16 to be tested are virtual control units of a motor vehicle. Alternatively, the man-made object 12 and at least one other man-made object 16 to be tested can be provided by an environmental model.

[0073] Figure 2A flowchart is shown of a method for providing a computer implementation of a trained machine learning algorithm A1 according to a preferred embodiment of the present invention, the machine learning algorithm being used to configure a virtual test system for testing vehicle functions of a motor vehicle.

[0074] The method includes: providing S1' first training data record TD1, the first training data record including first data D1 of a plurality of input ports 10 of an artifact 12 to be tested and second data D2 of a plurality of output ports 14 of at least another artifact 16 to be tested.

[0075] Furthermore, the method includes: providing a second training data record TD2, the second training data record including: a confidence value K for each of the plurality of input ports 10 of the artifact 12 to be tested, and for each of the plurality of output ports 14 of at least another artifact 16 to be tested.

[0076] Furthermore, the method includes: training the S3' machine learning algorithm A1 by optimizing algorithm A2, which specifically computes the extrema of the loss function in the case of using random learning to configure the virtual test system.

[0077] The optimization algorithm A2 calculates the allocation of the output port 14 of at least one other artifact 16 to be tested, using the name (in particular, a name with letters or alphanumeric characters) and / or metadata (in particular, a data type) of the input port 10 of the artifact 12 to be tested.

[0078] The confidence value K of the second training data record DS is less than a predetermined first threshold, specifically less than 99%, and greater than or equal to a predetermined second threshold SW2, specifically greater than or equal to 80%.

[0079] In addition to the first training data record TD1 and the second training data record TD2, the machine learning algorithm A1 is also trained based on the user-implemented use of or non-use of the output port 14 of at least another artifact 16 to be tested with the highest confidence value K, or without the output port 14 being assigned.

[0080] Figure 3 A schematic diagram of system 1 for configuring a virtual testing system according to a preferred embodiment of the present invention is shown. The virtual testing system is used to test the vehicle functions of a motor vehicle.

[0081] System 1 includes a data storage 18 configured to provide a data record DS, which includes first data D1 from a plurality of input ports 10 of an artifact 12 to be tested and second data D2 from a plurality of output ports 14 of at least another artifact 16 to be tested.

[0082] In addition, system 1 includes a computing unit 20 configured to apply a machine learning algorithm A1 to a data record DS, wherein for each of the plurality of input ports 10 of the artifact 12 to be tested, a confidence value K can be determined for each of the plurality of output ports 14 of at least another artifact 16 to be tested.

[0083] In addition, system 1 includes: computing unit 20 is further configured to, for each of the plurality of input ports 10 of the artifact 12 to be tested, allocate at least one output port 14 of another artifact 16 to be tested with the highest confidence value K according to a first condition B1; create a list of output ports 14 with the highest confidence value K according to a second condition B2, or not allocate an output port 14 according to a third condition B3, for configuring the connection between the input ports 10 of the artifact 12 to be tested and the adapted output ports 14 of at least one other artifact 16 to be tested.

[0084] While specific implementations have been illustrated and described herein, it will be apparent to those skilled in the art that various alternative and / or equivalent implementations exist. It should be noted that one or more exemplary implementations are merely examples and are not intended to limit the scope, applicability, or configuration in any way.

[0085] More precisely, the foregoing summary and detailed description provide convenient guidance to those skilled in the art for implementing at least one exemplary embodiment, wherein, it is self-evident, different changes can be made in the functional scope and arrangement of the elements without departing from the scope of protection of the present invention.

[0086] Generally, this application is intended to cover modifications, adaptations, or variations of the embodiments shown herein. For example, the order of method steps may be changed. Furthermore, the methods may be implemented at least partially sequentially or in parallel.

Claims

1. A computer-implemented method for configuring a virtual testing system for testing the vehicle functions of a motor vehicle, the method comprising the following steps: Provides (S1) a data record (DS) comprising first data (D1) of a plurality of input ports (10) of the artifact (12) to be tested and second data (D2) of a plurality of output ports (14) of at least another artifact (16) to be tested; characterized in that, A machine learning algorithm (A1) is applied to a data record (DS), wherein a confidence value (K) is determined for each of the plurality of input ports (10) of the artifact (12) to be tested, for each of the plurality of output ports (14) of the at least other artifact (16) to be tested. For each of the plurality of input ports (10) of the artifact (12) to be tested, an output port (14) of the at least other artifact (16) to be tested with the highest confidence value (K) is assigned (S3a) according to a first condition (B1); a list of output ports (14) with the highest confidence value (K) is created (S3b) according to a second condition (B2) or no output port (14) is assigned (S3c) according to a third condition (B3) for configuring the connection between the input ports (10) of the artifact (12) to be tested and the adapted output ports (14) of the at least other artifact (16) to be tested.

2. The computer-implemented method according to claim 1, wherein, The first condition (B1) is given as follows: if the confidence value (K) is greater than or equal to a predetermined first threshold (SW1), then the output port (14) of the at least other artifact (16) to be tested is automatically assigned the highest confidence value (K).

3. The computer-implemented method according to claim 2, wherein, The second condition (B2) is given as follows: if the confidence value (K) is less than a predetermined first threshold (SW1) and greater than or equal to a predetermined second threshold (SW2), then a list of output ports (14) of a predetermined size with the highest confidence value (K) is created.

4. The computer-implemented method according to claim 3, wherein, Based on the created list of predetermined sizes, manual selection of the appropriate output port (14) can be made.

5. The computer-implemented method according to claim 3 or 4, wherein, The confidence values ​​(K) included in the created list are sorted by confidence value (K); and the list includes confidence values ​​(K) that are less than a predetermined first threshold (SW1) and greater than or equal to a predetermined second threshold (SW2).

6. The computer-implemented method according to claim 3 or 4, wherein, The third condition (B3) is given as follows: if the confidence value (K) is less than the predetermined second threshold (SW2), then the allocation of the output port (14) is not implemented (S3c).

7. The computer-implemented method according to any one of claims 1 to 4, wherein, The data record (DS) also has third data (D3) from multiple output ports (14) of the artifact (12) to be tested and fourth data (D4) from multiple input ports (10) of at least another artifact (16) to be tested.

8. The computer-implemented method according to claim 7, wherein, The data record (DS) has data of all input ports (10) and all output ports (14) of the artifact (12) to be tested and the at least one other artifact (16) to be tested, or selections of the input ports (10) and output ports (14) of the artifact (12) to be tested and the at least one other artifact (16) to be tested, made by the user or automatically suggested.

9. The computer-implemented method according to any one of claims 1 to 4, wherein, Record each implemented allocation from the output port (14) of the at least one other artifact (16) to the input port (10) of the artifact (12) to be tested in a log file.

10. The computer-implemented method according to any one of claims 1 to 4, wherein, The artifact to be tested (12) and the at least one other artifact to be tested (16) are a virtual control unit and / or environmental model of a motor vehicle.

11. The computer-implemented method according to claim 2, wherein, The first condition (B1) is given by the following: if the confidence value (K) is greater than or equal to 99%, then the output port (14) of the at least other artifact (16) to be tested is automatically assigned the highest confidence value (K).

12. The computer-implemented method according to claim 3, wherein, The second condition (B2) is given as follows: if the confidence value (K) is less than a predetermined first threshold (SW1) and greater than or equal to 80%, then a list of output ports (14) of a predetermined size with the highest confidence value (K) is created.

13. The computer-implemented method according to claim 3, wherein, The second condition (B2) is given as follows: if the confidence value (K) is less than 99% and greater than or equal to 80%, then a list of output ports (14) of a predetermined size with the highest confidence value (K) is created.

14. A method for providing a computer-implemented machine learning algorithm (A1) used in the method according to any one of claims 1 to 10 to configure a virtual testing system for testing vehicle functions of a motor vehicle, the method comprising the steps of: Provide (S1') a first training data record (TD1), which includes first data (D1) of multiple input ports (10) of the artifact (12) to be tested and second data (D2) of multiple output ports (14) of at least another artifact (16) to be tested. Provide (S2') a second training data record (TD2), which includes: For each of the plurality of input ports (10) of the artifact (12) to be tested, a confidence value (K) for each of the plurality of output ports (14) of the at least other artifact (16) to be tested; and The machine learning algorithm (A1) is trained (S3') by an optimization algorithm (A2), which calculates the extreme value of the loss function to configure the virtual test system.

15. The computer-implemented method according to claim 14, wherein, The optimization algorithm (A2) calculates the allocation of the output port (14) of the at least one other artifact (16) to be tested using the name and / or metadata of the input port (10) of the artifact (12) to be tested.

16. The computer-implemented method according to claim 14 or 15, wherein, The confidence value (K) of the second training data record (TD2) is less than a predetermined first threshold and greater than or equal to a predetermined second threshold (SW2).

17. The computer-implemented method according to claim 14 or 15, wherein, The machine learning algorithm (A1) is trained based on the user-implemented choice to either assign the output port (14) of the at least other artifact (16) to be tested with the highest confidence value (K) or not assign the output port (14).

18. A computer program product having program code to perform the method according to any one of claims 1 to 13 and 14 to 17 when the computer program is executed on a computer.

19. A computer-readable storage medium having program code of a computer program for implementing the method according to any one of claims 1 to 13 and 14 to 17 when the computer program is executed on a computer.

Images