Computer-implemented method for grouping data

Abstract

The present invention relates to a computer-implemented method for grouping data, wherein the data comprise real data sets (6) and simulation data sets (3), wherein the real data sets (6) were each recorded during a real journey of a real motor vehicle, wherein the simulation data sets (3) were each recorded during a simulated journey of a simulated motor vehicle, wherein the real data sets (6) each comprise real bus data (5) and the simulation data sets (3) each comprise simulated bus data (1), wherein the real bus data (5) are data from a communication bus of the respective real motor vehicle and the simulated bus data (1) are data from a communication bus of the respective simulated motor vehicle, wherein the method comprises the following steps: - Comparison of the real bus data (5) with the simulated bus data (1) by an artificial intelligence; and - Summary of a first subgroup of real data sets (6) and a second subgroup of simulation data sets (3) into one group (7) based on comparison.

Description

[0001] The present invention relates to a computer-implemented method for grouping data according to claim 1.

[0002] Traffic scenarios can be complex and difficult to simulate realistically. Realistic simulated scenarios are beneficial for training artificial intelligence to correctly assess real-world scenarios and make appropriate decisions.

[0003] From DE 10 2020 210 600 A1, a system for performing XiL-based simulations is known in which an artificial intelligence is trained using operational data sets, which then generates test data sets for XiL simulations. The test data sets are classified by the artificial intelligence.

[0004] In contrast, the present invention is based on the objective of recognizing, in a data set comprising real data and simulation data, which data belong to similar scenarios.

[0005] This problem is solved by a method according to claim 1. Embodiments of the invention are specified in the dependent claims.

[0006] According to the procedure, the data comprises real-world datasets and simulation datasets. The real-world datasets were recorded during actual journeys with a real vehicle. The simulation datasets were recorded during simulated journeys with a simulated vehicle. Both the real-world and simulation datasets can be, for example, OpenScenario files or may include OpenScenario files.

[0007] The real-world datasets and / or the simulation datasets can each contain a scenario description that provides information about the specific scenario in which the respective real-world or simulation dataset was recorded. A scenario can, for example, encompass one or more situations in road traffic.

[0008] The real-world datasets each comprise real bus data. The simulation datasets each comprise simulated bus data. The real-world bus data is data from a communication bus of the respective real vehicle. The simulated bus data is data from a communication bus of the respective simulated vehicle.

[0009] The real bus data is compared with the simulated bus data by an artificial intelligence. Based on this comparison, a first subset of the real data and a second subset of the simulated data are combined into a single group. This can also be performed by the artificial intelligence. Using the comparison as the basis for the combination can specifically mean that a result of the comparison triggers the respective real or simulated data set to be added to the group. This combination can also be referred to as embedding the real and simulated data sets.

[0010] Real bus data and simulated bus data are particularly well-suited for comparison, as similar bus data strongly suggests that the scenarios in which the respective data were recorded are similar. Conversely, different bus data strongly suggests that the scenarios in which the respective data were recorded are different. The combined real and simulation datasets are therefore particularly well-suited as training data for artificial intelligence.

[0011] The real-world datasets and simulation datasets grouped together also enable both the identification of scenarios through a simulation description and the generation of a simulation description from a scenario that belongs to a real-world dataset.

[0012] According to one embodiment of the invention, the real data sets can each comprise real camera data from the respective real motor vehicle. This is particularly advantageous, for example, to be able to generate initial camera data for one or more of the simulation data sets of the second subgroup using the camera data from the real data sets of the first subgroup. Camera data is understood to mean, in particular, image and / or video data recorded by a camera arranged in or on the motor vehicle.

[0013] According to one embodiment of the invention, the real bus data and the simulated bus data can each comprise time series and data recorded by sensors. For example, the simulated bus data can consist of simulated time series and data recorded by simulated sensors. The real bus data can consist of real time series and data recorded by real sensors. Such bus data is particularly well suited for comparing the similarity of the scenarios in which the respective bus data was recorded.

[0014] According to one embodiment of the invention, a similarity measure between the real bus data and the simulated bus data can be determined by comparison. The real bus data of the real datasets of the first subgroup and the simulated bus data of the simulation datasets of the second subgroup can each exhibit a first similarity measure with each other that lies above a threshold value. Real bus data of a third subgroup of the real datasets and simulated bus data of a fourth subgroup of the simulation datasets can each exhibit a second similarity measure with the real bus data of the real datasets of the first subgroup and / or with the simulated bus data of the simulation datasets of the second subgroup, which lies below the threshold value. The similarity measure can provide an indication of the degree of similarity determined when comparing the datasets. The higher the similarity measure, the more similar the datasets are.

[0015] The similarity of the real-world and simulation datasets grouped together is advantageous because they correspond to similar scenarios. For example, a scenario description allows the extraction of relevant real-world and / or simulation datasets from the group.

[0016] It is possible that further subgroups of the real-world datasets and further subgroups of the simulation datasets can be combined into further groups. In this way, multiple groups can be created where the real-world and simulation datasets combined into each group exhibit a high degree of similarity and therefore likely correspond to similar scenarios.

[0017] According to one embodiment of the invention, the comparison, and preferably also the summarization, can be performed as self-supervised learning. For example, the real bus data and the simulated bus data can be free of labels added by a human user. This is particularly advantageous for minimizing the effort required for the comparison.

[0018] According to one embodiment of the invention, the artificial intelligence can be an artificial neural network that spatially groups similar data together and spatially separates dissimilar data. For this purpose, for example, "deep metric learning" can be used.

[0019] In this embodiment, the similarity measure can, for example, be the spatial proximity of the data to each other.

[0020] According to one embodiment of the invention, the real-world and simulation datasets grouped together can be used to generate new datasets. These new datasets can each comprise generated bus data and initial camera data. For example, the new datasets can be OpenScenario files. Due to the high degree of similarity between the real-world and simulation datasets in the group, the new datasets are realistic and similar to the real-world and simulation datasets in the group. Therefore, there is a high probability that they correspond to a scenario similar to the scenarios represented by the real-world and simulation datasets in the group.

[0021] This embodiment is particularly advantageous for generating realistic datasets without user interaction, where it is clear which realistic scenario they correspond to, so that they can be used, for example, as training data for further artificial intelligence.

[0022] According to one embodiment of the invention, the real data sets and simulation data sets combined in the group, and possibly also the new data sets, can be used as training data for a further artificial intelligence.

[0023] According to one embodiment of the invention, a description of the simulation data sets of the second subgroup can be generated from the real data sets of the first subgroup.

[0024] According to one embodiment of the invention, second generated camera data for one of the simulation data sets of the second subgroup can be generated using the real data sets of the first subgroup, preferably from the real camera data.

[0025] It is possible to execute a method according to one embodiment of the invention by a system. Such a system can comprise a digital processing unit and a digital electronic data storage device. The processing unit can, for example, be called a processor. Instructions can be stored in the data storage device. The processing unit can be configured to read and execute the instructions. The instructions can be configured to cause the processing unit, upon execution of the instructions, to carry out a method according to one embodiment of the invention.

[0026] Further features and advantages of the present invention will become clear from the following description of preferred embodiments with reference to the accompanying figures. These show Fig. 1 a schematic representation of a method according to an embodiment of the invention; and Fig. 2 a schematic representation of a method according to an embodiment of the invention.

[0027] Simulated bus data 1 and scenario descriptions 2 are contained in simulation datasets 3. These simulation datasets 3 can, for example, be or include OpenScenario files. Real camera data 4 and real bus data 5 are contained in real datasets 6.

[0028] In a comparison, similarities between the simulation datasets 3 and the real-world datasets 6 are determined. This can be done, for example, using an artificial neural network, particularly with deep metric learning. The simulation datasets 3 and the real-world datasets 6 with a similarity score above a certain threshold are grouped together as group 7. It is particularly possible that the comparison focuses primarily or exclusively on the simulated bus data 1 and the real-world bus data 5. This is especially advantageous because these datasets are readily comparable, even though the simulation datasets 3 do not contain camera data. The simulated bus data 1 and the real-world bus data 5 can, for example, each include time series and sensor signals.

[0029] The simulation datasets 3 and real-world datasets 6, grouped together as Group 7, thus exhibit a high degree of similarity in their respective simulated and real-world bus data 1 and 5. Therefore, the simulation datasets 3 and real-world datasets 6 grouped as Group 7 are particularly well-suited as training data for a further artificial intelligence, which can be used to generate simulation scenarios from real-world data and vice versa.

[0030] In Fig. Figure 2 illustrates how the simulated bus data 1, the real bus data 5, and the real camera data 4 are used as training data for the further artificial intelligence 8 with a generative model. The further artificial intelligence 8 then generates new datasets with generated bus data 9 and a generated scenario description 10. The new datasets can, for example, be in the form of OpenScenario files.

[0031] The in Fig.The method described in Figure 1 is particularly advantageous for generating realistic new datasets. Since the data from Group 7 are used to train the further artificial intelligence 8, this data consists of similar scenarios, meaning that the further artificial intelligence 8 is trained to generate datasets that correspond to such scenarios and are therefore particularly realistic. QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] DE 10 2020 210 600 A1

[0003]

Claims

[1] A computer-implemented method for combining data into a group, wherein the data comprise real data sets (6) and simulation data sets (3), wherein the real data sets (6) were each recorded during an actual journey of a real motor vehicle, wherein the simulation data sets (3) were each recorded during a simulated journey of a simulated motor vehicle, wherein the real data sets (6) each comprise real bus data (5) and the simulation data sets (3) each comprise simulated bus data (1), wherein the real bus data (5) is data of a communication bus of the respective real motor vehicle and the simulated bus data (1) is data of a communication bus of the respective simulated motor vehicle, wherein the method comprises the following steps: - Comparison of the real bus data (5) with the simulated bus data (1) by an artificial intelligence; and - combining a first subgroup of the real data sets (6) and a second subgroup of the simulation data sets (3) into a group (7) based on the comparison. [2] Method according to claim 1, characterized by that the real data sets (6) each comprise real camera data (4) of the respective real motor vehicle. [3] Method according to one of the preceding claims, characterized by that the real bus data (5) and the simulated bus data (1) each comprise time series and data recorded by sensors. [4] Method according to one of the preceding claims, characterized bythat during the comparison a similarity measure of the real bus data (5) and the simulated bus data (1) is determined, wherein the real bus data (5) of the real data sets (6) of the first subgroup and the simulated bus data (1) of the simulation data sets (3) of the second subgroup each have a first similarity measure with one another which lies above a threshold value, wherein real bus data of a third subgroup of the real data sets and simulated bus data of a fourth subgroup of the simulation data sets each have a second similarity measure with the real bus data (5) of the real data sets (6) of the first subgroup and / or with the simulated bus data (5) of the simulation data sets (3) of the second subgroup, wherein the second similarity measure lies below the threshold value. [5] Method according to one of the preceding claims, characterized by that the comparison is carried out as self-supervised learning. [6] Method according to one of the preceding claims, characterized by that artificial intelligence is an artificial neural network that spatially combines similar data and spatially separates different data. [7] Method according to one of the preceding claims, characterized by that the real data sets (6) and simulation data sets (3) summarized in the group are used to generate new data sets, wherein the new data sets each comprise generated bus data (9) and first generated camera data. [8] Method according to one of the preceding claims, characterized by that the real data sets (6) and simulation data sets (3) summarized in the group are used as training data for another artificial intelligence (8). [9] Method according to one of the preceding claims, characterized bythat a description of the simulation data sets (3) of the second subgroup is generated from the real data sets (6) of the first subgroup. [10] Method according to one of claims 2 to 9, characterized by that second generated camera data are generated for one of the simulation data sets (3) of the second subgroup using the real data sets (6) of the first subgroup.

Images