Method for generating a training data set for an autonomous vehicle and system for generating a training data set

Abstract

The present invention relates to a method for generating a training dataset for an autonomously driving vehicle, the method including: defining a section in the form of position coordinates of a plurality of section landmarks; conducting a test drive with a first test vehicle along a measurement route selected based on the section definition; and detecting vehicle surrounding environment data along the measurement route using a sensor system of the first test vehicle. The method according to the present invention is characterized in that during the course of a first test drive, telemetry data of the first test vehicle is transmitted at time-discrete intervals to an external telemetry data detection unit, the telemetry data including position coordinates of a plurality of actual vehicle positions reached by the first test vehicle; a test drive allowance area and / or a target section allowance area are determined; the test drive allowance area is determined by interpolating the position coordinates of the actual vehicle positions or by interpolating cartographically refined actual vehicle positions; the target section allowance area is determined by interpolating the position coordinates of section target points or by interpolating cartographically refined section target points; a section coverage value is determined by comparing the position coordinates of the section target points with the test drive allowance area and / or by comparing the position coordinates of the actual vehicle position with the target section allowance area and / or by comparing the test drive allowance area with the target section allowance area; if the section coverage value is smaller than a predetermined coverage threshold, an updated section configuration is generated for a second test drive with the first test vehicle and / or another test vehicle.

Description

[Technical Field]

[0001] The present invention relates to a method for generating a training data set for an autonomously driving vehicle and to a system for generating a training data set, as defined in more detail in the preamble of claim 1. [Background technology]

[0002] Known driver assistance systems provide an automated driving mode monitored by a human vehicle driver for selected driving situations, such as slow driving in traffic jams. To achieve highly automated or fully automated driving, i.e., purely autonomous driving, learning robot systems are required, in addition to further developments in vehicle environment measurement technology and sensor data fusion. These learning robot systems are referred to as "intelligent driver assistance systems" based on their independent detection of the surrounding environment and the associated ability to interpret the detected data. AI systems combined with machine learning form the basis for developing this type of driving robot. In particular, unsupervised machine learning requires large amounts of data, including training datasets, which enable the application of statistical model-based algorithms.

[0003] One possible way to obtain a comprehensive training dataset for autonomous vehicles is to use swarm tactics. To this end, many owners of vehicles equipped with driver assistance systems that detect the vehicle's surroundings are expected to provide data generated during daily driving for vehicle development. For example, when the vehicle's autopilot is activated, data detected by the surrounding environment sensors can be automatically transmitted to the vehicle manufacturer's data server via a wireless communication connection. However, with this approach, the driver would also allow sensitive information about the driver, which can recognize their personal behavior and living environment, to be stored and further processed without the driver's control. Therefore, automobile manufacturers are pursuing an alternative approach to generating training datasets, in which test fleets are used to perform numerous test drives for data detection. This allows their test vehicles to be equipped with extensive sensor systems that provide much more detailed and extensive detection of other road users and objects in the vehicle's surroundings than the surrounding environment sensor technology integrated into mass-produced vehicles.

[0004] Another advantage of using test vehicles is that the quality of the generated training dataset is higher than that generated using swarm tactics. This is due to the ability to select suitable driving sections and driving conditions for the test vehicle's test drive and to perform driving by professional drivers. In contrast, a dataset of a private car user reflects the user's habits regarding section selection and typical driving time, so it cannot be excluded that certain ambient environment data will be over-represented in the training dataset generated by the swarm. However, a significant effort is required to properly select the test drive and the additional evaluation step to check whether the driving specifications were met after the test drive. Differences between the specified measurement route and the actual test drive may be caused by detours or unexpected events such as vehicle breakdowns, traffic restrictions, or accidents. For example, this may be due to driver error, such as missing a planned exit and thus being forced to detour via an alternative route.

[0005] Patent Document 1 describes a method for improving the control of an autonomous vehicle. It proposes transmitting journey-related data about objects located ahead along a planned journey route from an external server database to the vehicle, thereby simplifying the data about the surrounding environment detected by the vehicle's sensors and enabling predicted object associations based on the data set transmitted from the server, even in bad weather. Additionally, processed object-related data is returned to the server database. For this purpose, differences between the originally stored object data and newly measured object data during the journey are returned to the server database along with position references, which the server database uses to perform matching operations and determine reference points for different objects relative to each other along the journey. A fundamental prerequisite for performing the object-associated data processing is that the autonomous vehicle used can automatically interpret the surrounding environment measurement data by learning from a training data set. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] U.S. Patent No. 11,085,774 Summary of the Invention [Problem to be solved by the invention]

[0007] The problem on which the present invention is based is to provide an improved method for generating a training data set for an autonomously driving vehicle, as well as a system for generating a training data set to be used for said method. [Means for solving the problem]

[0008] According to the invention, this problem is solved by a method for generating a training data set with the features of claim 1. Advantageous configurations and developments as well as systems for generating a training data set emerge from the dependent claims.

[0009] The method for generating a training data set for an autonomously driving vehicle as described above includes defining a section in the form of position coordinates of a plurality of section landmarks, conducting a test drive with a first test vehicle along a measurement route selected based on the section definition, and detecting vehicle environment data along the measurement route using a sensor system of the first test vehicle.

[0010] The method according to the invention for generating a training data set comprises: - transmitting telemetry data of the first test vehicle to an external telemetry data detection unit at discrete time intervals during the course of the first test drive, the telemetry data including position coordinates of a plurality of actual vehicle positions reached by the first test vehicle; - the test drive tolerance range and / or the target section tolerance range are determined; - the test driving tolerance zone is determined by interpolation of the position coordinates of the actual vehicle position or by interpolation of the cartographically refined actual vehicle position; the target section tolerance zone is determined by interpolation of the position coordinates of the section target points or by interpolation of cartographically refined section target points; - determining a section coverage value by comparing the position coordinates of the section target points with the test driving tolerance area and / or by comparing the position coordinates of the actual vehicle position with the target section tolerance area and / or by comparing the test driving tolerance area with the target section tolerance area; If the section coverage value is less than a predetermined coverage threshold, an updated section configuration is generated for a second test drive with the first test vehicle and / or at least one other test vehicle. It is characterized by:

[0011] The method according to the invention simplifies and automates the evaluation step required after the test drive to determine the degree of correlation between the measurement route defined by the section landmarks and the test drive that was actually performed, which makes it possible to automatically determine whether sufficient environmental data has already been determined for the given measurement route or whether a new test drive with the test vehicle using an updated configuration for the measurement route is required.

[0012] The position information for the test drive can be determined from a set of sensor data recorded during the test drive, which typically has a link to continuously detected vehicle environment data. The inventors have recognized that, particularly in large test vehicle fleets, the overall effort for evaluating the test drive and for determining new measurement routes can be reduced if test vehicle telemetry data is evaluated to determine the current location information instead of position information from the vehicle environment data.

[0013] Vehicle telemetry data is primarily used for fleet management and includes location information for identifying individual test vehicles, as well as diagnostic data transmitted to an external telemetry data detection unit for vehicle monitoring. For example, information about maintenance intervals and driving-related systems, such as the state of charge of an electric vehicle battery, is returned. Telemetry data transmissions are time-discrete, typically at intervals of minutes, e.g., two minutes. Compared to the sensor data sets of detected vehicle environment data, the volume of telemetry data is significantly reduced, significantly reducing the computational effort required to track the test vehicle's position. However, the low frequency of telemetry data transmissions makes it difficult to directly compare the position coordinates of section landmarks derived from the measurement route's section definition with the position coordinates of the actual vehicle position determined from the telemetry data for the test vehicle's actual location.

[0014] The inventors have therefore realised that a further step requires data interpolation to provide at least one tolerance zone for determining the degree of correlation between the predetermined measurement route and the distance actually travelled during the test drive. In accordance with the first embodiment, a test drive tolerance zone is determined, and a comparison of the relative position with the test drive tolerance zone is carried out for each predetermined section target point of the measurement route. To advantageously determine the test drive tolerance zone, a series of section segments are first generated using linear interpolation of successive actual vehicle positions, and the resulting section segments are then extended into successive test drive tolerance zones by a circular process object having a predetermined tolerance radius.

[0015] In a further development, the aforementioned interpolation for determining the test driving allowance zone is not performed directly using the position coordinates of the actual vehicle position, but using cartographically refined actual vehicle positions. Here, cartographically refined actual vehicle positions are understood to mean an extended position data set generated by subsequently inserting additional position points between two consecutive actual vehicle positions determined from telemetry data. For this purpose, position points, for example at equal intervals, between the transmitted actual vehicle positions are defined based on a map stored in an external database, which contains position coordinates of drivable routes that provide the shortest road connections available to the test vehicle between the observed consecutive actual vehicle positions.

[0016] The comparison of the position coordinates of the section landmarks with the test driving allowance area can be performed in various ways. In the simplest case, section landmarks whose position coordinates lie within the test driving allowance area are determined. For section landmarks located outside the test driving allowance area, the maximum permissible distance to the test driving allowance area is considered as another selection criterion for section landmarks that are still associated with the test driving allowance area. Here, the vertical distance to the boundary of the test driving allowance area can be used as the distance to the test driving allowance area. Alternatively, this distance is defined as the distance from each section landmark to the closest section segment for the time-sequential vehicle actual positions determined by the aforementioned linear interpolation. In this case, for a particularly time-efficient implementation, a search is performed in a data set of vehicle actual positions structured as a KD tree to determine the closest section segment.

[0017] After determining the section target points associated with the test driving allowance area, a section coverage value is determined, wherein in one possible embodiment, the number of section target points associated with the test driving allowance area is set in relation to the total number of section target points. In an alternative embodiment, a series of section segments are generated using linear interpolation of the section target points, and the cumulative length of each section segment formed between two section target points associated with the test driving allowance area is set in relation to the total length of all section segments.

[0018] In a further method step, it is checked whether the section coverage value is lower than a predetermined coverage threshold, and if so, an updated section configuration is performed for a second test drive that has to be performed by the same test vehicle, i.e. the first test vehicle and / or at least one other test vehicle from the vehicle fleet.

[0019] In the second embodiment, data interpolation for defining the tolerance range is performed by interpolating the position coordinates of the section target points or by interpolating the position coordinates of the cartographically refined section target points, and the target section tolerance range is determined by this interpolation. By checking the relative positions between the position points and the geographically limited tolerance range as described above, in a subsequent step, the position coordinates of the actual vehicle position are compared with the target section tolerance range, and from this comparison, a section coverage value is determined, as in the first embodiment.

[0020] For the third embodiment, the interpolation is performed twice, and the test drive allowable range and the target section allowable range are defined. The extent of the overlap area between the test drive allowable range and the target section allowable range is then determined to define a section coverage value, which is set relative to the extent of the position area including the test drive allowable range and the target section allowable range. The section coverage value can also be determined from this ratio.

[0021] The system according to the present invention for generating a training data set includes a telemetry data detection unit and is suitable for implementing the method according to the present invention described above. In a preferred embodiment, the system for generating a training data set performs an adaptation of the tolerance radius for a circular process object for data interpolation. For this purpose, the method according to the present invention using telemetry data of a fleet of vehicles is first implemented in the system for generating a training data set with a small selected tolerance radius, and the resulting system runtime is determined. The tolerance radius is then gradually increased until it falls below a predetermined runtime setting.

[0022] In another embodiment, the method according to the invention for generating a training data set is combined with an additional evaluation of the test runs used for this purpose. Advantageously, the resources used for the test vehicle fleet or for individual test runs, such as fuel consumption or the validity of the vehicle maintenance measures carried out, are evaluated.

[0023] Further advantageous configurations of the method for generating a training data set are explained in more detail below with reference to the drawings and will also become apparent from the examples. [Brief explanation of the drawings]

[0024] [Figure 1] 1 illustrates an embodiment of a method according to the present invention for generating a training data set. [Figure 2] 2 shows a first embodiment for the interpolation step of the example according to FIG. 1; [Figure 3] 2 shows a second embodiment for the interpolation step of the example according to FIG. 1; DETAILED DESCRIPTION OF THE INVENTION

[0025] The block diagram shown in Figure 3 illustrates a first embodiment of the method according to the invention for generating a training data set. In method step A, a section is defined 1 by wirelessly transmitting the position coordinates of a number of section landmarks to the navigation system of the test vehicle. In this regard, Figure 1 shows a part of the section 1 for a measurement route with a branch, in which three section landmarks 3.1, 3.2, 3.3 to be reached are shown.

[0026] With respect to method step B, the test vehicle performs a test drive based on section configuration 1 and detects the vehicle's surroundings using an expanded range of measurement techniques, including optical sensors, radar systems, laser-based measurement systems, and ultrasonic and infrared sensors, thereby increasing the raw data for generating the training data set. Additionally, the telemetry data is transmitted in a time-discrete manner, for example, at intervals of 1 / 120 seconds, to an external telemetry data detection unit, whereby position coordinates for the vehicle's actual position reached at the time of telemetry data collection are extracted from the telemetry data and provided to the data processing unit. Here, FIG. 1 shows the vehicle's actual positions 4.1, ..., 4.4 for a portion of the test drive.

[0027] In method step C, the test driving allowance region 5 is determined, whereby successive vehicle actual positions 4.1, ..., 4.4 are first connected using linear interpolation so that a series of section segments 6 is determined. By way of example, only one section segment 7 is shown, which is formed between the vehicle actual positions 4.2 and 4.3. For further interpolation, the series of section segments 6 are extended into the successive test driving allowance region 5 shown in FIG. 3 by means of a circular process object 8 which has a predetermined tolerance radius and which is moved along the individual section segments 7 relative to its center point.

[0028] In method step D shown in Fig. 3, the position coordinates of the section landmarks 3.1, 3.2, 3.3 are compared with the test driving allowance area 5. In the first embodiment, it is determined which of the section landmarks 3.1, 3.2, 3.3 are located directly within the test driving allowance area 5. Alternatively, a second embodiment is shown in Fig. 1, in which section landmarks 3.1, 3.2, 3.3 are considered if their vertical distance 9 to the nearest section segment 7 is shorter than a predetermined maximum distance. In order to map the section landmarks 3.1, 3.2, 3.3 to the nearest section segment 7 as computationally efficient as possible, a search is preferably performed in a data set of vehicle actual positions 4.1, ..., 4.4, which is structured as a KD tree.

[0029] A further method step E relates to determining the section coverage value, in the simplest case, the number of section target points 3.1, 3.2, 3.3 associated with the test drive allowance region 5 is determined in relation to the total number of section target points 3.1, 3.2, 3.3. In an advantageous alternative embodiment not shown in detail, a series of target section segments is generated using linear interpolation of the section target points 3.1, 3.2, 3.3, and the cumulative length of the target section segments formed between each two section target points 3.1, 3.2, 3.3 associated with the test drive allowance region 5 is determined in relation to the total length of all target section segments.

[0030] Method step F relates to comparing the calculated section coverage value with a predetermined coverage threshold. If the measurement route has not been driven accurately enough, a section coverage value lower than the predetermined coverage threshold will persist. The system then updates section configuration 1 and outputs it to the first test vehicle and / or another test vehicle of the fleet for a second test drive to generate a training data set.

[0031] Additionally, a preferred system for generating the training data set, not shown in detail, is designed such that an automatic adaptation of the tolerance radius for the circular process object 8 for interpolating the position coordinates of the vehicle actual positions 4.1, ..., 4.4 is performed based on run-time settings.

[0032] 2 shows another development of the method according to the invention for generating a training data set, in which instead of a direct interpolation of the position coordinates of the vehicle actual positions 4.1, ..., 4.4, an intermediate step is carried out to determine the test drive allowance area 5. In this regard, a database with route information is used to generate cartographically refined vehicle actual positions 10.1, ..., 10.9, in which, in addition to the telemetered and transmitted vehicle actual positions 4.1, ..., 4.4, further position points are added along the shortest road connections available for the test drive before the interpolation step.

Claims

1. 1. A method for generating a training dataset for an autonomous vehicle, comprising: A section definition (1) in the form of position coordinates of a plurality of section target points (3.1, 3.2, 3.3); Conducting a test run by a first test vehicle along a measurement route (2) selected based on the section setting (1); and detecting vehicle environment data along the measurement route (2) using a sensor system of the first test vehicle. - in the course of the first test drive, telemetry data of the first test vehicle is transmitted to an external telemetry data detection unit at discrete intervals in time, the telemetry data comprising position coordinates of a plurality of actual vehicle positions (4.1, . . . , 4.4) reached by the first test vehicle; - the test drive tolerance range (5) and / or the target section tolerance range are determined; - the test driving allowance area (5) is determined by interpolation of the position coordinates of the vehicle's actual position (4.1,...,4.4) or by interpolation of the cartographically refined vehicle's actual position (10.1,...,10.9); - the target section tolerance area is determined by interpolation of the section target points (3.1, 3.2, 3.3) or by interpolation of the position coordinates of the cartographically refined section target points; - determining a section coverage value by comparing the position coordinates of the section target points (3.1, 3.2, 3.3) with the test driving tolerance area (5) and / or by comparing the position coordinates of the actual vehicle position (4.1, . . . , 4.4) with the target section tolerance area and / or by comparing the test driving tolerance area (5) with the target section tolerance area; - A method for generating a training dataset, characterized in that if the section coverage value is less than a predetermined coverage threshold, an updated section configuration is generated for a second test drive with the first test vehicle and / or another test vehicle.

2. 2. The method for generating a training data set according to claim 1, characterized in that, to determine the test driving allowable range (5), a linear interpolation is performed on successive actual vehicle positions (4.1, . . . , 4.4) in time and / or on successive section target points (3.1, 3.2, 3.3), the interpolation resulting in a series of section portions (6).

3. 3. The method for generating a training data set according to claim 2, characterized in that the series of section portions (6) are extended into successive test drive tolerance areas (5) and / or successive target section tolerance areas by circular process objects (8) having a predetermined tolerance radius.

4. 4. A method for generating a training data set according to claim 1, wherein a comparison of the position coordinates of the section target points (3.1, 3.2, 3.3) with the test driving allowance area (5) determines section target points (3.1, 3.2, 3.3) whose position coordinates lie within the test driving allowance area (5), and / or a comparison of the actual vehicle positions (4.1, ..., 4.4) with the target section allowance area determines actual vehicle positions (4.1, ..., 4.4) whose position coordinates lie within the target section allowance area.

5. 5. The method for generating a training data set according to claim 1, wherein the position coordinates of the section target points (3.1, 3.2, 3.3) are compared with the test driving tolerance area (5) to determine section target points (3.1, 3.2, 3.3) that are within a maximum permissible distance to the test driving tolerance area, and / or the position coordinates of the vehicle actual positions (4.1, ..., 4.4) are compared with the target section tolerance area to determine actual vehicle positions (4.1, ..., 4.4) that are within a maximum permissible distance to the target section tolerance area.

6. 6. The method for generating a training data set according to claim 5, characterized in that the distance to the test driving tolerance area (5) is defined as the vertical distance of each section target point (3.1, 3.2, 3.3) to the boundary of the test driving tolerance area (5) and / or the distance to the target section tolerance area is defined as the vertical distance of each vehicle actual position (4.1, . . . , 4.4) to the boundary of the target section tolerance area.

7. 6. The method for generating a training data set according to claim 5, characterized in that the distance to the test driving tolerance area is determined as the distance between each section target point (3.1, 3.2, 3.3) and the nearest section portion (7) defined by linear interpolation of successive actual vehicle positions (4.1, . . . , 4.4) and / or the distance to the target section tolerance area is determined as the distance between the nearest section portion defined by linear interpolation of successive section target points (3.1, 3.2, 3.3).

8. 8. The method for generating a training data set according to claim 7, characterized in that a search is performed in a data set structured as a KD tree of the vehicle actual positions (4.1, . . . , 4.4) and / or the section target points (3.1, 3.2, 3.3) to define the closest section portion (7), determined by linear interpolation.

9. To determine the section coverage value, the number of section target points (3.1, 3.2, 3.3) associated with the test driving allowance area is determined in relation to the total number of section target points (3.1, 3.2, 3.3), or a series of target section segments is generated by linear interpolation of the section target points (3.1, 3.2, 3.3), and the cumulative length of each target section segment formed between two section target points (3.1, 3.2, 3.3) associated with the test driving allowance region is determined in relation to the total length of all target section segments, or The number of the vehicle actual positions (4.1, . . . , 4.4) associated with the target section tolerance area is determined in relation to the total number of the vehicle actual positions (4.1, . . . , 4.4), or 2. The method for generating a training data set according to claim 1, wherein a series of vehicle actual position segments are generated using linear interpolation of the vehicle actual positions (4.1, . . . , 4.4), and the cumulative length of the vehicle actual position segments each extending between two vehicle actual positions (4.1, . . . , 4.4) associated with the target section tolerance region is set in relation to the total length of all vehicle actual position segments.

10. 1. A system for generating a training data set, comprising:

10. A system, characterized in that the system includes a telemetry data detection unit and is adapted to be able to carry out the method according to any one of claims 1 to 9.

11. 11. The system for generating a training data set according to claim 10, characterized in that automatic adaptation of the tolerance radius for circular process objects for interpolating position coordinates can be performed based on run-time settings.

Images