Training data set generation method and training data set generation system for autonomous vehicle

Abstract

The invention relates to a training data set generation method for an automatic driving vehicle. The method comprises the following steps: executing road section presetting in the form of position coordinates of a plurality of road section set points; the measurement travel is carried out with the first test vehicle along a measurement path selected according to the road section specification, and vehicle environment data is acquired along the measurement path by means of a sensor system of the first test vehicle. The method according to the invention is characterized in that in the first measurement driving process, telemetry data of the first test vehicle is transmitted to an external telemetry data acquisition unit in a time discrete form, and the telemetry data comprises position coordinates of the actual position of the vehicle reached by the first test vehicle; and determining a measurement driving tolerance interval and / or a target road segment tolerance interval; wherein the measurement driving tolerance interval is determined by interpolating the position coordinates of the actual position of the vehicle or the actual position of the vehicle subjected to cartographic correction; and wherein a target road segment tolerance interval is determined by interpolating the position coordinates of the road segment setpoints or the road segment setpoints after cartographic correction; and determining a road section coverage value by comparing the position coordinates of the road section setpoint with the measured driving tolerance interval, and / or by comparing the position coordinates of the actual position of the vehicle with the target road section tolerance interval, and / or by comparing the measured driving tolerance interval with the target road section tolerance interval; and if the road segment coverage value is less than the preset coverage threshold, generating an updated road segment preset for a second measured travel using the first test vehicle and / or another test vehicle.

Description

Technical Field

[0001] The invention relates to a method of generating a training data set for an autonomous vehicle and a system for generating a training data set according to the type defined in more detail in the preamble of claim 1. Background Art

[0002] Driver assistance systems are known that provide an automated driving mode monitored by a human driver in selected driving scenarios (e.g. slow driving in a traffic jam). In order to achieve a higher degree of automation, even highly automated or fully automated driving, i.e. purely automated driving, in addition to further development of vehicle environment measurement technology and sensor data fusion technology, learning robot systems are also required. These systems are called "intelligent driver assistance systems" because they can autonomously capture the environment and interpret the data collected thereby. AI systems combined with machine learning form the basis for the development of such driving robots. In this regard, unsupervised machine learning requires in particular a large amount of data including training data sets in order to be able to apply algorithms based on statistical models.

[0003] One possible approach to obtaining extensive training data sets for autonomous vehicles is to use a swarm strategy. For this purpose, the owners of many vehicles equipped with driver assistance systems for recording the vehicle's surroundings are asked to provide data generated during daily driving for vehicle development. For example, if autonomous driving is activated in the vehicle, the data collected by the surrounding sensors can be automatically transmitted to the vehicle manufacturer's data server via a wireless communication connection. However, this approach also requires the driver's consent to the uncontrolled storage and further processing of sensitive information about him, which can be used to gain insights into his personal behavior and his living environment. Therefore, vehicle manufacturers have adopted another approach to generating training data sets, for which a fleet of test vehicles is used to carry out a large number of measurement drives for data collection. This makes it possible to equip the factory's own test vehicles with extensive sensor systems, which can record other traffic participants and objects in the vehicle's surroundings in a more detailed and extensive manner than the vehicle's surrounding measurement technology installed in series production.

[0004] Another advantage of using test vehicles is that the quality of the generated training data sets can be improved compared to data generated by swarm strategies. This is because suitable driving routes and driving scenarios can be selected for the test vehicle test drives and professional drivers can be arranged to carry out the drives. In contrast, the data sets of private car users reflect their habits in terms of route selection and typical driving times, so that the problem of oversampling of specific environmental data in the training data sets generated by swarm cannot be eliminated. However, the appropriate selection of the measurement drives requires a lot of planning work and additional evaluation steps after the measurement drives are carried out to check whether the driving regulations are met. Deviations between the specified measurement route and the actual measurement drive can occur due to detours or unexpected events, such as vehicle breakdowns, traffic restrictions or accidents. Errors caused by the driver who is carrying out the work, such as missing the planned exit and having to take an alternative route, should also be taken into account.

[0005] US11,085,774B2 describes a method for improving automatic vehicle control. For objects located ahead on the planned driving route, it is proposed that driving-related data be transmitted to the vehicle from an external server database, so that the environmental data collected by the vehicle via sensors can be associated with the expected objects in a simplified manner, even in bad weather conditions with the help of the data set transmitted by the server. In addition, the data processed for the object are returned to the server database. For this purpose, the difference between the initially saved object data and the object data remeasured when driving past is returned to the server database together with a position reference, which uses these data to perform a comparison operation and determine reference points for different objects associated with each other along the driving route. For the object-related data processing shown, the basic prerequisite for its implementation is that the autonomous driving vehicle used has learned the relevant capabilities through the training data set in order to perform machine interpretation of the environmental measurement data. Summary of the invention

[0006] The object of the present invention is to provide an improved training data set generation method for an autonomous driving vehicle and a training data set generation system used for this purpose.

[0007] According to the invention, this object is achieved by a method for generating a training data set having the features of claim 1. Advantageous embodiments and developments as well as a system for generating a training data set are revealed by the dependent claims.

[0008] A general method for generating a training data set for an autonomous vehicle comprises performing a route preset in the form of position coordinates of a plurality of route set points, performing a measurement drive with a first test vehicle along a measurement path selected according to the route preset, and collecting vehicle environment data along the measurement path with the aid of a sensor system of the first test vehicle.

[0009] The advantages of the training data set generation method according to the present invention are:

[0010] - during the first measurement drive, transmitting telemetry data / distance measurement data of the first test vehicle to an external telemetry data acquisition unit in a time-discrete form, and the telemetry data includes position coordinates of a plurality of actual vehicle positions reached by the first test vehicle;

[0011] -determining a measured driving tolerance interval and / or a target road section tolerance interval;

[0012] - wherein the measured travel tolerance interval is determined by interpolating the position coordinates of the actual position of the vehicle or the actual position of the vehicle after cartographic correction;

[0013] - wherein the target road section tolerance interval is determined by interpolating the position coordinates of the road section set points or the road section set points corrected by cartography;

[0014] - by comparing the position coordinates of a set point of the segment with a measured driving tolerance interval, and / or by comparing the position coordinates of the actual position of the vehicle with a target segment tolerance interval, and

[0015] / or determining a segment coverage value by comparing a measured driving tolerance interval with a target segment tolerance interval; and

[0016] If the segment coverage value is less than a preset coverage threshold, generating an updated segment preset for a second measurement drive using the first test vehicle and / or a further test vehicle.

[0017] The method according to the invention makes it possible to simplify and automate the evaluation steps required after the performed measurement drive in order to determine the degree of agreement / correlation between the measurement path determined by the segment setpoints and the measurement drive segment actually travelled. On this basis, a decision can be made by the machine as to whether sufficient environmental data have been determined for the preset measurement path or whether the test vehicle must perform a new measurement drive using an updated measurement path preset.

[0018] The location information of the measurement drive can be determined from the sensor data cloud recorded during the measurement drive, which is usually associated with the continuously collected vehicle environment data. The inventors have found that the overall cost of evaluating the measurement drive and determining new measurement routes can be reduced, especially for larger fleets of test vehicles, if the telemetry data of the test vehicle is analyzed for the determination of the location information instead of using the location information from the vehicle environment data.

[0019] The telemetry data of the vehicle is mainly used for fleet management and, in addition to location information for locating the individual test vehicles, also includes diagnostic data, which is transmitted to an external telemetry data acquisition unit for vehicle monitoring. For example, information on maintenance intervals and information on driving-related systems, such as the charge state of the traction battery, is fed back. The telemetry data is sent in a time-discrete form and is usually sent at time intervals of minutes, for example at time intervals of two minutes. Compared to the sensor data cloud of the collected vehicle environment data, the significantly reduced data volume of the telemetry data allows for significantly reduced computing power consumption for position tracking of the test vehicle. However, the small transmission clock beat / transmission frequency of the telemetry data leads to the following problem: namely, for the measurement route, the direct comparison between the position coordinates of the section set points in the section preset and the position coordinates of the actual vehicle position actually reached by the test vehicle determined from the telemetry data is not directly suitable for the evaluation of the measurement drive.

[0020] Therefore, the inventors have found that it is necessary to perform data interpolation in a further step, which at least provides a tolerance interval to determine the degree of agreement between the preset measurement path and the actually traversed measurement travel section. For a first embodiment, a measurement travel tolerance interval is determined, and for each specified section set point of the measurement path, a relative position comparison is performed relative to the measurement travel tolerance interval. Here, in order to advantageously determine the measurement travel tolerance interval, a series of section segments are first generated by linear interpolation of the actual positions of the vehicle consecutive in time, and the obtained section segments are expanded by a circular process object with a preset tolerance radius to form a coherent measurement travel tolerance interval.

[0021] In a further development, provision is made that the above-mentioned interpolation is not performed directly using the position coordinates of the actual position of the vehicle, but using the actual position of the vehicle that has been cartographically refined / corrected to determine the measured travel tolerance interval. In this case, the actual position of the vehicle that has been cartographically corrected belongs to an extended position data set that is generated by the subsequent insertion of additional position points between two temporally consecutive actual positions of the vehicle determined from the telemetry data. For this purpose, additional position points are determined between the transmitted actual positions of the vehicle, for example at equidistant intervals, based on a map stored in an external database with position coordinates of traversable paths, each of which provides the respective shortest road connection that can be used by the test vehicle between the observed temporally consecutive actual positions of the vehicle.

[0022] The comparison of the position coordinates of the route set point with the measured travel tolerance interval can be performed in different ways. In the simplest case, those route set points whose position coordinates are located within the measured travel tolerance interval are determined. For the route set points outside the interval, as a further selection criterion for the route set points that are still associated with the measured travel tolerance interval, the maximum allowable distance relative to the measured travel tolerance interval is considered. Here, the vertical distance to the edge of the measured travel tolerance interval can be used as the distance relative to / to the measured travel tolerance interval. Alternatively, the distance is determined as the distance from the corresponding route set point to the nearest route section determined by the aforementioned linear interpolation for the actual position of the vehicle that is continuous in time. Here, for the determination of the nearest route section, in order to achieve a particularly efficient design in terms of runtime efficiency, a search is performed in the vehicle actual position data set using a KD tree structure.

[0023] After determining the route setpoints assigned to the measured travel tolerance interval, a route coverage value is determined, wherein, for a possible embodiment, the number of route setpoints assigned to the measured travel tolerance interval is determined in relation to the total number of route setpoints. For an alternative embodiment, a series of route segments are generated by linear interpolation of the route setpoints, and the cumulative length of the route segments respectively extending between two route setpoints assigned to the measured travel tolerance interval is set in relation to the total length of all route segments.

[0024] As a further method step, it is queried whether the segment coverage value is less than a preset coverage threshold. If so, the segment preset is updated for a second measurement run, which must be carried out by the same test vehicle (i.e., the first test vehicle and / or at least one other test vehicle in the fleet).

[0025] For the second embodiment, the data interpolation for setting the tolerance interval is performed by interpolation calculation of the position coordinates of the segment set points or of the segment set points corrected by cartography, which leads to the determination of the target segment tolerance interval. According to the above-described check of the relative position between the position point and the geographically restricted tolerance interval, the position coordinates of the actual position of the vehicle are then compared with the target segment tolerance interval in a subsequent step, and the segment coverage value is determined similarly to the first embodiment on the basis of this comparison.

[0026] For the third embodiment, two interpolations are performed, and the measured travel tolerance interval and the target section tolerance interval are set. Next, the range of the overlapping area between the measured travel tolerance interval and the target section tolerance interval is detected in order to set the section coverage value, and the range of the overlapping area is set in correlation with the range of the location area containing the measured travel tolerance interval and the target section tolerance interval. Based on this correlation, the section coverage value can be determined.

[0027] A training data set generation system according to the invention comprises a telemetry data acquisition unit and is suitable for performing the method according to the invention as described above. For an advantageous embodiment, the training data set generation system performs an automatic adaptation of the tolerance radius for circular process objects for data interpolation. To this end, a small tolerance radius is first selected on the training data set generation system for performing the method according to the invention using fleet telemetry data and the resulting system runtime is determined. Next, the tolerance radius is gradually increased until a predetermined runtime preset / specification is not reached.

[0028] In a further embodiment, the method according to the invention for generating a training data set is combined with an additional evaluation of the test runs carried out for this purpose. Advantageously, a plausibility check is performed on the resources invested in the test vehicle fleet or in the individual test runs, such as fuel consumption or vehicle maintenance measures to be taken. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] Further advantageous designs of the method for generating a training data set according to the present invention are also obtained from the embodiments, which will be described in more detail below with reference to the accompanying drawings.

[0030] in:

[0031] Figure 1 An embodiment of the method for generating a training data set according to the present invention is shown; and

[0032] Figure 2 Shows Figure 1 A first design of the interpolation step of the illustrated embodiment; and

[0033] Figure 3 Shows Figure 1 A second embodiment of the interpolation step of the exemplary embodiment shown. DETAILED DESCRIPTION

[0034] Figure 3 The block diagram shows a first embodiment of the method for generating a training data set according to the invention. In method step A, a route segment is preset 1, for which the position coordinates of a plurality of route segment set points are wirelessly transmitted to the navigation system of the test vehicle. For this purpose, Figure 1 A section of the route specification 1 is shown on a measuring route 2 with a route branch, wherein three route setpoints 3 . 1 , 3 . 2 , 3 . 3 to be reached are shown.

[0035] For method step B, the test vehicle performs a measurement drive according to route preset 1 and acquires the vehicle surroundings with an extended measurement technology system including optical sensors, radar systems, laser-based measurement systems, and ultrasonic and infrared sensors, and records raw data for the generation of a training data set. In addition, the telemetry data is transmitted in a time-discrete manner, for example at a frequency of 1 / 120 s, to an external telemetry data acquisition unit, wherein position coordinates are extracted from the telemetry data for the actual position of the vehicle reached when the telemetry data was sent and provided to the data processing unit. In this case, Figure 1 The actual position 4 . 1 , . . . , 4 . 4 of the vehicle for a section of the measurement journey is shown.

[0036] In method step C, the measured travel tolerance interval 5 is determined, wherein firstly the consecutive actual vehicle positions 4.1, ..., 4.4 are connected together by means of linear interpolation, thereby determining a series of route segments 6. As an example, a single route segment 7 is marked, which is located between the actual vehicle positions 4.2 and 4.3. For further interpolation, this series of route segments 6 is extended by a circular process object 8 with a preset tolerance radius, which is moved relative to its center along the individual route segments 7, to form Figure 3 The consecutive measurement driving tolerance interval 5 is shown in FIG.

[0037] for Figure 3 In the method step D shown in FIG. 1 , the position coordinates of the route setpoints 3.1, 3.2, 3.3 are compared with the measured driving tolerance interval 5. Here, for a first embodiment, it is determined which of the route setpoints 3.1, 3.2, 3.3 are directly within the measured driving tolerance interval 5. Alternatively, in Figure 1 , which takes into account route setpoints 3.1, 3.2, 3.3 as long as the vertical distance 9 between them and the nearest route segment 7 is less than a predetermined maximum distance. In order to perform the association between the route setpoints 3.1, 3.2, 3.3 and the corresponding nearest route segment 7 with the greatest possible computational efficiency, a search is preferably performed in the vehicle actual position 4.1, ..., 4.4 data set in a KD tree structure.

[0038] The following method step E involves determining the route coverage value, wherein in the simplest case the number of route setpoints 3.1, 3.2, 3.3 assigned to the measuring travel tolerance interval 5 is determined in relation to the total number of route setpoints 3.1, 3.2, 3.3. In an advantageous alternative embodiment, not shown in detail, a series of target route segments is generated by means of a linear interpolation of the route setpoints 3.1, 3.2, 3.3, and the cumulative length of the target route segments extending between two route setpoints 3.1, 3.2, 3.3 assigned to the measuring travel tolerance interval 5 is set in relation to the total length of all target route segments.

[0039] Method step F involves comparing the calculated route coverage value with a preset coverage threshold. If the measured route traveled is not accurate enough, a route segment coverage value is obtained that is less than the preset coverage threshold. Next, the training data record generation system updates the route segment preset 1 and outputs it again to the first test vehicle and / or another test vehicle of the fleet to perform a second measurement drive.

[0040] Additionally, a preferred training data set generation system (not shown in detail) is accordingly designed such that an automatic adaptation of the tolerance radius is carried out for the circular process object 8 according to the runtime specification in order to interpolate the position coordinates of the actual vehicle position 4 . 1 , . . . , 4 .

[0041] Figure 2 An improved design of the method for generating a training data set according to the invention is shown, in which the position coordinates of the actual vehicle position 4.1, ..., 4.4 are not directly interpolated for the determination of the tolerance interval 5 for the measurement drive, but an intermediate step is performed. For this step, the cartographically corrected actual vehicle position 10.1, ..., 10.9 is generated with the aid of a database with route information, in which, in addition to the telemetrically transmitted actual vehicle position 4.1, ..., 4.4, further position points are added along the shortest road connection provided for the measurement drive before the interpolation step.

Claims

1. A method for generating a training data set for an autonomous driving vehicle, comprising executing a route preset (1) in the form of position coordinates of a plurality of route set points (3.1, 3.2, 3.3), performing a measurement drive with a first test vehicle along a measurement path (2) selected according to the route preset (1), and collecting vehicle environment data along the measurement path (2) by means of a sensor system of the first test vehicle; It is characterized in that - during the first measurement driving process, transmitting the telemetry data of the first test vehicle to an external telemetry data acquisition unit in a time-discrete form, and the telemetry data includes the position coordinates of a plurality of actual vehicle positions (4.1, ..., 4.4) reached by the first test vehicle; - determining a measured driving tolerance interval (5) and / or a target road section tolerance interval; - wherein the measured driving tolerance interval (5) is determined by interpolating the position coordinates of the vehicle's actual position (4.1, ..., 4.4) or the vehicle's actual position (10.1, ..., 10.9) after cartographic correction; - Wherein, the target road section tolerance interval is obtained by setting the road section points (3.1, 3.2, 3.3) or by interpolating the set points of the road section after cartographic correction; - by comparing the position coordinates of the route set points (3.1, 3.2, 3.3) with the measured driving tolerance interval (5), and / or by comparing the actual vehicle position (4.1, ..., 4.4) with the target road section tolerance interval, and / or by comparing the measured driving tolerance interval (5) with the target road section tolerance interval to determine a road section coverage value; and If the segment coverage value is less than a preset coverage threshold, generating an updated segment preset for a second measurement drive using the first test vehicle and / or a further test vehicle.

2. The method for generating a training data set according to claim 1, It is characterized in that In order to determine the measured driving tolerance interval (5), linear interpolation is performed on the temporally consecutive vehicle actual positions (4.1, ..., 4.4), and / or in order to determine the target route tolerance interval, linear interpolation is performed on the consecutive route set points (3.1, 3.2, 3.3), which form a series of route sections (6).

3. The method for generating a training data set according to claim 2, It is characterized in that The series of route segments (6) are expanded using a circular process object (8) with a preset tolerance radius to form a continuous measured driving tolerance interval (5) and / or a continuous target route tolerance interval.

4. The method for generating a training data set according to claim 1, It is characterized in that By comparing the position coordinates of the section set points (3.1, 3.2, 3.3) with the measured driving tolerance interval (5), those section set points (3.1, 3.2, 3.3) whose position coordinates are within the measured driving tolerance interval (5) are determined, and / or by comparing the position coordinates of the vehicle actual positions (4.1, ..., 4.4) with the target section tolerance interval, the actual positions (4.1, ..., 4.4) of the vehicles whose position coordinates are within the target section tolerance interval are determined.

5. The method for generating a training data set according to any one of the preceding claims, It is characterized in that By comparing the position coordinates of the segment set points (3.1, 3.2, 3.3) with the measured travel tolerance interval (5), those segment set points (3.1, 3.2, 3.3) that are within the maximum allowable distance relative to the measured travel tolerance interval are determined, and / or by comparing the position coordinates of the vehicle actual position (4.1, ..., 4.4) with the target segment tolerance interval, those vehicle actual positions (4.1, ..., 4.4) that are within the maximum allowable distance relative to the target segment tolerance interval are determined.

6. The method for generating a training data set according to claim 5, It is characterized in that The distance between the corresponding section set point (3.1, 3.2, 3.3) and the measured driving tolerance interval (5) is determined as the vertical distance between the section set point and the edge of the measured driving tolerance interval (5), and / or the distance between the corresponding vehicle actual position (4.1, ..., 4.4) and the target section tolerance interval is determined as the vertical distance between the vehicle actual position and the edge of the target section tolerance interval.

7. The method for generating a training data set according to claim 5, It is characterized in that The distance between the corresponding segment set point (3.1, 3.2, 3.3) and the measured driving tolerance interval is determined as the distance between the segment set point and the nearest segment section (7) obtained by linear interpolation of the temporally consecutive vehicle actual positions (4.1, ..., 4.4), and / or the distance between the corresponding vehicle actual position (4.1, ..., 4.4) and the target segment tolerance interval is determined as the distance between the vehicle actual position and the nearest segment section obtained by linear interpolation of consecutive segment set points (3.1, 3.2, 3.3).

8. The method for generating a training data set according to claim 7, It is characterized in that In order to determine the nearest route segment section (7) obtained by linear interpolation, a search is performed in the data set of vehicle actual positions (4.1, ..., 4.4) and / or route segment setpoints (3.1, 3.2, 3.3) in a KD tree structure.

9. The method for generating a training data set according to claim 1, It is characterized in that In order to determine the segment coverage value, the number of segment setpoints (3.1, 3.2, 3.3) assigned to the measured driving tolerance interval is determined in relation to the total number of segment setpoints (3.1, 3.2, 3.3), or generating a series of target route segments by linear interpolation of the route set points (3.1, 3.2, 3.3), and setting the cumulative length of the target route segments extending between two route set points (3.1, 3.2, 3.3) respectively assigned to the measured driving tolerance interval in relation to the total length of all target route segments, or determining the number of vehicle actual positions (4.1, ..., 4.4) assigned to the target route section tolerance interval in relation to the total number of vehicle actual positions (4.1, ..., 4.4), Alternatively, a series of vehicle actual position segments are generated by linear interpolation of the vehicle actual position (4.1, ..., 4.4), and the cumulative length of the vehicle actual position segments extending between two vehicle actual positions (4.1, ..., 4.4) assigned to the target road section tolerance interval is set in relation to the total length of all vehicle actual position segments.

10. A training data set generation system, It is characterized in that The system comprises a telemetry data acquisition unit and is designed to be able to perform the method according to any one of claims 1 to 9.

11. The training data set generation system according to claim 10, It is characterized in that Automatic adaptation of the tolerance radius of circular process objects for position coordinate interpolation can be performed according to a runtime specification.

Images