Mapping for autonomous driving

Abstract

The invention relates to a method for training a generator of a generating adversary network. First, initial sensor data from at least one sensor type, acquired in a first spatial region, are provided. Subsequently, map data is generated from the initial sensor data using the generator. The generated map data is then compared with previously available map data using a discriminator of the generating adversary network to determine whether the generated map data is distinguishable from the previously available map data. If the generated map data is distinguishable from the previously available map data, at least one parameter for generating the generated map data is adjusted. The method is then repeated, in particular the steps of generating the generated map data and comparing the generated map data with previously available map data.The process is completed when the generated map data is no longer distinguishable from previously available map data.

Description

[0001] The invention relates to a method for creating a map. Parts of the method relate to training a generator of a generating opponent network so that the generator can create the map. Furthermore, the invention relates to a control method for a vehicle. Other aspects of the invention relate to computing units and control units for carrying out the methods.

[0002] Prior art includes methods in which vehicles use pre-generated maps, at least as support for the automated execution of a driving function. However, creating these maps is complex and expensive. Prior art also includes generating adversarial networks in which two networks, a generator and a discriminator, are trained against each other. The generator produces data that is very similar to an original dataset, while the discriminator attempts to distinguish between genuine and spurious data. After training, the generator can be used to generate data that is very similar to the original data. For example, a map can be generated from satellite images in this way. Disclosure of the invention

[0003] One object of the invention is to provide a method for training a generator of a generating adversary network. Another object of the invention is to provide a method for controlling a vehicle. Further objects of the invention relate to computing units or control units for carrying out the methods. These objects are solved by the subject matter of the independent claims. Advantageous embodiments are specified in the dependent claims.

[0004] In a first aspect, the invention relates to a method for training a generator of a generative adversary network. A generative adversary network can, for example, be referred to as a GAN (generative adversary network). This method comprises the steps described below.

[0005] Initially, initial sensor data from at least one sensor type, acquired within a defined spatial area, is provided. This initial sensor type could be, in particular, a camera sensor, a radar sensor, or a lidar sensor. The initial sensor data can be acquired using one or more sensors of the initial sensor type. For example, several vehicles, each equipped with a sensor of the initial sensor type, could have been traveling within the initial spatial area and acquired the initial sensor data, which is then provided. The initial spatial area can refer to a spatially limited area for which good map data is available, for example, because this data has already been created manually.In particular, it can be provided that the initial sensor data is georeferenced, meaning that, in addition to the sensor's measurement data, position data and / or orientation data determined via GPS or another satellite navigation system and / or an inertial navigation system are included in the initial sensor data. The vehicles involved can be specially equipped with sensors. However, it is also possible to use vehicles that have already been delivered and are operated by users to generate the sensor data.

[0006] Subsequently, map data is generated from the initial sensor data using the generator. This generated map data can include, for example, georeferenced objects and / or predicted sensor data from another sensor type.

[0007] The generated map data is then compared with previously available map data using a discriminator from the generating opponent's network to determine whether the generated map data is distinguishable from previously available map data. In particular, this allows us to determine whether the generator of the generating opponent's network has been sufficiently trained.

[0008] If the generated map data is distinguishable from previously available map data, at least one parameter used to generate the map data is adjusted. Optionally, it may be possible to adjust several such parameters. The process is then repeated, in particular the steps of generating the map data and comparing it with previously available map data.

[0009] The process is completed when the generated map data is indistinguishable from previously available map data. At that point, the generator is sufficiently trained to interpret sensor data from a different spatial area than the first one.

[0010] The generator and the discriminator can, in particular, be artificial neural networks.

[0011] In one embodiment of the method, secondary sensor data from at least one second sensor type are provided. The generated map data is created from the first and second sensor data. In particular, the first and second sensor types can be different. For example, the first sensor type can be a camera sensor and the second sensor type a lidar sensor. Using different sensor types can improve the generation of the map data.

[0012] In one embodiment of the method, the generated map data includes object data. This object data can then be output and used, for example, as a map.

[0013] In one embodiment of the method, the generated map data includes predicted sensor data from another sensor type. The predicted sensor data can then be output and used, for example, for vehicles that do not have the corresponding sensor.

[0014] In one embodiment of the method, after completion of the process, once initial sensor data and further sensor data are available in a second spatial area, the generation of the generated map data, the comparison of the generated map data with the previously available map data, and, if necessary, the adjustment of the parameter are performed again. This makes it possible, in particular, to use further sensor data from the second spatial area to improve the generator even after the initial training, as soon as sufficient measured sensor data from the second spatial area is available. This enables further improvement of the map generation. In particular, these process steps allow the generator to be further improved during its ongoing active use.

[0015] In one embodiment of the method, the generated map data and / or parameters are output to a vehicle. If, as described above, multiple parameters are provided, several parameters can also be output to the vehicle. Using the parameter(s), the vehicle can then, for example, parameterize its own generator, which can be designed analogously to the generator of the generating adversary network, and thus further process the sensor data acquired by the vehicle using the trained generator.

[0016] A second aspect of the invention relates to a computing unit with an input interface and a processor. The computing unit may optionally also have an output interface. The computing unit is configured to perform the described method. In particular, sensor data can be received via the input interface, which can be configured, for example, as an internet interface, network interface, or radio interface. The generated map data and / or the parameter(s) can be output to a vehicle via the output interface.

[0017] According to a third aspect, the invention relates to a control method for a vehicle comprising the steps described below. First, one or more parameters are received for generating map data and / or generated map data. Subsequently, at least one driving function of the vehicle is controlled based on the parameter(s) and / or the map data. Controlling the driving function can, in particular, include steering, influencing acceleration, and / or braking.

[0018] In one embodiment of the control method, the parameter(s) are used to operate a generator in the vehicle to determine predicted sensor data from another sensor type based on sensor data. This predicted sensor data can then also be used to control the driving function.

[0019] According to a fourth aspect, the invention relates to a control unit for a vehicle which is equipped to carry out one of the control procedures.

[0020] Exemplary embodiments of the invention are explained with reference to the following drawings. The schematic drawing shows: Fig. 1. A flowchart of a procedure for training a generator of a generating adversary network; Fig. 2 a computing unit and several vehicles; and Fig. 3. A flowchart of a tax procedure for a vehicle.

[0021] Fig. Figure 1 shows a flowchart 100 of a procedure for training a generator of a generative adversary network. A generative adversary network can be referred to, for example, as a GAN (generative adversary network). In a first procedure step 110, initial sensor data from at least one first sensor type, acquired in a first spatial region, are provided. The first sensor type can be, in particular, a camera sensor, a radar sensor, or a lidar sensor. The initial sensor data can be acquired by one sensor of the first sensor type or by several sensors. For example, several vehicles, each equipped with a sensor of the first sensor type, could have been traveling in the first spatial region and acquired initial sensor data, which is then provided.The first spatial area can refer to a spatially limited region for which good map data is available, for example, because this data has already been created manually. In particular, it can be stipulated that the initial sensor data is georeferenced, meaning that, in addition to the sensor's measurement data, position and / or orientation data determined via GPS or another satellite navigation system and / or an inertial navigation system are included in the initial sensor data. The vehicles used can be specially equipped with sensors. However, it is also possible to use vehicles that have already been delivered and are operated by users to generate the sensor data.

[0022] In a second process step 120, map data is generated from the initial sensor data using the generator. The generated map data can include, for example, georeferenced objects and / or predicted sensor data from another sensor type.

[0023] In a third process step 130, the generated map data is compared with previously available map data using a discriminator of the generating opponent network, and it is determined whether the generated map data is distinguishable from previously available map data. In particular, this allows it to be determined whether the generator of the generating opponent network has already been trained sufficiently well.

[0024] If the generated map data is distinguishable from previously available map data, at least one parameter for generating the map data is adjusted in a parameter adjustment step 140. Optionally, it may be provided that several such parameters are adjusted. The procedure is then carried out again, in particular the steps of generating the map data and comparing the generated map data with previously available map data, i.e., the second procedure step 120 and the third procedure step 130.

[0025] The process is completed with a final step 150 when the generated map data is no longer distinguishable from previously available map data. At this point, the generator is sufficiently trained to interpret sensor data from a different spatial area than the first one.

[0026] The generator and the discriminator can, in particular, be artificial neural networks.

[0027] In one embodiment of the method, in the first process step, 110 second sensor data points from at least one second sensor type are provided. The generated map data is then created in the second process step from the first and second sensor data points. In particular, the first and second sensor types can be different. For example, the first sensor type can be a camera sensor and the second sensor type a lidar sensor. Using different sensor types can improve the generation of the map data.

[0028] In one embodiment of the method, the map data generated in the second process step 120 includes object data. This object data can then be output and used, for example, as a map.

[0029] In one embodiment of the method, the map data generated in the second process step 120 includes predicted sensor data from another sensor type. The predicted sensor data can then be output and used, for example, for vehicles that do not have the corresponding sensor.

[0030] In Fig. Figure 1 further illustrates optional process steps of another embodiment of the method, which are explained below. After the final step 150 of the method, once initial sensor data and further sensor data recorded in a second spatial area are available and have been provided via a further first process step 111, the generation of the generated map data is carried out again in a fourth process step 160. In a fifth process step 170, the comparison of the generated map data with the previously available map data is carried out again.

[0031] If the map data generated in the fourth process step 160 is distinguishable from previously available map data, at least one parameter for generating the map data is adjusted in a further parameter adjustment step 141. Optionally, several such parameters may be adjusted. The process is then repeated, in particular the fourth process step 160 (generating the map data) and the fifth process step 170 (comparing the generated map data with previously available map data).

[0032] The process is completed with a further final step 151 when the generated map data is no longer distinguishable from previously available map data. This allows, in particular, the use of additional sensor data from the second spatial area to improve the generator, even after its initial training, as soon as sufficient sensor data from the second spatial area is available. This enables further improvement of the map generation. Specifically, these process steps 160, 170, and 141 allow the generator to be further improved during its ongoing active use.

[0033] In Fig. Figure 1 also shows an optional output step 180, which in one embodiment of the method can be performed after the final step 150 and / or the further final step 151. In output step 180, the generated map data and / or the parameters are output to a vehicle. If, as described above, several parameters are provided, several parameters can also be output to the vehicle. Using the parameter(s), the vehicle can then, for example, parameterize its own generator, which can be designed analogously to the generator of the generating adversary network, and thus further process the sensor data acquired by the vehicle using the trained generator.

[0034] Fig. Figure 2 shows several vehicles 210, 220, 230, 250 and a computing unit 300. A first vehicle 210 has a first sensor 241, a second sensor 242, and a third sensor 243. A second vehicle 220 has a first sensor 241 and a second sensor 243. A third vehicle 230 has a first sensor 241, a second sensor 242, and a third sensor 243. Furthermore, the first vehicle 210, the second vehicle 220, and the third vehicle 230 have a communication interface 244.

[0035] The first sensor 241 can be, in particular, a camera sensor. The second sensor 242 can be, in particular, a radar sensor. The third sensor 243 can be, in particular, a lidar sensor. However, other assignments of these sensors are also possible. Via the communication interface 244, the vehicles 210, 220, 230 can transmit sensor data, in particular from the first sensor 241, but also from the other sensors 242, 243, to the processing unit 300. The processing unit 300 has an input interface 320 for this purpose. As in Fig. As shown in Figure 2, several vehicles 210, 220, 230, each equipped with a sensor 241 of the first sensor type, could have been traveling in the first spatial area and thereby acquired initial sensor data, which is then provided to the processing unit 300. The first spatial area can refer to a spatially limited area for which good map data is available, for example, because this data has already been created manually. In particular, it can be provided that the initial sensor data is georeferenced, meaning that, in addition to the sensor's measurement data, position data and / or orientation data determined by GPS or another satellite navigation system and / or an inertial navigation system are included in the initial sensor data. The vehicles 210, 220, 230 can be specially equipped with sensors.However, it is also possible to use vehicles 210, 220, 230 that have already been delivered and are operated by users to generate the sensor data.

[0036] The computing unit 300 has a processor 310 in addition to the input interface 320. The computing unit 300 can also optionally have an output interface 330. The computing unit 330 is configured to handle the following in connection with Fig. The procedure described in section 1 is to be carried out. In particular, the sensor data can be received via the input interface 320, which can be configured, for example, as an internet interface, network interface, or radio interface. This can specifically affect the first process step 110 and the subsequent first process step 111. The generated map data and / or the parameter(s) can be output via the output interface 330 to another vehicle 250 via a communication interface 244 of the other vehicle 250, thus enabling, for example, the execution of output step 180. Furthermore, it is also possible to output the generated map data and / or the parameter(s) to vehicles 210, 220, and 230.

[0037] Fig. Figure 3 shows a flowchart 400 of a control procedure for a vehicle with the steps described below. First, in a receive step 410, one or more parameters for generating map data and / or generated map data are received. Subsequently, in a control step 420, at least one driving function of the vehicle is controlled based on the parameter(s) and / or the map data. Controlling the driving function can include, in particular, a steering movement, influencing acceleration, and / or braking.

[0038] In Fig. Figure 3 further shows an optional generator step 430 of an embodiment of the control method, which is performed between the receive step 410 and the control step 420. In generator step 430, the parameter(s) are used to operate a generator in the vehicle to determine predicted sensor data of another sensor type from sensor data. For example, the other vehicle 250 can use the sensor data of the first sensor 241 to calculate sensor data of a third sensor by means of generator step 430, even though no third sensor is installed in the other vehicle 250. These predicted sensor data can then also be used to control the driving function and thus be incorporated into the control step 420.

[0039] This in connection with Fig. The tax procedure described in section 3 can also be carried out by vehicles 210, 220, 230.

[0040] In Fig. 2 is further shown that the vehicles 210, 220, 230, 250 each have a control unit 260 for a vehicle which is configured to use one of the control procedures of the Fig. 3 to be carried out.

[0041] Although the invention has been described in detail by means of the preferred embodiments, the invention is not limited to the disclosed examples and other variations can be derived from them by a person skilled in the art without leaving the scope of protection of the invention.

Claims

[1] Method for training a generator of a generating adversary network, comprising the following steps: - Providing (110) initial sensor data from at least one initial sensor type, recorded in an initial spatial area; - Generating (120) generated map data from the initial sensor data using the generator; - Compare (130) the generated map data with pre-existing map data using a discriminator of the generating adversary network and determine whether the generated map data are distinguishable from pre-existing map data; - Adjusting (140) a parameter for generating the generated map data if the generated map data is distinguishable from previously available map data, and repeating the generation and comparison process; - Termination (150) of the procedure when the generated map data are no longer distinguishable from previously available map data. [2] Method according to claim 1, wherein second sensor data of at least one second sensor type are provided and the generated map data are produced from the first sensor data and the second sensor data. [3] Method according to claim 1 or 2, wherein the generated map data includes object data. [4] Method according to any one of claims 1 to 3, wherein the generated map data includes predicted sensor data of another sensor type. [5] Method according to claim 4, wherein after completion (150) of the method, once first sensor data and further sensor data recorded in a second spatial area are available, the generation (160) of the generated map data, the comparison (170) of the generated map data with the previously available map data and, if necessary, the adjustment (141) of the parameter are carried out again. [6] Method according to any one of claims 1 to 5, wherein the generated map data and / or the parameter are output to a vehicle (210, 220, 230, 250). [7] Computing unit (300) comprising an input interface (320) and a processor (310), wherein the computing unit (300) is configured to perform the method according to any one of claims 1 to 6. [8] Tax procedure for a vehicle (210, 220, 230, 250) with the steps: - Receiving (410) a parameter for generating generated map data and / or receiving generated map data; - Control (420) of at least one driving function of the vehicle based on the parameter and / or on the map data. [9] Control method according to claim 8, wherein the parameter is used to operate a generator in the vehicle to determine predicted sensor data of another sensor type from sensor data (430). [10] Control unit (260) for a vehicle (210, 220, 230, 250) which is configured to perform one of the control methods of claims 8 or 9.

Images