Method for generating synthetic sensor data for specific sensor generations

Abstract

The invention relates to a method (100) for generating synthetic sensor data (2) of a specific sensor generation (1), comprising the following steps: providing (101) sensor data (3), the sensor data (3) being generated from a detection of at least one sensor (4) of a first sensor type, compressing (102) the sensor data (3) by using an encoder module (5) to generate a compressed representation (6) of the sensor data (3); -generating (103) synthetic sensor data (2) based on at least one characteristic of a second sensor type (7), at least one characteristic of a specific sensor generation (8) and the compressed representation (6) of the sensor data (3) by using a common decoder module (9) and a decoder module (11) specific to the specific sensor generation (1).

Description

Technical Field

[0001] The present invention relates to a method for generating synthetic sensor data for a specific sensor generation. The present invention also relates to a computer program, a device, and a storage medium for this purpose. Background Art

[0002] Generating synthetic sensor data through machine learning models is an advanced method in artificial intelligence, which enables the generation of real but artificially generated data that simulates real sensor outputs. This technology is particularly useful in scenarios where access to real sensor data is restricted or impossible due to cost, data protection considerations, or practical limitations. Machine learning models, especially those based on neural networks, can be trained to detect the characteristics (Charakteristika) and patterns of real sensor data and then generate similar data. This makes it possible to develop more robust and efficient machine learning systems because these models can be trained using a large amount of data that reflects reality in different scenarios and conditions.

[0003] However, for a specific sensor generation, it may be necessary to retrain the corresponding machine learning model each time according to methods known in the prior art. Summary of the Invention

[0004] The subject matter of the present invention is a method having the features of claim 1, a computer program having the features of claim 10, a device having the features of claim 11, and a computer-readable storage medium having the features of claim 12. Other features and details of the present invention are derived from the corresponding dependent claims, the description, and the drawings. Here, the features and details described in the context of the method according to the present invention of course also apply in the context of the computer program according to the present invention, the device according to the present invention, and the computer-readable storage medium according to the present invention, and vice versa, respectively, such that the disclosure regarding the various aspects of the present invention is always cross-referenced or can be cross-referenced.

[0005] The subject matter of the present invention is in particular a method for generating synthetic sensor data for a specific sensor generation, the method comprising the following steps, where these steps can be repeated and / or executed one after another. A specific sensor generation is in particular a specific model of a sensor, such as a new product line, where this specific model in particular also has corresponding specific physical characteristics or characteristic expressions.

[0006] In a first step, sensor data is preferably provided, where the sensor data is generated from the detection of at least one sensor of a first sensor type. For example, the sensor data can be image data. The at least one sensor can be, for example, a camera sensor or an infrared camera sensor, a radar sensor, a LiDAR (Light Detection and Ranging) sensor, or an ultrasonic sensor, such that the image data can also be designed as infrared, radar, LiDAR, or ultrasonic image data. However, the above list is not exhaustive, such that other sensors besides the ones mentioned are also conceivable.

[0007] In a further step, the sensor data is preferably compressed by using an encoder module to generate a compressed representation of the sensor data. In other words, the sensor data is particularly transformed into a latent space. Here, a feature vector can be extracted based on the sensor data, which represents the essential features of the sensor data. So in other words, the encoder module is particularly designed as a component that transforms the sensor data into a more compact form. This transformation preferably aims to extract the basic features and / or structure of the sensor data and, in so doing, reduce redundant or unimportant information.

[0008] In a further step, synthetic sensor data is preferably generated by using a common decoder module and a decoder module specific to a particular sensor generation of the second sensor type, based on at least one characteristic of the second sensor type, at least one characteristic of a particular sensor generation, and a compressed representation of the sensor data. The particular sensor generation is in particular a particular sensor generation of the second sensor type. For example, if the second sensor type is specific to radar sensor technology, at least one characteristic may describe a basic property of the sensor data from the radar sensor. For example, the particular sensor generation may then be a particular model or product line of the radar sensor. At least one characteristic of the particular sensor generation may be, for example, the number of transmit antennas of the radar sensor or the orientation of the transmit and receive antennas of the radar sensor. Furthermore, a characteristic of the particular sensor generation may be how much data is available for that particular sensor generation. Correspondingly, the machine learning model adaptation discussed in the following sections is also affected, i.e., the less data is available, the less the particular sensor generation is preferably considered in the machine learning model adaptation regarding the common decoder module. The advantage here may be that during the backpropagation (English: “Backpropagation”) for training the entire machine learning model, i.e., in particular the machine learning model including the encoder and decoder parts, the weights and parameters of the decoder module specific to the sensor generation are adapted based on the training data for the particular sensor generation. However, the influence of the backpropagation (English: “Backpropagation”) on the weights and parameters of the common decoder module can be adapted in a ratio proportional to the amount of training data available for the particular sensor generation compared to the data or training data commonly available for all sensor generations. For example, for a new sensor generation that is still in the pre-development stage (Vorausentwicklung), the decoder module specific to the sensor generation can be adapted with limited available data, but preferably only the common decoder module is adapted in a ratio of the amount of training data for the new sensor generation compared to the older sensor generations. Another possibility may also be to estimate the decoder module specific to the sensor generation or sub-generation based on the sensor generation or sub-generation characteristics for a preliminary estimate of the synthetic sensor data for the new particular decoder module. This estimate can be derived based on knowledge of the decoder modules specific to other sensor generations and their corresponding sensor generation-specific characteristics. This method can also be used to initialize the weights and parameters of a new decoder module specific to a sensor generation or sub-generation being designed and thus shorten the training time. At least one additional level (Ebene) can also be set for the sub-generation of the sensor generation. Correspondingly, the synthetic sensor data is also generated based on at least one characteristic of at least one sub-generation and by using a decoder module specific to the sub-generation of the second sensor type.It can be stipulated that at least two decoder modules specific to a particular sensor generation of the second sensor type are provided. Then, a suitable common decoder module can be determined or selected based on at least one characteristic of the second sensor type. In addition, the selector module can determine or select a suitable decoder module specific to a particular sensor generation of the second sensor type based on at least one characteristic of the particular sensor generation of the second sensor type or via an input provided, such as an input from a user, where the input includes a description of the particular sensor generation.

[0009] The first sensor type and the second sensor type can be specific to different sensor technologies or the same sensor technology. Possible sensor technologies include, for example, camera sensors, infrared camera sensors, radar sensors, LiDAR sensors, or ultrasonic sensors. However, the above list is not exhaustive, so that in addition to the mentioned technologies, other analog sensor technologies can also be applied. For example, the first sensor type can be a radar imager The second sensor type can be a radar sensor, for example, in a vehicle.

[0010] According to an advantageous expansion of the invention, it can be stipulated that the common decoder module is a trained machine learning model that is trained for preprocessing a compressed representation for the second sensor type and in particular for another sensor type. The preprocessing preferably includes at least partial reconstruction of the synthetic sensor data to be generated. The decoder module specific to a particular sensor generation of the second sensor type is preferably a further trained machine learning model that is trained to generate synthetic sensor data based on the preprocessed compressed representation.

[0011] Furthermore, within the scope of the present invention, it is conceivable that the encoder module, the common decoder module, and the decoder module specific to a particular sensor generation of the second sensor type are each machine learning models, in particular neural networks, where the encoder module is preferably a foundation machine learning model. The foundation machine learning model (English: “Foundation Model”) is in particular a large-scale, pre-trained machine learning model based on a large amount of heterogeneous data. For example, it serves as an infrastructure based on which more specific and adapted models can be developed for a large number of applications. The foundation machine learning model is characterized in particular by the ability to recognize and generalize complex patterns and relationships in large datasets. The machine learning models mentioned are preferably machine learning models trained separately. The same encoder module can be provided for training different common decoder modules and / or different decoder modules specific to a particular sensor generation of the second sensor type. The weights of this encoder module can be adapted jointly during training. Furthermore, when training different decoder modules specific to a particular sensor generation of the second sensor type, the same common decoder module can be provided. The weights of this common decoder module and the encoder module can be adapted jointly during training.

[0012] Alternatively, it is also conceivable that the encoder module, the common decoder module, and the decoder module specific to a particular sensor generation of the second sensor type are designed and trained as a single machine learning model, in particular a foundation machine learning model.

[0013] Within the scope of the present invention, it can be provided that an encoder module specific to a particular sensor generation of the first sensor type is also provided, and compression is performed by using the encoder module specific to a particular sensor generation of the first sensor type and the encoder module. Thereby, the compression of the sensor data can be performed specifically for a particular sensor generation of the first sensor type and thus more precisely. In other words, according to this alternative, the encoder side is also designed to be multi-stage. Thereby, for example, a particular sensor generation can be taken into account by the corresponding encoder module specific to the particular sensor generation of the first sensor type.

[0014] Further advantages within the scope of the present invention can be achieved if the method further comprises the following steps:

[0015] - Simulating a sensor of the second sensor type based on the generated synthetic sensor data.

[0016] For example, an application can be envisaged in which sensor data in the form of camera image data is simulated and thus radar data for a radar device is provided.

[0017] Furthermore, within the scope of the present invention, it may be advantageous that the method further comprises the following steps:

[0018] - providing other sensor data, wherein the other sensor data is specific to a second sensor type and represents the same scenario as the sensor data, and wherein the other sensor data is generated from the detection of at least one other sensor,

[0019] - comparing the detected other sensor data with the generated synthetic sensor data,

[0020] - detecting an error or impairment of the at least one other sensor based on the result of the comparison.

[0021] The scenario in this case may be specific to the movement in the environment or represent the movement in the environment. For example, within the scope of the foregoing steps, sensor data can be detected with sensors of a first sensor type and other sensor data can be detected with sensors of a second sensor type to detect the same scenario. Advantageously, the synthetic sensor data can be generated so precisely that real sensors, i.e., detection sensors, can be checked for errors or impairments. For example, the impairment here may be weather-related visibility limitations of a camera sensor. Then the synthetic sensor data can be generated based on the sensor data of the sensors of the first sensor type. Alternatively, an error or impairment of at least one sensor of the first sensor type can also be detected based on the comparison in this case.

[0022] Optionally, the method may further comprise the following steps:

[0023] - compensating for the error or impairment of the at least one other sensor based on the synthetic sensor data by modifying the detected other sensor data based on the synthetic sensor data.

[0024] For example, impairments such as weather-related visibility limitations of a camera sensor can be advantageously supplemented here by the generated synthetic sensor data, since sensors of the first sensor type that detect sensor data, such as radar sensors, may not be restricted. If an error or impairment of at least one sensor of the first sensor type is detected based on the comparison, it can be provided in an alternative that the error or the impairment is also compensated for by a corresponding modification. However, in this case, an opposite method may be required to generate the corresponding synthetic sensor data. Here, "opposite (Entgegengesetzt)" particularly means that the corresponding encoder part according to the present invention would then be necessary for the second sensor type and the corresponding decoder part according to the present invention would be necessary for the first sensor type.

[0025] According to a further possibility, it can be provided that the method further comprises the following steps:

[0026] - Generating or checking at least one marking (Kennzeichnung) for training a machine learning model in a training dataset, wherein the training dataset is specific to sensor data of a second sensor type.

[0027] This marking can also be referred to and understood as a label. The training dataset can be a training dataset for a machine learning model trained using these markings or labels. The generated synthetic sensor data can advantageously provide increased variability in the training dataset For example, when simulating known objects with additional details or objects that have not yet appeared in the training dataset by means of synthetic sensor data, the generation of at least one marking can be carried out.

[0028] It is also conceivable that the sensor data and the synthetic sensor data are specific to road traffic, and the method further comprises the following steps:

[0029] - Generating or adapting a road signature based on the generated synthetic sensor data.

[0030] A road signature in particular refers to: a characteristic pattern (Muster) or a series of characteristics that mark a specific road or road network. A road signature can include various elements, for example, which can be physical or abstract in nature. Physical characteristics include, for example, the width and condition of the road, the type of buildings along the road, road markings, curbs, sidewalks, and road lighting. Features such as trees, green belts, or special building elements can also be physical characteristics. In addition, a road signature can include traffic patterns, i.e., for example, the type and intensity of traffic flow, the main means of transportation (e.g., cars, bicycles, pedestrians), and the presence of public transportation. For example, by means of the present invention, it can be advantageously carried out: updating the road signature from one generation of sensors to the next generation of sensors, which can provide additional details or characteristics in the road signature.

[0031] The method according to the invention can be used in a vehicle. The vehicle can, for example, be designed as a motor vehicle and / or a passenger vehicle and / or an autonomous vehicle. The vehicle can have, for example, vehicle means and / or a driver assistance system for providing an autonomous driving function. The vehicle means can be implemented to at least partially automatically control and / or accelerate and / or brake and / or steer the vehicle.

[0032] The subject matter of the present invention is also a computer program, in particular a computer program product, which includes instructions that, when the computer program is executed by a computer, cause the computer to execute the method according to the present invention. Therefore, the computer program according to the present invention brings the same advantages as those described in detail with reference to the method according to the present invention.

[0033] The subject matter of the present invention is also a device for data processing that is configured to execute the method according to the present invention. For example, as the device, a computer that is configured to execute the computer program according to the present invention can be provided. The computer can have at least one processor for executing the computer program. A non-volatile data memory can also be provided, in which the computer program is stored and can be read by the processor from the non-volatile data memory for execution.

[0034] The subject matter of the present invention can also be a computer-readable storage medium that has a computer program according to the present invention and / or includes instructions that, when executed by a computer, cause it to execute the method according to the present invention. The storage medium is designed, for example, as a data memory, such as a hard disk and / or a non-volatile memory and / or a memory card. The storage medium can be integrated into the computer, for example.

[0035] In addition, the method according to the present invention can also be implemented as a computer-implemented method. Description of the Drawings

[0036] Further advantages, features, and details of the present invention result from the following description, in which embodiments of the present invention are described in detail with reference to the drawings. The features mentioned in the claims and the description are necessary for the present invention either individually or in any combination. Among them:

[0037] Figure 1 A schematic visualization of a method, a sensor generation, a sensor, a device, a storage medium, and a computer program according to an embodiment of the present invention is shown.

[0038] Figure 2 A schematic diagram of a method according to an embodiment of the present invention is shown. Detailed Description of the Embodiments

[0039] Figure 1 A method 100, a sensor generation 1, a sensor 4, a device 10, a storage medium 15, and a computer program 20 according to an embodiment of the present invention are shown.

[0040] Figure 1An embodiment of method 100 for generating synthetic sensor data 2 for a specific sensor generation 1 is particularly shown. In a first step 101, sensor data 3 is provided, where the sensor data 3 is generated from the detection of at least one sensor 4 of a first sensor type. In a second step 102, the sensor data 3 is compressed by using an encoder module 5 to generate a compressed representation 6 of the sensor data 3. In a third step 103, synthetic sensor data 2 is generated based on at least one characteristic of a second sensor type 7, at least one characteristic of a specific sensor generation 8, and the compressed representation 6 of the sensor data 3 by using a common decoder module 9 and a decoder module 11 specific to the specific sensor generation 1. The first sensor type and the second sensor type are here specific to different sensor technologies, but can also be specific to the same sensor technology. For example, the first sensor type can be a radar imager, and the second sensor type can be a radar sensor, for example in a vehicle.

[0041] The invention according to the embodiment particularly uses a machine learning model, preferably a generative machine learning model, for generating synthetic sensor data 2, such as reflections for sensor generation 1, based on a scene from sensor data 3, such as an image or a video stream. In particular, a base machine learning model is used for the machine learning model here. It is preferably capable of being used across multiple sensor generations 1.

[0042] For this purpose, an input of sensor data 3 of a sensor technology such as a camera sensor or a LiDAR sensor 4 is preferably provided, and based on this input, an output of synthetic sensor data 2 specific to another sensor technology is generated, such as reflections for a radar sensor or a LiDAR sensor or an ultrasonic sensor.

[0043] A possible architecture of the machine learning model is described below, which can generate synthetic sensor data 2 for different sensor generations 1. In particular, the machine learning model can be improved for all sensor generations 1, even if it is trained only using training data from one of the sensor generations 1. For example, the machine learning model can be designed as a "Plug-and-Play" solution, in which the architecture depends on the input and the required output of a specific sensor generation 1. With the help of an encoder module 5 or a similar block, a compressed, i.e., in particular low-dimensional, representation 6 can be generated based on the input. Depending on the current specific sensor generation 1, if the input remains the same, the subsequent decoder module can be "Plug-and-Play". According to the embodiment, a common decoder module 9 and at least one decoder module 11 specific to a sensor generation 1 or a sensor sub-generation can be set for a specific sensor type.

[0044] Figure 2An embodiment of a method according to such a "plug and play" solution is shown. The sensor data 3 is initially provided to an encoder module 5 here. The encoder module 5 compresses the sensor data 3 into a compressed representation 6 of the sensor data 3. Then a common decoder module 9 preprocesses this compressed representation 6. In this embodiment, the next level is a selector 12, which receives at least one characteristic of a second sensor type 7 as input, and synthetic sensor data 2 is to be generated for the second sensor type. Then, characteristics 7 and at least one characteristic 8 of the second sensor type 7 can be provided for the current specific sensor generation 1, so that in subsequent steps, a decoder module 11 specific to the second sensor type of the specific sensor generation generates corresponding synthetic sensor data 2.

[0045] Due to the modular architecture, according to the embodiment, this encoder module 5 can also be reused for other sensor types with the same input, namely sensor data 3 specific to the same type of sensor 4, such as a camera sensor. For example, the encoder module 5 from a machine learning model for image data input and radar reflection output can thus also be reused for a machine learning model for image data input and output of ultrasonic or LiDAR sensor data 3.

[0046] The advantage of the method according to the embodiment is in particular that even when using training data specific to a particular sensor generation 1, other parts such as the encoder module 5 can be trained and improved, or the entire machine learning model can also be improved, that is, the entire machine learning model is in particular designed as a basic machine learning model. The impact of training the sensor-generation-specific decoder module 11 can be greater here. For example, in the case of backpropagation for fine-tuning weights, the sensor-generation-specific decoder module 11 is fine-tuned to a large extent, while the impact on the common decoder module 9 is smaller. The proportion of the impact is in particular based on sensor-generation-specific characteristics.

[0047] At least one characteristic of a specific sensor generation 8 can also include a plurality of sensor generations 1, which define aspects such as the number of transmitting antennas, the orientation of the transmitting and receiving antennas, or the amount of training data available for this specific sensor generation in terms of the total amount of training data for the overall training model for all generations. Thus, the entire machine learning model can also be improved by training for a specific sensor generation 1.

[0048] Using the present invention, according to an embodiment, it is possible to advantageously perform a performance estimation for a future sensor generation 1. By using the above method, it is particularly possible to create an estimation for a future sensor generation 1, for example, a sensor hardware frontend for a specific scenario, such as an estimation of reflection. In this case, for example, only a few important characteristics of the new sensor generation 1 are known, such as the number of transmit antennas of the new sensor generation 1 and / or the orientation of the antennas. If necessary, the knowledge of the decoder module 11 specific to a previous sensor generation and its corresponding sensor generation-specific characteristics 8, together with the new sensor generation-specific characteristics 8, can also be used to estimate the sensor generation-specific decoder module 11 for the new sensor generation 1. By this method, for example, a reflection for a specific scenario can be generated for the new sensor generation 1 in order to pre-verify the performance of the sensor 4 for these specific scenarios. This can advantageously accelerate the development of the frontend hardware for the new sensor generation through early feedback.

[0049] Furthermore, synthetic sensor data 2 such as reflection can be generated by using sensor data 3 such as an image or a video stream at the input end with the highest resolution in, for example, a radar imager, and then the radar imager can be simulated by a target simulator. Then, depending on its processing capacity, this scenario can be used to simulate the current sensor generation 1, and a specific sensor generation 1, for example, has sensed or not sensed this.

[0050] With the help of an emulator, synthetic sensor data 2 can be generated for any scenario. Then, any sensor generation 1 to be tested can be used with its own specific characteristics, and depending on the hardware configuration of the sensor generation 1 to be tested, the same synthetic sensor data 2 can be used for the sensor hardware of all generations in an in-loop simulation.

[0051] For example, the emulator unit emulates the reflection of an object, and based on the unit triggered by the electromagnetic wave received from the actual sensor 4 to be detected, the corresponding simulation can be sent back from the emulator unit to the actual hardware (HW) to be detected. By this method, a specific extreme case scenario (Eckfallszenarie) can be simulated by using a target simulator to determine the sensor generation 1 that provides the best performance for such a scenario. By early verifying the performance of the sensor, the development of the hardware frontend for the sensor can also be accelerated.

[0052] Performance and self-vehicle localization (Ego-Fahrzeuglokalisierung) can also be improved by performing early fusion in a sensor fusion unit according to the invention described in the embodiments. In particular, a zonal architecture (zonale Architektur) is increasingly being provided in vehicles, in which a vehicle computer is used to process data from multiple sensors in order to fuse the data for environment perception. A method according to an embodiment for generating synthetic sensor data 2 such as reflections can be embedded in a central computer, which generates synthetic sensor data 2 from sensor data 3, i.e., for example, a camera or video stream, and compares it with other sensor data provided by individual sensor generations, for example radar sensors. This can provide early fusion feedback for the sensor data of multiple sensors such as cameras and radars, and can also improve the overall performance of the fusion system. It may also be helpful to be notified in the case of discrepancies (Diskrepanzen) between different sensors, such as cameras / radars, etc.

[0053] In the early development stage of the fusion unit and the A / B sensor sample phases (Sensorprobenphasen), direct feedback on the performance of the sensor system or a specific sensor generation 1 can be given. For example, the fusion unit can obtain sensor data 3 from a radar sensor to be tested. The fusion unit can also execute the method 100 proposed above for generating synthetic sensor data 3, particularly reflections, based on sensor data 3, particularly camera images. The position of an object can be determined based on the generated synthetic sensor data 3, particularly reflections. These determined positions can be compared with the positions actually provided by the radar sensor. If there is a difference between the two, a snapshot (Schnappschuss) can be created using the camera image and the object position provided by the radar sensor, and if possible, a short description can also be used to create a snapshot. The snapshot data can be analyzed later to improve performance.

[0054] The machine learning model proposed above can be used particularly in a central fusion unit and further generate synthetic sensor data 2, such as reflections for a specific sensor generation 1, based on incorrect or corrupted sensor data 3, such as camera input, for which the machine learning model has been trained for the specific sensor generation. The sensors 4 actually installed on the vehicle preferably also process the sensor data 3, particularly the reflections detected by the sensors. Missing or corrupted sensor data 3 can be analyzed based on the synthetic sensor data 2 and used as input for the camera system, which cannot correctly identify the reflections due to lighting conditions, dirty lenses, etc.

[0055] Furthermore, in the case of problems such as those caused by lighting conditions or lens dirt, correction of the perception of the sensor 4, e.g., a camera, can be provided. The method according to an embodiment can also be trained to generate synthetic sensor data 2, e.g., a camera image, where the camera image is based on the sensor data 3 at the input side, e.g., the reflection of the sensor 4 (e.g., radar, LiDAR, etc.). Then, the generated synthetic sensor data 2 (especially the generated camera image) can be compared with the actual sensor data 3 (e.g., a camera image) in order to correct the actual sensor data 3 or the actual camera image.

[0056] Various sensor system models known in the prior art for estimating the performance of sensors for a specific scenario are sometimes unrealistic and especially have low performance, e.g., the RAYTRACING model for radar systems. With the method proposed above according to an embodiment, which can be trained for the desired performance level, the sensor system models required for "X-in-the-loop simulation" will no longer be needed. At least one machine learning model described according to an embodiment can be trained with sufficient training data so that it is very close to reality until these sensor system models are mature enough. These sensor system models can also be improved by the machine learning model proposed above to improve accuracy.

[0057] The method according to an embodiment can also be used, for example, to generate reflections from an image. Then, the reflections are processed on a computer by using sensor algorithms especially in the absence of actual sensor hardware in order to obtain the object and its position relative to the scene at the output side. Then, the object information can be used to label data for the camera image, i.e., to label the data with labels (English "labels") or to verify the labeling for the camera image. In such a way, the label quality of the camera image can be improved. The proposed idea can be used to provide labels for any sensor, e.g., camera / radar / LiDAR, etc.

[0058] With the progress of sensor generation 1, the road signs of earlier sensor generation 1 may become invalid in the future because more objects can be sensed with the newer sensor generation 1 or can be sensed with improved performance or accuracy. For example, radar systems can currently generate locations (Standorte) with fewer reflections of objects, while future sensor generations of radar imagers may be able to generate point clouds of objects and thus identify more objects. With the method according to an embodiment, road signs from an earlier sensor generation 1 can also be used for a newer sensor. For this purpose, for example, if the characteristics of the newer sensor generation are known, reflections can be generated from camera images for the newer sensor generation 1. Then, the characteristics of the newer generation can be used to process the reflections generated by the newer sensor generation 1 to output the identified objects. Static objects from the output, such as guardrails or road signs along the road, can be mapped to existing sensor road sign objects, and the difference (Delta) between the two can be added to the road signs for the newer generation. Thus, when a vehicle equipped with the newer sensor generation 1 drives along this road, the vehicle can, for example, access the generation-specific road signs from the cloud. Thus, the method according to an embodiment particularly provides the possibility of incrementally updating the road sensor signs of sensor generation 1.

[0059] The above description of the embodiments has described the present invention only within the scope of examples. Of course, if technically reasonable, the various features of the embodiments can be freely combined with each other without departing from the scope of the present invention.

Claims

1. A method (100) for generating synthetic sensor data (2) for a specific sensor generation (1), the method comprising the steps of: - Providing (101) sensor data (3), wherein the sensor data (3) is generated from the detection of at least one sensor (4) of a first sensor type, - Compressing (102) the sensor data (3) by using an encoder module (5) to generate a compressed representation (6) of the sensor data (3), - Generating (103) the synthetic sensor data (2) based on at least one characteristic of a second sensor type (7), at least one characteristic of the specific sensor generation (8), and the compressed representation (6) of the sensor data (3) by using a common decoder module (9) and a decoder module (11) specific to the specific sensor generation (1).

2. The method (100) according to claim 1, wherein, the common decoder module (9) is a trained machine learning model, which is trained to preprocess the compressed representation (6) for the second sensor type and especially for another sensor type, wherein the decoder module (11) specific to the specific sensor generation (1) of the second sensor type is a further trained machine learning model, which is trained to generate the synthetic sensor data (2) based on the preprocessed compressed representation (6).

3. The method (100) according to any one of the preceding claims, wherein, the encoder module (5), the common decoder module (9), and the decoder module (11) specific to the specific sensor generation (1) of the second sensor type are respectively machine learning models, especially neural networks, wherein the encoder module (5) is preferably a basic machine learning model.

4. The method (100) according to any one of the preceding claims, wherein, an encoder module specific to the sensor generation of the first sensor type is further provided, and the compression (102) is performed by using the encoder module specific to the sensor generation of the first sensor type and the encoder module (5).

5. The method (100) according to any one of the preceding claims, wherein, the method (100) further comprises the steps of: - Simulating a sensor of the second sensor type based on the generated synthetic sensor data (2).

6. The method (100) according to any one of the preceding claims, wherein, the method (100) further comprises the steps of: - Providing other sensor data (3), wherein the other sensor data (3) is specific to the second sensor type and represents the same scenario as the sensor data (3), and the other sensor data (3) is generated from the detection of at least one other sensor (4), - Comparing the detected other sensor data (3) with the generated synthetic sensor data (2). - Detect an error or impairment of the at least one other sensor (4) based on the result of the comparison.

7. The method (100) according to claim 6, wherein, the method (100) further comprises the steps of: - Compensate for an error or impairment of the at least one other sensor (4) based on the synthetic sensor data (2) by modifying the detected other sensor data (3) based on the synthetic sensor data (2).

8. The method (100) according to any one of the preceding claims, wherein, the method (100) further comprises the steps of: Generate or check at least one label for a training dataset for training a machine learning model, wherein the training dataset is specific to sensor data (3) of the second sensor type.

9. The method (100) according to any one of the preceding claims, wherein, the sensor data (3) and the synthetic sensor data (2) are specific to road traffic, and the method (100) further comprises the steps of: Generate or adapt road signs (2) based on the generated synthetic sensor data (2).

10. A computer program (20) comprising instructions which, when executed by a computer (10), cause the computer to perform the method (100) according to any one of the preceding claims.

11. A device (10) for data processing, the device being arranged to perform the method (100) according to any one of claims 1 to 9.

12. A computer-readable storage medium (15) comprising instructions which, when executed by a computer (10), cause the computer (10) to perform the steps of the method (100) according to any one of claims 1 to 9.

Images