Method for testing robustness of machine learning algorithm for classifying objects in environment of motor vehicle

Abstract

The invention relates to a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle, in which the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, and wherein the method (1) comprises the steps of: providing image data showing an environment of the motor vehicle, wherein the image data contains a representation (2) of at least one object; modifying the representation of the at least one object such that the representation appears distorted (3); and simulating the control of the motor vehicle based on objects classified in the modified representation of the at least one object by the machine learning algorithm in order to test the robustness of the machine learning algorithm (4).

Description

Technical Field

[0001] The present invention relates to a method for testing the robustness of machine learning algorithms, by means of which the influence of external influences or interventions on the control of a motor vehicle can be examined, and the safety when controlling a motor vehicle based on machine learning algorithms can be improved. Background Art

[0002] Machine learning algorithms are based on the use of statistical methods in order to train a data processing device such that the data processing device can perform a specific task, while the data processing device was not explicitly programmed for this purpose initially. The goal of machine learning here is to construct algorithms that can learn from data and make predictions. These algorithms create mathematical models, using which, for example, data can be classified.

[0003] Robustness is furthermore understood as the ability of a machine learning algorithm to withstand changes, such as external attacks. Regarding external attacks, a distinction is usually made between white-box attacks and black-box scenarios. In the case of white-box attacks, the attacker has knowledge about the structure of the machine learning algorithm, about the type of training method, and about the current data with which the machine learning algorithm was trained. In the case of black-box scenarios, the attacker does not have this knowledge, but only sees the input data and the result that the network outputs for this.

[0004] Such machine learning algorithms are applied when controlling driver assistance systems of motor vehicles or when operating autonomous motor vehicles, where the machine learning algorithm can be configured to classify objects represented in the environmental data of the motor vehicle, such as road signs or street lamps, and where the motor vehicle or the driver assistance system or the autonomous motor vehicle is controlled based on the classified objects. However, especially when controlling a motor vehicle, high requirements are placed on safety and thus also on the robustness of such machine learning algorithms in order to avoid safety-critical situations as much as possible.

[0005] A method for training a neural network is known from publication DE 10 2019 209 560 A1, which method includes providing a training data set having training images that show the vehicle environment from the perspective of the vehicle, where a plurality of training images show traffic signs, generating additional training images by enhancing the training images showing traffic signs, enhancing the training images showing traffic signs by partially covering the traffic signs and / or by changing the light emission state of one or more light-emitting elements of a variable message sign, and training the neural network based on at least the enhanced training images.

[0006] Here, the task on which the present invention is based is to reliably and without great expense test the robustness of machine learning algorithms for classifying objects in the environment of a motor vehicle.

[0007] This task is solved by a method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle, according to the features of claim 1.

[0008] This task is furthermore solved by a system for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle, according to the features of claim 8. Summary of the Invention

[0009] According to one embodiment of the invention, this task is solved by a method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing the environment of the motor vehicle, and wherein the method comprises: providing image data showing the environment of the motor vehicle, wherein the image data contains a representation of at least one object; modifying the representation of the at least one object such that the representation appears distorted; and simulating the control of the motor vehicle based on the objects classified by the machine learning algorithm in the modified representation of the at least one object, in order to test the robustness of the machine learning algorithm.

[0010] Image data is hereby understood as data that can be reproduced as an image or a graphic with the aid of a specific program. Representing an object in the image data furthermore means that the corresponding image data shows the object or has a representation of the object.

[0011] Manipulating or modifying the representation of an object such that the representation appears distorted furthermore means that the representation of the object is changed in a realistic but at the same time unobtrusive way, for example in order to simulate an external attack.

[0012] Thus, a method is described by means of which safety-related or safety-critical aspects can be identified or revealed early on and in particular already during the development of the machine learning algorithm. Based on the corresponding test results, it can furthermore be checked whether the algorithm is robust enough with respect to external influences, such as external attacks, and is thus suitable for controlling a motor vehicle.

[0013] It is possible to react early to possible safety-related aspects, where the method works regardless of the structure of the machine learning algorithm, the type of training method, and knowledge of the current data used for training the machine learning algorithm. Additionally, it has the following advantages: The time until the machine learning algorithm can actually be used for classifying objects in the environment of a motor vehicle can be reduced, and at the same time, resources such as storage capacity required for (re)training or optimizing or making the machine learning algorithm more robust can be saved.

[0014] Overall, a method is thus described by which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be reliably and with little effort tested.

[0015] Here, the step of modifying the representation of the at least one object may include: modifying the representation of the at least one object such that the object is misclassified by the machine learning algorithm; and / or modifying the representation of the at least one object based on how a third party can change the object; and / or modifying the representation of the at least one object based on different lighting conditions and / or weather conditions.

[0016] Modifying the representation of the at least one object such that the object is misclassified by the machine learning algorithm here means that the machine learning algorithm assigns the object to a different class than the class to which the object actually belongs based on the modified representation, or even does not assign it to any class, even if the classification result is distorted.

[0017] Modifying the representation of the at least one object based on how a third party can change the object additionally means that realistic or expected changes in conditions and / or known external attacks can be simulated.

[0018] Modifying the representation of the at least one object based on different lighting conditions additionally means adapting the representation of the object to possible other lighting conditions. For example, a road sign can be perceived differently depending on sunlight irradiation. Modifying the representation of the at least one object based on different weather conditions additionally means adapting the representation of the object to possible weather conditions such as rain and / or fog.

[0019] Thus, the representation of the at least one object can be adapted to all known, realistic, and expected condition changes in order to simulate the behavior of the machine learning algorithm and thus also of the motor vehicle in the presence of these known, realistic, and expected changes.

[0020] In one embodiment, the step of modifying the representation of the at least one object includes applying an image processing algorithm.

[0021] An image processing algorithm is understood here to be an algorithm that is designed to change or modify image data. For example, an object can be rotated and / or scaled differently, or other objects can be placed on the actual object.

[0022] Thus, the representation of the at least one object may be modified by a known and common algorithm, without requiring a costly and resource-intensive adaptation.

[0023] In another embodiment, the step of modifying the representation of at least one object comprises applying a machine learning algorithm trained to simulate external intervention.

[0024] The fact that the machine learning algorithm is trained to simulate external attacks means that the machine learning algorithm is an adversarial generator or is designed to manipulate image data for the machine learning algorithm.

[0025] The representation can thereby be automatically adapted as precisely as possible to known manipulation or change patterns.

[0026] Furthermore, the image data may be sensor data recorded by surroundings sensors of the motor vehicle.

[0027] A sensor, also referred to as a detector, (measurement variable or measurement) sensor element or (measurement) probe, is understood here to be a technical component that can detect specific physical or chemical properties and / or material properties of its environment qualitatively or quantitatively as a measurement variable.

[0028] Furthermore, an environmental sensor is understood to be a sensor of the motor vehicle which is designed to detect data about the environment of the motor vehicle or at least a part of the environment.

[0029] The method for testing the robustness of a machine learning algorithm may thus be based on facts outside the actual data processing device on which the machine learning algorithm is tested.

[0030] Another embodiment of the present invention also describes a method for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing the environment of the motor vehicle, wherein the method includes testing the robustness of the trained algorithm of the machine learning algorithm by the method described above for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle to provide a test result; and optimizing the machine learning algorithm based on the provided test result.

[0031] Optimizing the machine learning algorithm in this context in particular means retraining the machine learning algorithm based on the test results in order to avoid safety-critical situations based on objects classified by the machine learning algorithm as far as possible, especially when controlling a motor vehicle.

[0032] Accordingly, a method for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle is described, the method being based on methods that can be used to reliably and without great expense test the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle. In particular, the method is based on methods that can identify or reveal safety-related or safety-critical aspects early on and especially during the development of the machine learning algorithm. Based on the corresponding test results, it can also be checked whether the algorithm is robust enough with respect to external influences, such as external attacks, and is thus suitable for controlling a motor vehicle. Possible safety-related aspects can be reacted to early on, and the method works regardless of knowledge of the structure of the machine learning algorithm, the type of training method, and the current data used for training the machine learning algorithm. In addition, it has the advantage that the time until the machine learning algorithm can actually be used for classifying objects in the environment of a motor vehicle can be reduced, and at the same time resources such as storage capacity required for (re)training or optimizing or making the machine learning algorithm more robust can be saved.

[0033] A further embodiment of the invention furthermore describes a method for controlling a motor vehicle based on a machine learning algorithm, wherein the machine learning algorithm is trained to classify objects in image data representing the environment of the motor vehicle, and wherein the method comprises: providing a machine learning algorithm for classifying objects in the environment of the motor vehicle, wherein the machine learning algorithm has been optimized by the method for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle described above; and controlling the motor vehicle based on the provided machine learning algorithm.

[0034] Therefore, a method for controlling a motor vehicle based on a machine learning algorithm is described, which is based on a method that can be used to reliably and without great expense test the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle. In particular, the method is based on a method that can identify or reveal safety-related or safety-critical aspects early, and in particular already during the development of the machine learning algorithm. Based on the corresponding test results, it can also be checked whether the algorithm is robust enough against external influences, such as external attacks, and thus suitable for controlling a motor vehicle. Possible safety-related aspects can be reacted to early, and the method works even without knowledge of the structure of the machine learning algorithm, the type of training method, and the current data used for training the machine learning algorithm. In addition, it has the advantage that the time until the machine learning algorithm can actually be used for classifying objects in the environment of a motor vehicle can be reduced, and at the same time resources such as storage capacity required for (re)training, optimizing, or making the machine learning algorithm more robust can be saved.

[0035] Another embodiment of the invention further describes a system for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing the environment of the motor vehicle, and wherein the system is configured to perform the method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle described above.

[0036] Therefore, a system is described by which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be reliably and without great expense tested. In particular, a system is described that is configured to be able to identify or reveal safety-related or safety-critical aspects early, and in particular already during the development of the machine learning algorithm. Based on the corresponding test results, it can also be checked whether the algorithm is robust enough against external influences, such as external attacks, and thus suitable for controlling a motor vehicle. Possible safety-related aspects can be reacted to early, and the method works even without knowledge of the structure of the machine learning algorithm, the type of training method, and the current data used for training the machine learning algorithm. In addition, it has the advantage that the time until the machine learning algorithm can actually be used for classifying objects in the environment of a motor vehicle can be reduced, and at the same time resources such as storage capacity required for (re)training, optimizing, or making the machine learning algorithm more robust can be saved.

[0037] Another embodiment of the present invention further describes a system for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, and wherein the system includes the system described above for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle, the system being configured to test the robustness of the trained algorithm of the machine learning algorithm to provide a test result; and an optimization unit configured to optimize the machine learning algorithm based on the provided test result.

[0038] Accordingly, a system for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle is described, the system being based on a system that can be used to reliably and without great expense test the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle. In particular, the system is based on a system that is configured to identify or reveal safety-related or safety-critical aspects early and especially during the development of the machine learning algorithm. Based on the corresponding test results, it can also be checked whether the algorithm is robust enough with respect to external influences, such as external attacks, and thus suitable for controlling a motor vehicle. Possible safety-related aspects can be reacted to early, and the method works regardless of knowledge of the structure of the machine learning algorithm, the type of training method, and the current data used for training the machine learning algorithm. In addition, it has the advantage that the time until the machine learning algorithm can actually be used for classifying objects in the environment of a motor vehicle can be reduced, and at the same time, resources such as storage capacity required for (re)training or optimizing or making the machine learning algorithm more robust can be saved.

[0039] Another embodiment of the present invention further describes a system for controlling a motor vehicle based on a machine learning algorithm, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, and wherein the system includes: a providing unit configured to provide a machine learning algorithm for classifying objects in the environment of the motor vehicle, wherein the machine learning algorithm has been optimized by the system for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle described above; and a control unit configured to control the motor vehicle based on the provided machine learning algorithm.

[0040] Therefore, a system for controlling a motor vehicle based on a machine learning algorithm is described, which is based on a system that can reliably and without great expense test the robustness of a machine learning algorithm used for classifying objects in the environment of a motor vehicle. In particular, the system is based on a system that is configured to identify or reveal safety-related or safety-critical aspects early and especially already during the development of the machine learning algorithm. Based on the corresponding test results, it can furthermore be checked whether the algorithm is robust enough against external influences, such as external attacks, and thus suitable for classifying objects in the environment of a motor vehicle. Possible safety-related aspects can be reacted to early, where the method works regardless of knowledge of the structure of the machine learning algorithm, the type of training method, and the current data used for training the machine learning algorithm. Furthermore, it has the advantage that the time until the machine learning algorithm can actually be used for classifying objects in the environment of a motor vehicle can be reduced, while at the same time resources such as storage capacity required for (re)training or optimizing or making the machine learning algorithm more robust can also be saved.

[0041] Another embodiment of the invention furthermore describes a computer program having program code for performing the method described above for testing the robustness of a machine learning algorithm used for classifying objects in the environment of a motor vehicle when the computer program is executed on a computer.

[0042] Another embodiment of the invention furthermore describes a computer-readable data carrier having the program code of a computer program for performing the method described above for testing the robustness of a machine learning algorithm used for classifying objects in the environment of a motor vehicle when the computer program is executed on a computer.

[0043] The computer program and the computer-readable data carrier have the following advantages respectively herein: The computer program and the computer-readable data carrier are respectively configured to execute a method that can be used to test the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle reliably and without great expense. In particular, the computer program and the computer-readable data carrier are respectively configured to execute a method that is configured to identify or reveal safety-related or safety-critical aspects early, and in particular during the development of the machine learning algorithm. Based on the corresponding test results, it can also be verified whether the algorithm is robust enough against external influences, such as external attacks, and thus suitable for controlling a motor vehicle. Possible safety-related aspects can be reacted to early, and the method works regardless of the structure of the machine learning algorithm, the type of training method, and the knowledge of the current data used for training the machine learning algorithm. In addition, it has the following advantages: The time until the machine learning algorithm can actually be used for classifying objects in the environment of a motor vehicle can be reduced, and at the same time, resources such as storage capacity required for (re)training, optimizing, or making the machine learning algorithm more robust can also be saved.

[0044] Generally speaking, it can be determined that a method for testing the robustness of a machine learning algorithm is described by the present invention. With this method, the influence of external influences or interventions on the control of a motor vehicle can be verified, and the safety when controlling a motor vehicle based on a machine learning algorithm can be improved.

[0045] The described design solutions and improvements can be combined with each other arbitrarily.

[0046] Other possible design solutions, improvements, and implementations of the present invention also include combinations of features of the present invention not explicitly mentioned previously or below in connection with the embodiments. Description of the Drawings

[0047] The accompanying drawings are intended to facilitate a further understanding of the embodiments of the present invention. The drawings illustrate the embodiments and explain the principles and concepts of the present invention in conjunction with the description.

[0048] Considering the drawings, many of the other embodiments and the mentioned advantages can be obtained. The elements shown in the drawings are not necessarily shown in the correct proportions to each other.

[0049] Figure 1 A flowchart showing a method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle according to an embodiment of the present invention; and

[0050] Figure 2Schematic block diagram showing a system for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle according to an embodiment of the present invention. Detailed implementation

[0051] In the figures, unless otherwise stated, the same reference numerals denote the same or functionally identical elements, components or assemblies.

[0052] Figure 1 Flowchart showing a method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle 1 according to an embodiment of the present invention.

[0053] In particular Figure 1 A method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle 1 is shown, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle.

[0054] In order to control a motor vehicle or a function of a motor vehicle or an autonomous driving motor vehicle by means of a machine learning algorithm, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, environmental data is usually detected by one or more environmental sensors of the motor vehicle. The environmental sensors can here be, for example, cameras, radar sensors and / or lidar sensors. Subsequently, the data of the individual sensors can be correlated with each other, and a machine learning algorithm trained accordingly on the basis of training data marked or provided with corresponding information can classify the objects in the correlated data. Subsequently, the classified data can be transmitted to the control software for controlling the motor vehicle.

[0055] However, especially when controlling a motor vehicle, high requirements are placed here on safety and thus also on the robustness of such a machine learning algorithm in order to avoid safety-critical situations as far as possible.

[0056] Here Figure 1 Method 1 is shown, wherein in a first step 2 image data showing the environment of the motor vehicle is provided, wherein the image data contains a representation of at least one object, wherein in step 3 the representation of at least one object is modified such that the representation appears distorted, and wherein in step 4, based on the objects classified by the machine learning algorithm in the modified representation of at least one object, the control of the motor vehicle is simulated in order to test the robustness of the machine learning algorithm.

[0057] Method 1 is thus based on a simulation-based scenario for testing the driving functions of a motor vehicle or for checking the robustness of the algorithm with respect to changes in the conditions or changes in the detected environment of the motor vehicle.

[0058] Therefore, a method 1 is described, by which safety-related or safety-critical aspects can be identified or revealed early, especially during the development of machine learning algorithms. Based on the corresponding test results, it can also be verified whether the algorithm is robust enough against external influences, such as external attacks, and thus suitable for controlling a motor vehicle. Possible safety-related aspects can be reacted to early, and the method 1 works regardless of the structure of the machine learning algorithm, the type of training method, and the knowledge of the current data used for training the machine learning algorithm. This has the advantage that the time until the machine learning algorithm can actually be used for classifying objects in the environment of the motor vehicle can be reduced, and at the same time, resources such as storage capacity required for (re)training, optimizing, or making the machine learning algorithm more robust can be saved.

[0059] Overall, a method 1 is thus described, by which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be tested reliably and with little effort.

[0060] Here, for example, the machine learning algorithm can be a classifier based on an artificial neural network.

[0061] According to Figure 1 an embodiment of, the step 3 of modifying the representation of at least one object here includes: modifying the representation of at least one object such that the object is misclassified by the machine learning algorithm; and / or modifying the representation of at least one object based on how a third party can change the object; and / or modifying the representation of at least one object based on different lighting conditions and / or weather conditions.

[0062] Here, for example, the representation can be modified such that a stop sign is misclassified as a priority sign by the machine learning algorithm, simulating that a sticker or a comparable object has been pasted on the road sign, or the stop sign is perceived differently due to different solar irradiations. The modification of the representation in step 3 can be performed here, for example, by an image processing algorithm, where the image processing algorithm can, for example, modify the representation such that a simulated sticker can be pasted on the classified road sign, and / or the brightness of the representation can be adapted to simulate different light intensities.

[0063] In addition, the modification of the representation of at least one object can also be performed based on a machine learning algorithm trained to simulate external interventions. Here, the corresponding machine learning algorithm can be trained based on corresponding labeled training data, especially for generating artifacts, spots, or noise added to the representation.

[0064] According to Figure 1 an embodiment of, the image data is furthermore sensor data recorded by an environment sensor of the motor vehicle.

[0065] Subsequently, for example, corresponding test or inspection results can be used to (re)train or optimize a machine learning algorithm, for example in order to rule out safety-critical situations caused by changed conditions or interventions in the detected environmental data as far as possible when controlling a motor vehicle.

[0066] A method for testing the robustness of a machine learning algorithm can furthermore be implemented as a closed-loop method, in which the robustness or error of the machine learning algorithm can be analyzed in a representation that can be easily changed separately in a plurality of iterative loops or repetitions, and can be implemented as an open-loop method.

[0067] Furthermore, during the development phase, various tests or inspections can be carried out using different settings, for example different vehicle speeds or different camera angles.

[0068] Figure 2 A block diagram of a system for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle 10 according to an embodiment of the invention is shown.

[0069] In particular, Figure 2 A system for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle 10 is again shown, wherein the machine learning algorithm is trained to classify objects in image data representing the environment of the motor vehicle.

[0070] As Figure 2 shown, the system has a providing unit 11 here, which is configured to provide image data showing the environment of the motor vehicle, wherein the image data contains a representation of at least one object; a modifying unit 12, which is configured to modify the representation of at least one object such that the representation appears distorted; and a simulation unit 13, which is configured to simulate the control of the motor vehicle based on the classification of an object in the environment in the modified representation of at least one object by means of a machine learning algorithm in order to test the robustness of the machine learning algorithm.

[0071] The providing unit can here be, for example, a receiver, which is configured to receive corresponding sensor data. The modifying unit and the simulation unit can furthermore be implemented, for example, based on code stored in a memory and executable by a processor.

[0072] According to Figure 2 an embodiment, the modifying unit 12 is here again configured to modify the representation of at least one object such that the object is incorrectly classified by the machine learning algorithm, and / or to modify the representation of at least one object based on how a third party can change the object and / or based on different lighting conditions and / or weather conditions.

[0073] In particular, the modification unit 12 is configured to apply an image processing algorithm for modifying the representation.

[0074] Furthermore, the modification unit 12 is configured to apply a machine learning algorithm in order to modify the representation, the machine learning algorithm being trained to simulate an external intervention.

[0075] According to Figure 2 an embodiment, the image data is furthermore sensor data recorded by an environmental sensor of a motor vehicle again.

[0076] Furthermore, the illustrated system 10 is configured to perform the above-described method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle.

Claims

1. A method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, and wherein the method (1) comprises the following steps: - Providing image data showing the environment of the motor vehicle, wherein the image data contains a representation of at least one object (2); - Modifying the representation of the at least one object such that the representation appears distorted (3); and - Simulating the control of the motor vehicle based on the objects classified by the machine learning algorithm in the modified representation of the at least one object in order to test the robustness of the machine learning algorithm (4).

2. The method according to claim 1, wherein the step (3) of modifying the representation of the at least one object comprises: Modifying the representation of the at least one object such that the object is incorrectly classified by the machine learning algorithm; and / or modifying the representation of the at least one object based on how a third party could change the object; and / or modifying the representation of the at least one object based on different lighting conditions and / or weather conditions.

3. The method according to claim 1 or 2, wherein the step (3) of modifying the representation of the at least one object comprises applying an image processing algorithm.

4. The method according to any one of claims 1 to 3, wherein the step (3) of modifying the representation of the at least one object comprises applying a machine learning algorithm which is trained to simulate external interventions.

5. The method according to any one of claims 1 to 4, wherein the image data is sensor data recorded by an environment sensor of the motor vehicle.

6. A method for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, and wherein the method comprises the following steps: - Testing the robustness of the trained algorithm of the machine learning algorithm by the method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle according to any one of claims 1 to 5 in order to provide a test result; and - Optimizing the machine learning algorithm based on the provided test result.

7. A method for controlling a motor vehicle based on a machine learning algorithm, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, and wherein the method comprises the following steps: - Providing a machine learning algorithm for classifying objects in the environment of the motor vehicle, wherein the machine learning algorithm has been optimized by the method for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle according to claim 6; and - Controlling the motor vehicle based on the provided machine learning algorithm.

8. A system for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, and wherein the system (10) is configured to perform the method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle according to any one of claims 1 to 5.

9. A system for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, and wherein the system includes the system for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle according to claim 8, the system being configured to test the robustness of the trained algorithm of the machine learning algorithm to provide a test result; and an optimization unit configured to optimize the machine learning algorithm based on the provided test result.

10. A system for controlling a motor vehicle based on a machine learning algorithm, wherein the machine learning algorithm is trained to classify objects in image data representative of the environment of the motor vehicle, and wherein the system comprises: A providing unit configured to provide a machine learning algorithm for classifying objects in the environment of the motor vehicle, wherein the machine learning algorithm has been optimized by the system for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle according to claim 9; and a control unit configured to control the motor vehicle based on the provided machine learning algorithm.

11. A computer program having program code for performing the method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle according to any one of claims 1 to 5 when the computer program is executed on a computer.

12. A computer-readable data carrier having program code of a computer program for performing the method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle according to any one of claims 1 to 5 when the computer program is executed on a computer.

Images