Method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle

Abstract

The invention relates to a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the method (1) comprises the following steps: providing image data showing an environment of the motor vehicle, wherein the image data contain a depiction of at least one object (2), modifying the depiction of the at least one object such that it appears in distorted form (3), and simulating control of the motor vehicle based on objects classified by the machine learning algorithm in the modified depiction of the at least one object in order to test the robustness of the machine learning algorithm (4).

Description

[0001] Description

[0002] title

[0003] Method for testing the robustness of a machine learning algorithm for classifying objects in a motor vehicle environment

[0004] The invention relates to a method for testing the robustness of a machine learning algorithm, with which the influence of external influences or interventions on the control of the motor vehicle can be checked and the safety when controlling the motor vehicle can be increased based on the machine learning algorithm.

[0005] Machine learning algorithms are based on the use of statistical methods to train a computer to perform a specific task without having been explicitly programmed to do so. The goal of machine learning is to construct algorithms that can learn from data and make predictions. These algorithms create mathematical models that can be used, for example, to classify data.

[0006] Robustness is further understood as the ability of a machine learning algorithm to withstand changes, such as external attacks. Regarding external attacks, a general distinction is made between a white-box attack, in which the attacker has knowledge of the structure of the machine learning algorithm, the type of training procedure, and the available data used to train the machine learning algorithm, and a black-box scenario, in which an attacker lacks this knowledge but only sees the input data and the results the network outputs.Such machine learning algorithms are used in controlling a driver assistance system of a motor vehicle or in operating an autonomously driving motor vehicle. The machine learning algorithm can be configured to classify objects represented in the vehicle's environmental data, such as road signs or street lights, and the motor vehicle, or the driver assistance system, or the autonomously driving motor vehicle is controlled based on the classified objects. However, particularly when controlling a motor vehicle, high demands are placed on the safety and thus also on the robustness of such a machine learning algorithm in order to avoid safety-critical situations as much as possible.

[0007] From the document DE 10 2019 209 560 A1 a method for training a neural network is known, the method comprising providing a training data set comprising training images showing a vehicle environment from the perspective of a vehicle, wherein a plurality of the training images show traffic signs, generating additional training images by augmenting training images showing traffic signs, by augmenting a training image showing a traffic sign by partially covering the traffic sign and / or augmenting a training image showing a variable message sign by changing the lighting state of one or more lighting elements of the variable message sign and training the neural network based on at least the augmented training images.

[0008] The invention is based on the object of reliably and without great effort testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle.

[0009] The problem is solved by a method for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle according to the features of the patent claim. The problem is also solved by a system for testing the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle according to the features of patent claim 8.

[0010] Disclosure of the invention

[0011] According to one embodiment of the invention, this object is achieved by a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the method comprises providing image data showing an environment of the motor vehicle, wherein the image data includes a representation of at least one object, modifying the representation of the at least one object such that it appears distorted, and simulating a control of the motor vehicle based on objects classified by the machine learning algorithm based on the modified representation of the at least one object in order to test the robustness of the machine learning algorithm.

[0012] Image data refers to data that can be represented as an image or graphic using a special program. The fact that an object is represented in image data also means that the corresponding image data shows the object or contains a representation of the object.

[0013] The fact that the representation of the object is manipulated or modified in such a way that it appears falsified also means that the representation of the object is changed in a realistic but also inconspicuous way, for example to simulate an attack from outside.

[0014] This provides a method for identifying or uncovering safety-relevant or safety-critical aspects early on, especially during the development of the machine learning algorithm. Based on the corresponding test results, it can also be verified whether the algorithm is sufficiently robust against external influences, such as external attacks, and thus suitable for controlling a motor vehicle.

[0015] The fact that it is possible to react to possible safety-relevant aspects at an early stage, whereby the method also functions without knowledge of the structure of the machine learning algorithm, the type of training procedure and the available data with which the machine learning algorithm was trained, also has the advantage that the time until the machine learning algorithm can actually be used to classify objects in the environment of a motor vehicle can be reduced, while at the same time resources required for (re-)training or optimising or making the machine learning algorithm more robust, for example storage capacities, can be saved.

[0016] Overall, a method is provided with which the robustness of a machine learning algorithm for classifying objects in a motor vehicle environment can be tested reliably and without great effort.

[0017] The step of modifying the representation of the at least one object may comprise modifying the representation of the at least one object such that the object is incorrectly classified by the machine learning algorithm, and / or modifying the representation of the at least one object based on how the object could be altered by third parties, and / or modifying the representation of the at least one object based on different lighting and / or weather conditions.

[0018] The fact that the representation of at least one object is modified in such a way that it is incorrectly classified by the machine learning algorithm means that the machine learning algorithm assigns the object, based on the modified representation, to a different class than the one to which the object actually belongs, or even to no class at all, i.e. the classification result is falsified.

[0019] The fact that the representation of at least one object is modified based on how the object could be changed by third parties also means that realistic or expected changes in conditions and / or known external attacks could be simulated.

[0020] The fact that the representation of the at least one object is modified based on different lighting conditions further means that the representation of the object is adapted to possible other lighting conditions. For example, street signs can be perceived differently depending on the sunlight. The fact that the representation of the at least one object is modified based on different weather conditions further means that the representation of the object is adapted to possible weather conditions, for example, rain and / or fog.

[0021] The representation of the at least one object can thus be adapted to all known, realistic and expected changes in the conditions in order to simulate the behavior of the machine learning algorithm and thus also of the motor vehicle in the presence of these known, realistic and expected changes.

[0022] In one embodiment, the step of modifying the representation of the at least one object comprises applying an image processing algorithm.

[0023] An image processing algorithm is an algorithm designed to change or modify image data. For example, objects can be rotated and / or scaled differently, or additional objects can be added to the original object.

[0024] The modification of the representation of the at least one object can thus be performed using known and common algorithms, without the need for complex and resource-intensive adaptations. In a further embodiment, the step of modifying the representation of the at least one object comprises applying a machine learning algorithm that is trained to simulate external interventions.

[0025] The fact that the machine learning algorithm is trained to simulate external attacks means that the machine learning algorithm is an adversarial generator or is trained to manipulate the image data for the machine learning algorithm.

[0026] This means that the display can be automatically adapted as precisely as possible to known manipulation or change patterns.

[0027] Furthermore, the image data may be sensor data recorded by environmental sensors of the motor vehicle.

[0028] A sensor, which is also referred to as a detector, (measured variable or measuring) sensor or (measured) probe, is a technical component that can record certain physical or chemical properties and / or the material properties of its environment qualitatively or quantitatively as a measured variable.

[0029] An environmental sensor is further understood to mean a sensor of the motor vehicle which is designed to detect data about an environment or at least a part of an environment of the motor vehicle.

[0030] The procedure for testing the robustness of the machine learning algorithm can thus be based on conditions outside the actual data processing system on which the machine learning algorithm is tested.

[0031] A further embodiment of the invention also provides a method for optimizing a machine learning algorithm for classifying objects in an environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the method comprises testing the robustness of the trained algorithm of the machine learning algorithm by a method described above for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle to provide test results, and optimizing the machine learning algorithm based on the provided test results.

[0032] The optimization of the machine learning algorithm means, in particular, that the machine learning algorithm is retrained based on the test results in order to avoid, as far as possible, safety-critical situations based on objects classified by the machine learning algorithm, especially when driving the motor vehicle.

[0033] Thus, a method for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle is specified. This method is based on a method with which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be reliably and easily tested. In particular, this method is based on a method with which safety-relevant or safety-critical aspects can be identified or uncovered early on, particularly during the development of the machine learning algorithm. Based on the corresponding test results, it can also be checked whether the algorithm is sufficiently robust against external influences, such as external attacks, and is therefore suitable for controlling a motor vehicle.The fact that it is possible to react to possible safety-relevant aspects at an early stage, whereby the method can also operate without knowledge of the structure of the machine learning algorithm, the type of training procedure and the available data with which the machine learning algorithm was trained, also has the advantage that the time until the machine learning algorithm can actually be used to classify objects in the environment of a motor vehicle can be reduced, while at the same time resources required for (re-)training or optimising or making the machine learning algorithm more robust, for example storage capacities, can be saved.

[0034] A further embodiment of the invention also provides a method for controlling a motor vehicle based on a machine learning algorithm, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the method comprises providing a machine learning algorithm for classifying objects in an environment of the motor vehicle, wherein the machine learning algorithm has been optimized by a method described above for optimizing a machine learning algorithm for classifying objects in an environment of a motor vehicle, and controlling the motor vehicle based on the provided machine learning algorithm.

[0035] Thus, a method for controlling a motor vehicle based on a machine learning algorithm is specified. This method is based on a method with which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be reliably and easily tested. In particular, this method is based on a method with which safety-relevant or safety-critical aspects can be identified or uncovered early on, particularly during the development of the machine learning algorithm. Based on the corresponding test results, it can also be verified whether the algorithm is sufficiently robust against external influences, such as external attacks, and thus suitable for controlling a motor vehicle.The fact that it is possible to react to possible safety-relevant aspects at an early stage, whereby the method can also operate without knowledge of the structure of the machine learning algorithm, the type of training procedure and the available data with which the machine learning algorithm was trained, also has the advantage that the time until the machine learning algorithm can actually be used to classify objects in the environment of a motor vehicle can be reduced, while at the same time resources required for (re-)training or optimising or making the machine learning algorithm more robust, for example storage capacities, can be saved.

[0036] A further embodiment of the invention also provides a system for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the system is designed to carry out a method described above for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle.

[0037] Thus, a system is provided with which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be reliably and easily tested. In particular, a system is provided that is designed to detect or uncover safety-relevant or safety-critical aspects early on, especially during the development of the machine learning algorithm. Based on the corresponding test results, it can also be verified whether the algorithm is sufficiently robust against external influences, such as external attacks, and thus suitable for controlling a motor vehicle.The fact that it is possible to react to possible safety-relevant aspects at an early stage, whereby the method can also operate without knowledge of the structure of the machine learning algorithm, the type of training procedure and the available data with which the machine learning algorithm was trained, also has the advantage that the time until the machine learning algorithm can actually be used to classify objects in the environment of a motor vehicle can be reduced, while at the same time resources required for (re-)training or optimising or making the machine learning algorithm more robust, for example storage capacities, can be saved.A further embodiment of the invention also provides a system for optimizing a machine learning algorithm for classifying objects in an environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the system comprises a system as described above for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle, which system is designed to test the robustness of the trained algorithm of the machine learning algorithm in order to provide test results, and an optimization unit which is designed to optimize the machine learning algorithm based on the provided test results.

[0038] Thus, a system for optimizing a machine learning algorithm for classifying objects in the environment of a motor vehicle is specified. This system is based on a system with which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be reliably and easily tested. In particular, this system is based on a system that is designed to detect or uncover safety-relevant or safety-critical aspects early on, particularly during the development of the machine learning algorithm. Based on the corresponding test results, it can also be checked whether the algorithm is sufficiently robust against external influences, such as external attacks, and is therefore suitable for controlling a motor vehicle.The fact that it is possible to react to possible safety-relevant aspects at an early stage, whereby the method can also operate without knowledge of the structure of the machine learning algorithm, the type of training procedure and the available data with which the machine learning algorithm was trained, also has the advantage that the time until the machine learning algorithm can actually be used to classify objects in the environment of a motor vehicle can be reduced, while at the same time resources required for (re-)training or optimising or making the machine learning algorithm more robust, for example storage capacities, can be saved.

[0039] A further embodiment of the invention also provides a system for controlling a motor vehicle based on a machine learning algorithm, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the system has a provision unit which is designed to provide a machine learning algorithm for classifying objects in an environment of the motor vehicle, wherein the machine learning algorithm has been optimized by a system described above for optimizing a machine learning algorithm for classifying objects in an environment controlling a motor vehicle, and a control unit which is designed to control the motor vehicle based on the provided machine learning algorithm.

[0040] Thus, a system for controlling a motor vehicle based on a machine learning algorithm is specified, which is based on a system with which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be reliably and easily tested. In particular, this is based on a system that is designed to be able to recognize or uncover safety-relevant or safety-critical aspects early on, in particular already during the development of the machine learning algorithm. Based on the corresponding test results, it can also be checked whether the algorithm is robust enough against external influences, such as external attacks, and is therefore suitable for classifying objects in the environment of a motor vehicle.The fact that it is possible to react to possible safety-relevant aspects at an early stage, whereby the method can also operate without knowledge of the structure of the machine learning algorithm, the type of training procedure and the available data with which the machine learning algorithm was trained, also has the advantage that the time until the machine learning algorithm can actually be used to classify objects in the environment of a motor vehicle can be reduced, while at the same time resources required for (re-)training or optimising or making the machine learning algorithm more robust, for example storage capacities, can be saved.

[0041] A further embodiment of the invention also provides a computer program with program code for carrying out a method described above for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle when the computer program is executed on a computer.

[0042] A further embodiment of the invention also provides a computer-readable data carrier with program code of a computer program for carrying out a method described above for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle when the computer program is executed on a computer.

[0043] The computer program and the computer-readable data carrier each have the advantage that they are each designed to carry out a method with which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be reliably and easily tested. In particular, they are each designed to carry out a method designed to detect or uncover safety-relevant or safety-critical aspects early on, in particular already during the development of the machine learning algorithm. Based on the corresponding test results, it can also be checked whether the algorithm is sufficiently robust against external influences, such as external attacks, and is therefore suitable for controlling a motor vehicle.The fact that it is possible to react to possible safety-relevant aspects at an early stage, whereby the method can also operate without knowledge of the structure of the machine learning algorithm, the type of training procedure and the available data with which the machine learning algorithm was trained, also has the advantage that the time until the machine learning algorithm can actually be used to classify objects in the environment of a motor vehicle can be reduced, while at the same time resources required for (re-)training or optimising or making the machine learning algorithm more robust, for example storage capacities, can be saved.

[0044] In summary, the present invention provides a method for testing the robustness of a machine learning algorithm, with which the influence of external influences or interventions on the control of the motor vehicle can be checked and the safety when controlling the motor vehicle can be increased based on the machine learning algorithm.

[0045] The described designs and further training courses can be combined as desired.

[0046] Further possible embodiments, developments and implementations of the invention also include combinations of features of the invention described previously or below with regard to the embodiments that are not explicitly mentioned.

[0047] Short description of the drawings

[0048] The accompanying drawings are intended to provide a further understanding of embodiments of the invention. They illustrate embodiments and, in conjunction with the description, serve to explain principles and concepts of the invention.

[0049] Other embodiments and many of the aforementioned advantages will become apparent upon review of the drawings. The elements illustrated in the drawings are not necessarily drawn to scale.

[0050] 1 shows a flowchart of a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle according to embodiments of the invention; and

[0051] Fig.2 is a schematic block diagram of a system for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle according to embodiments of the invention.

[0052] In the figures of the drawings, the same reference symbols designate the same or functionally equivalent elements, parts or components, unless otherwise stated.

[0053] Fig. 1 shows a flowchart of a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle 1 according to embodiments of the invention.

[0054] In particular, Fig.l shows a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle 1, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle.

[0055] To control a motor vehicle, or functions of a motor vehicle, or an autonomously driving motor vehicle using a machine learning algorithm, where the machine learning algorithm is trained to classify objects in image data representing the vehicle's surroundings, environmental data is usually collected by one or more environmental sensors of the motor vehicle. These environmental sensors can be, for example, cameras, radar sensors, and / or lidar sensors. The data from the individual sensors can then be linked to one another, and a machine learning algorithm trained accordingly based on labeled training data or training data provided with corresponding information can classify objects in the linked data.The classified data can then be transmitted to control software for controlling the motor vehicle.

[0056] However, particularly when driving a motor vehicle, high demands are placed on the safety and thus also on the robustness of such a machine learning algorithm in order to avoid safety-critical situations as far as possible.

[0057] Fig. 1 shows a method 1, wherein in a first step 2 image data showing an environment of the motor vehicle are provided, wherein the image data contain a representation of at least one object, wherein in a step 3 the representation of the at least one object is modified such that it appears falsified, and wherein in a step 4 a control of the motor vehicle is simulated based on objects classified by the machine learning algorithm in the modified representation of the at least one object in order to test the robustness of the machine learning algorithm.

[0058] Method 1 is therefore based on a simulation-based approach to test the driving function of the motor vehicle, or to check the robustness of the algorithm against changes in the conditions or changes in the detected environment of the motor vehicle.

[0059] Thus, a method 1 is provided with which safety-relevant or safety-critical aspects can be identified or uncovered early on, especially during the development of the machine learning algorithm. Based on the corresponding test results, it can also be verified whether the algorithm is sufficiently robust against external influences, such as external attacks, and thus suitable for controlling a motor vehicle.The fact that it is possible to react to possible safety-relevant aspects at an early stage, whereby method 1 also works without knowledge of the structure of the machine learning algorithm, the type of training procedure and the available data with which the machine learning algorithm was trained, also has the advantage that the time until the machine learning algorithm can actually be used to classify objects in the environment of a motor vehicle can be reduced, while at the same time resources required for (re-)training or optimising or making the machine learning algorithm more robust, for example storage capacities, can be saved.

[0060] Overall, a method 1 is provided with which the robustness of a machine learning algorithm for classifying objects in the environment of a motor vehicle can be tested reliably and without great effort.

[0061] The machine learning algorithm can, for example, be a classifier based on an artificial neural network.

[0062] According to the embodiments of Fig. 1, step 3 of modifying the representation of the at least one object comprises modifying the representation of the at least one object such that the object is incorrectly classified by the machine learning algorithm, and / or modifying the representation of the at least one object based on how the object could be changed by third parties, and / or modifying the representation of the at least one object based on different lighting and / or weather conditions.

[0063] For example, the representation can be modified in such a way that a stop sign is incorrectly classified as a yield sign by the machine learning algorithm, that a sticker or similar object is simulated as being stuck to a road sign, or that the sign is perceived differently due to different sunlight. The modification of the representation in step 3 can be performed, for example, by an image processing algorithm. The image processing algorithm can modify the representation, for example, to simulate a sticker being stuck to a classified road sign, and / or to adjust the brightness of the representation to simulate different light intensities.

[0064] Furthermore, the modification of the representation of the at least one object can also be performed based on a machine learning algorithm trained to simulate external interventions. The corresponding machine learning algorithm can be trained, in particular, based on corresponding labeled training data to generate artifacts, spots, or noise, which are added to the representation.

[0065] According to the embodiments of Fig. 1, the image data also comprises sensor data recorded by environmental sensors of the motor vehicle.

[0066] The corresponding test or verification results can then be used, for example, to (re-)train or optimize the machine learning algorithm, for example to exclude, as far as possible, safety-critical situations when driving the motor vehicle due to changed conditions or interventions in the detected environmental data.

[0067] The method for testing the robustness of the machine learning algorithm can be implemented as a closed-loop method, where the robustness or errors of the machine learning algorithm can be analyzed in several iterations or repetitions with slightly different representations each time, or as an open-loop method. Furthermore, various tests or checks can be conducted during the development phase with different settings, for example, different vehicle speeds or different camera angles.

[0068] Fig. 2 shows a block diagram of a system for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle 10 according to embodiments of the invention.

[0069] In particular, Fig.2 again shows a system for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle 10, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle.

[0070] As Fig. 2 shows, the system has a provision unit 11, which is designed to provide image data showing an environment of the motor vehicle, wherein the image data contains a representation of at least one object, a modification unit 12, which is designed to modify the representation of the at least one object in such a way that it appears distorted, and a simulation unit 13, which is designed to simulate a control of the motor vehicle based on the classification of objects in an environment by the machine learning algorithm in the modified representation of the at least one object in order to test the robustness of the machine learning algorithm.

[0071] The provision unit can, for example, be a receiver configured to receive corresponding sensor data. The modification unit and the simulation unit can also be implemented, for example, based on code stored in a memory and executable by a processor.

[0072] According to the embodiments of Fig. 2, the modification unit 12 is in turn designed to modify the representation of the at least one object such that the object is incorrectly classified by the machine learning algorithm and / or based on how the object could be modified by third parties and / or based on different lighting and / or weather conditions. In particular, the modification unit 12 is designed to have a

[0073] Apply image processing algorithm to modify the representation.

[0074] In addition, the modification unit 12 is designed to apply a machine learning algorithm, which is trained to simulate external interventions, in order to modify the representation.

[0075] According to the embodiments of Fig. 2, the image data are again sensor data recorded by environmental sensors of the motor vehicle.

[0076] In addition, the illustrated system 10 is designed to carry out a method described above for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle.

Claims

Claims 1. A method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the method (1) comprises the following steps: Providing image data showing an environment of the motor vehicle, wherein the image data includes a representation of at least one object (2); Modifying the representation of the at least one object in such a way that it appears distorted (3); and Simulating a control of the motor vehicle based on objects classified by the machine learning algorithm in the modified representation of the at least one object in order to test the robustness of the machine learning algorithm (4).

2. The method according to claim 1, wherein the step of modifying the representation of the at least one object (3) comprises modifying the representation of the at least one object such that the object is incorrectly classified by the machine learning algorithm, and / or modifying the representation of the at least one object based on how the object could be altered by third parties, and / or modifying the representation of the at least one object based on different lighting and / or weather conditions.

3. The method according to claim 1 or 2, wherein the step of modifying the representation of the at least one object (3) comprises applying an image processing algorithm.

4. The method according to any one of claims 1 to 3, wherein the step of modifying the representation of the at least one object (3) comprises applying a machine learning algorithm trained to simulate external interventions.

5. The method according to any one of claims 1 to 4, wherein the image data is sensor data recorded by environmental sensors of the motor vehicle.

6. A method for optimizing a machine learning algorithm for classifying objects in an environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the method comprises the following steps: Testing the robustness of the trained algorithm of the machine learning algorithm by a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle according to any one of claims 1 to 5 to provide test results; and Optimize the machine learning algorithm based on the provided test results.

7. A method for controlling a motor vehicle based on a machine learning algorithm, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the method comprises the following steps: Providing a machine learning algorithm for classifying objects in an environment of the motor vehicle, wherein the machine learning algorithm has been optimized by a method for optimizing a machine learning algorithm for classifying image data in an environment of a motor vehicle according to claim 6; and Controlling the motor vehicle based on the provided machine learning algorithm.

8. A system for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the system (10) is designed to carry out a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle according to one of claims 1 to 5.

9. A system for optimizing a machine learning algorithm for classifying objects in an environment of a motor vehicle, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the system comprises a system for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle according to claim 8, which is configured to test the robustness of the trained algorithm of the machine learning algorithm to provide test results, and an optimization unit configured to optimize the machine learning algorithm based on the provided test results.

10. System for controlling a motor vehicle based on a machine learning algorithm, wherein the machine learning algorithm is trained to classify objects in image data representing an environment of the motor vehicle, and wherein the system comprises a provision unit which is designed to provide a machine learning algorithm for classifying objects in an environment of the motor vehicle, wherein the machine learning algorithm is implemented by a system for optimizing a machine learning algorithm for Classifying objects in an environment of a motor vehicle according to claim 9, and comprising a control unit which is designed to control the motor vehicle based on the provided machine learning algorithm.

11. A computer program comprising program code for carrying out a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle according to any one of claims 1 to 5 when the computer program is executed on a computer.

12. Computer-readable data carrier with program code of a computer program to implement a method for testing the robustness of a machine learning algorithm for classifying objects in an environment of a motor vehicle according to one of the Claims 1 to 5 when the computer program is executed on a computer.

Images