Method and apparatus for reliably operating a splitter

Abstract

A method for determining depth and / or orientation and / or position information of an object in an input image (x) detected by means of an imaging sensor, wherein the input image (x) is converted by means of a segmenter (61), in particular by means of a first neural network, into a textureless representation (SEM), and a list of objects (y) is determined by means of an estimator (71), in particular by means of a second neural network, based on the textureless representation (SEM), the list of objects characterizing the objects present in the input image (x) and the corresponding depth and / or orientation and / or position information of these objects.

Description

Technical Field

[0001] The present invention relates to a method for determining depth and / or direction and / or position information, a locator, a method for determining a robustness value of the locator, a method for providing a control signal, a control system, a method for training the locator, a training system, a computer program, and a machine-readable storage medium. Background Art

[0002] A method for detecting an object in a provided input signal by means of a neural network is known from DE 10 2017 223 264.1, which has not been previously published, wherein the object is detected based on the provided input signal and wherein an actuator is controlled based on the detection of the object.

[0003] Advantages of the Invention

[0004] When operating a safety-critical application based on object detection, i.e., semantic segmentation and / or classification of input signals (the safety-critical application may, but is not necessarily, the control of a motor vehicle, for example), the challenge is to ensure that there cannot be a danger caused by incorrectly determined semantic segmentation and / or classification.

[0005] Evaluating the robustness of such a system requires a variety of test cases, which are difficult to present using real image data. Therefore, it is desirable to determine the robustness using synthetic image data.

[0006] However, it is known that neural networks, in particular, attach great importance to the texture of an object when assigning it to a category. Although textures can be synthetically generated so that they appear realistic to the human eye. However, it is difficult to reliably synthesize and generate the actual bandwidth of possible structures in a verifiable manner.

[0007] Correspondingly, the method having the features of independent claim 1 has the advantage that a machine learning system is trained to perform semantic segmentation and / or classification such that the determination of the semantic segmentation and / or classification is performed independently of the texture.

[0008] The method of the present invention is the subject matter of independent claim 1. Other aspects of the present invention are the subject matter of the dependent claims. Advantageous expansions are the subject matter of the dependent claims. Summary of the Invention

[0009] Thus, in a first aspect, the present invention relates to a computer-implemented method, in particular, for training a machine learning system, especially a neural network, to create a semantic segmentation and / or classification of an input image detected by means of an imaging sensor, wherein at least one training image is provided, which has an associated per-pixel semantic segmentation, and wherein a new training image is generated based on the training image by covering, in particular completely covering, at least one predefined region corresponding to a segmentation region of the provided semantic segmentation with a predefined texture in the training image.

[0010] Moreover, the machine learning system determines the semantic segmentation for the new training image, and the parameters for parameterizing the machine learning system are adapted based on the determined semantic segmentation and the provided semantic segmentation.

[0011] In particular, it is also possible to provide a plurality of new training images, each of which has been covered with at least partially different textures. Thereby, it is possible to ensure particularly effectively that the machine learning system is optimized through training to pay attention to textures.

[0012] A particularly simple extension provides that realistic textures are obtained by cutting the predefined texture from other provided training images.

[0013] In a further extension, it can be provided that the predefined texture is cut from and used in a region of other provided training images that is completely within a single segmentation region belonging to another semantic segmentation of the other provided training images. In this way, it is possible to ensure particularly simply an improved convergence of the above training direction, because otherwise the training data record would be contaminated with potentially contradictory textures.

[0014] In an alternative or additional extension, it can be provided that the predefined region is randomly selected, in particular randomly selected from a plurality of segmentation regions of the provided semantic segmentation. Thereby, it is possible to generate particularly simply a training data record that elicits as well as possible a high abstraction ability for the corresponding texture.

[0015] In the case of these methods, it is possible that the provided training image xs has been captured by means of an imaging sensor.

[0016] However, alternatively, it is also possible that the provided training image is generated according to a predefined list of geometric structures, in particular texturelessly. In this case, "textureless" can in particular mean that the color values within the corresponding boundaries of the geometric structure do not change. Generating geometric structures texturelessly is particularly simple. Subsequently, the texture can be applied in the next step as described above.

[0017] In one of the extension scenarios, it can be stipulated that the machine learning system to be trained consists of a front subsystem and a rear subsystem, and when adapting the parameters according to the determined semantic segmentation and the provided semantic segmentation, only the parameters parameterizing the front subsystem are adapted. The term "front" subsystem or "rear" subsystem may, in this case, mean that the machine learning system is constructed such that the input image of the machine learning system is first, in particular directly, fed as an input parameter to the front subsystem, and the associated output parameter provided at the output of the front subsystem is then, in particular directly, fed as an input parameter to the rear subsystem, and then the output signal of the machine learning system is provided at the output of the rear subsystem based on this input parameter of the rear subsystem. This construction has the advantage that the front subsystem is first trained to identify basic geometric shapes, and the rear subsystem can then be specifically trained to identify realistic shapes.

[0018] Subsequently, it can be advantageously stipulated that if a new training image is fed to the machine learning system, the determined semantic segmentation is provided at the output of the front subsystem.

[0019] Training of such a constructed machine learning system is possible in a particularly simple manner, where after training the front subsystem, at least one additional training image captured by an imaging sensor and the rated output signal assigned to one or more of the training images, in particular the desired semantic segmentation and / or classification, are provided to the machine learning system.

[0020] Wherein the machine learning system provides the output signal attached to the output of the rear subsystem based on at least one additional training image and the parameters parameterizing the rear subsystem are adapted according to the consistency between the rated output signal and the output signal attached to the output of the rear subsystem. In particular, only the parameters parameterizing the rear subsystem are adapted.

[0021] In another aspect of the present invention, it can be stipulated that a robustness value characterizing the robustness of the machine learning system is determined and the training is continued or not continued based on the determined robustness value.

[0022] In particular, it can be stipulated that if the robustness value corresponds to a low robustness of the machine learning system, the training is not continued; while if the robustness value corresponds to a high robustness of the machine learning system, the training is continued.

[0023] The method for determining the robustness of a machine learning system relates to another aspect of the present invention. The method can be carried out as follows: providing a per-pixel semantic segmentation to which the detected input image belongs, generating other input images according to the input image in a manner similar to other training images, in such a way that at least one pre-given region corresponding to the segmentation region of the provided semantic segmentation is covered, especially completely covered, with a pre-given texture in the other input images, and wherein the machine learning system determines an output signal according to the detected input image, especially the semantic segmentation and / or classification of the detected input image, and determines a second output signal according to the generated other input images, and a robustness value characterizing the robustness is determined according to the determined output signal and the determined second output signal.

[0024] In particular, it can be provided that: the robustness value is selected such that if the output signal and the second output signal are very different, especially differ by more than a pre-given value, then the robustness value corresponds to low robustness, and if the output signal and the second output signal are not very different, especially differ by no more than a pre-given value, then the robustness value corresponds to high robustness.

[0025] In another aspect, the present invention relates to a method for providing a control signal for controlling an actuator, wherein in a first stage, one of the aforementioned training methods is used to train a machine learning system, and in a second stage, a control signal is provided according to the semantic segmentation determined by the machine learning system.

[0026] In this case, in particular, it can be provided that: the aforementioned robustness value is determined as described above, and the control signal is provided correspondingly according to the value of the robustness value.

[0027] In this case, it can be provided that: the control signal is determined according to the normal operating mode of the actuator (when the robustness value corresponds to high robustness) or according to the protected operating mode of the actuator (when the robustness value corresponds to low robustness).

[0028] In other aspects, the present invention relates to a computer program and a machine-readable storage medium, which are configured to implement the above method. Description of the Drawings

[0029] Subsequently, embodiments of the present invention are further described with reference to the accompanying drawings. In the drawings:

[0030] Figure 1 The structure of an embodiment of the present invention is schematically shown;

[0031] Figure 2 An embodiment for controlling at least a partially autonomous robot is schematically shown;

[0032] Figure 3 An embodiment for controlling a production system is schematically shown;

[0033] Figure 4 An embodiment for controlling a personal assistant is schematically shown;

[0034] Figure 5 An embodiment for controlling an access system is schematically shown;

[0035] Figure 6 An embodiment for controlling a monitoring system is schematically shown;

[0036] Figure 7 An exemplary configuration of a machine learning system is schematically shown;

[0037] Figure 8 An aspect of a training device for training a machine learning system is schematically shown;

[0038] Figure 9 Another aspect of a training device for training a machine learning system is schematically shown;

[0039] Figure 10 A flow of a method for generating new (training) images is schematically shown;

[0040] Figure 11 An exemplary configuration of a system for evaluating the robustness of a machine learning system is schematically shown. Detailed Description

[0041] Figure 1 An actuator 10 interacting with a control system 40 in its surrounding environment 20 is shown. The actuator 10 and the surrounding environment 20 are also jointly referred to as an actuator system. The state of the actuator system is detected by a sensor 30 at preferably uniform time intervals, and the sensor may also be given by a plurality of sensors. The sensor signal S of the sensor 30 or, in the case of a plurality of sensors, each sensor signal S is transmitted to the control system 40. Accordingly, the control system 40 receives a sequence of sensor signals S. The control system 40 determines a control signal A based on this, and the control signal is transmitted to the actuator 10.

[0042] The control system 40 receives, in an optional receiving unit 50, a sequence of sensor signals S of the sensor 30, which receiving unit converts the sequence of sensor signals S into a sequence of input images x (alternatively, the sensor signals S can also be directly used as the input images x). The input image x can, for example, be a segment of the sensor signal S or a further processing of the sensor signal S. The input image x can, for example, include image data or an image, or include individual frames of a video recording. In other words, the input image x is determined based on the sensor signal S. The input image x is fed to a machine learning system, in this embodiment a neural network 60.

[0043] The neural network 60 is preferably parameterized by parameters φ, which are stored in a parameter memory P and provided by this parameter memory.

[0044] The neural network 60 determines an output signal y based on the input image x. The output signal y is fed to an optional modification unit 80, which determines a control signal A therefrom, and the control signal is fed to the actuator 10 in order to correspondingly control the actuator 10. In this case, the output signal y includes at least one classification of the input image x, where semantic segmentation is also possible, in which case categories are assigned to the individual sections of the input image x respectively.

[0045] The actuator 10 receives the control signal A, is correspondingly controlled and performs a corresponding action. In this case, the actuator 10 can include (not necessarily structurally integrated) control logic, which determines a second control signal for subsequently controlling the actuator 10 based on the control signal A.

[0046] In other embodiments, the control system 40 includes the sensor 30. In still other embodiments, alternatively or additionally, the control system 40 also includes the actuator 10.

[0047] In other preferred embodiments, the control system 40 includes one or more processors 45 and at least one machine-readable storage medium 46, on which commands are stored, which, when implemented on the processors 45, cause the control system 40 to perform the method according to the invention.

[0048] In an alternative embodiment, instead of or in addition to the actuator 10, a display unit 10a is provided.

[0049] Figure 2 An embodiment is shown in which the control system 40 is used to control at least a partially autonomous robot, here a at least a partially autonomous motor vehicle 100.

[0050] The sensor 30 can for example be one or more imaging sensors preferably arranged in a motor vehicle 100, such as one or more optical sensors and / or one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more LiDAR (Light Detection and Ranging) sensors and / or one or more thermal sensors.

[0051] The neural network 60 can detect objects in the surroundings of at least a partially autonomous robot, for example, based on the input image x. The output signal y can be a semantic segmentation and / or classification of the input image x (especially pixel by pixel). Then, the output signal A can be determined based on this.

[0052] The actuator 10 preferably arranged in the motor vehicle 100 can for example be the braking device, the drive device or the steering device of the motor vehicle 100. Then, the control signal A can be determined such that the actuator or actuators 10 are controlled such that the motor vehicle 100, especially when it comes to certain classes of objects, such as pedestrians, for example, prevents a collision with the objects identified by the neural network 60. In other words, the control signal A can be determined based on the determined class and / or corresponding to the determined class.

[0053] Alternatively, the at least partially autonomous robot can also be other mobile robots (not shown), such as robots that move forward by flying, floating, diving or walking. The mobile robot can for example also be at least a partially autonomous lawn mower or at least a partially autonomous cleaning robot. In these cases, the control signal A can also be determined such that the drive device and / or the steering device of the mobile robot are controlled such that the at least partially autonomous robot, for example, prevents a collision with the objects identified by the neural network 60.

[0054] In another alternative, the at least partially autonomous robot can also be a garden robot (not shown), which uses the imaging sensor 30 and the neural network 60 to determine the type or state of plants in the surroundings 20. Then, the actuator 10 can for example be a feeder for chemicals. The control signal A can be determined based on the determined type of plant or the determined state of the plant such that an amount of chemicals corresponding to the determined type or the determined state is applied.

[0055] In still other alternatives, the at least partially autonomous robot can also be a household appliance (not depicted), in particular a washing machine, a stove, an oven, a microwave oven or a dishwasher. Using the sensor 30, for example an optical sensor, the state of the object to be processed by the household appliance can be detected. For example, in the case of a washing machine, the state of the laundry in the washing machine can be detected. Then, using the neural network 60, the type or state of the object can be determined and the type or state of the object can be characterized by the output signal y. Then, the control signal A can be determined such that the household appliance is controlled according to the determined type or the determined state of the object. For example, in the case of a washing machine, the washing machine can be controlled according to what material the laundry in it is made of. Then, the control signal A can be selected according to what material of the laundry has been determined.

[0056] Figure 3 An embodiment is shown in which the control system 40 is used to control the production machine 11 of the production system 200 by controlling the actuator 10 that controls the production machine 11. The production machine 11 can be, for example, a machine for stamping, sawing, drilling and / or cutting.

[0057] Then, the sensor 30 can be, for example, an optical sensor that detects the characteristics of the finished product 12. It is possible that the actuator 10 that controls the production machine 11 is controlled according to the determined characteristics of the finished product 12 so that the production machine 11 correspondingly performs the subsequent processing steps of the finished product 12. It is also possible that the sensor 30 determines the characteristics of the finished product 12 to be processed by the production machine 11 and adapts the control of the production machine 11 accordingly for the next finished product.

[0058] Figure 4 An embodiment is shown in which the control system 40 is used to control the personal assistant 250. Preferably, the sensor 30 is an optical sensor that receives an image of the posture of the user 249.

[0059] According to the signal of the sensor 30, the control system 40 determines the control signal A of the personal assistant 250, for example in such a way that a neural network performs posture recognition. Then, the determined control signal A is transmitted to the personal assistant 250 and the personal assistant is thus controlled correspondingly. The determined control signal A can in particular be selected such that the control signal corresponds to the desired control envisaged by the user 249. The envisaged desired control can be determined according to the posture recognized by the neural network 60. Then, the control system 40 can select the control signal A for transmission to the personal assistant 250 according to the envisaged desired control and / or select the control signal A for transmission to the personal assistant 250 corresponding to the envisaged desired control.

[0060] The corresponding manipulation may include, for example, the personal assistant 250 calling information from the database and reproducing this information in a manner that is receivable by the user 249.

[0061] Instead of the personal assistant 250, household appliances (not shown), in particular washing machines, stoves, ovens, microwave ovens or dishwashers, may also be provided so as to be manipulated correspondingly.

[0062] Figure 5 An embodiment is shown in which the control system 40 is used to manipulate an access system 300. The access system 300 may, for example, include a physical access control device, such as a door 401. The sensor 30 may be, for example, an optical sensor (for example for detecting image or video data), which is set up to detect faces. With the aid of a neural network 60, the detected image can be interpreted. For example, the identity of a person can be determined. The actuator 10 may be a lock, which activates or does not activate the access control device according to a manipulation signal A, for example opens or does not open the door 401. For this purpose, the manipulation signal A may be selected according to the interpretation of the neural network 60, for example according to the determined identity of the person. Instead of a physical access control device, a logical access control device may also be provided.

[0063] Figure 6 An embodiment is shown in which the control system 40 is used to control a monitoring system 400. In Figure 5 The difference between the embodiment shown in and this embodiment is that instead of the actuator 10, a display unit 10a is provided, which is manipulated by the control system 40. For example, the neural network 60 can determine whether an object photographed by the optical sensor is suspicious, and the manipulation signal A can then be selected such that the object is highlighted in color by the display unit 10a.

[0064] Figure 7Schematically shows an exemplary configuration of a machine learning system 60. This configuration includes a front subsystem 61 and a rear subsystem 71. The front subsystem 61 and the rear subsystem 71 can be provided by a machine learning system, in particular each by a (in particular deep, that is, equipped with at least 2, 3, 5, 10, 15 or 20 hidden layers (English "hidden layers")) neural network. In the following, it is assumed that the front subsystem and the rear subsystem are provided by the first or second neural network. The input image x is fed to the first neural network 61, and based on this, the first neural network determines the corresponding per-pixel semantic segmentation SEM. This per-pixel semantic segmentation is fed to the second neural network 71, and based on this, the second neural network determines the output signal y. The machine learning system can be implemented as a computer program that is stored on a machine-readable storage medium 46 and is executed by a processor 45.

[0065] Figure 8 Shows a first aspect of a training device 140 for training a locator 60, that is, for training the front subsystem, here exemplarily the first neural network 61. The first neural network 61 is parameterized with parameters φ, which are provided by a parameter memory P.

[0066] A training image xs, other training images xt, a semantic segmentation SEMs corresponding to the semantic segmentation of the training image xs, and other semantic segmentations SEMt corresponding to the semantic segmentation of the other training images xt are provided to a texture exchanger 62. As illustrated in Figure 10 As illustrated in the figure, the texture exchanger 62 thereby determines a new training image xn. Of course, multiple training images, multiple other training images, and / or multiple new training images are also conceivable. The first neural network 61 determines a semantic segmentation SEM based on the new training image. This semantic segmentation SEM is either directly fed to a comparator 74 or (not shown) is further processed by the second neural network 71 into a further processed semantic segmentation and then fed to the comparator 74. Similarly, the provided semantic segmentations SEMs are provided to the comparator 74.

[0067] Based on the consistency between the semantic segmentation SEM and the provided semantic segmentations SEMs, the comparator determines new parameters φ' for parameterizing the first neural network 61. These new parameters are fed to the parameter memory P, where these new parameters replace the parameters φ. This can generally be achieved by determining a gradient that is used to minimize a pre-given cost function and backpropagation through the first neural network 61 (and, if necessary, through the second neural network 71).

[0068] The described method can be stored on a machine-readable storage medium 146 in a manner implementable as a computer program and implemented by a processor 145.

[0069] Figure 9 Schematically shows another aspect of a training device 140 for training a machine learning system 60 (as it can be performed especially after executing the method described in connection with Figure 8 The machine learning system 60 is parameterized using parameters φ, which are provided by a parameter memory P.

[0070] Additional input images xw and the associated nominal output signals yw are provided. The additional input images xw are fed to the machine learning system 60, which determines an output parameter y therefrom. The output parameter is fed to a comparator 74, just like the nominal output signal yw. Based on the degree of deviation between the output signal y and the nominal output signal yw, new parameters φ' for parameterizing the machine learning system are determined and these new parameters are fed to the parameter memory P, where these new parameters replace the parameters φ.

[0071] This can generally be achieved by minimizing a pre-given cost function.

[0072] The described method can be stored on a machine-readable storage medium 146 in a manner implementable as a computer program and implemented by a processor 145.

[0073] Figure 10 Schematically illustrates the flow of a method for generating new training images xn. Figure 10 a First describes a method for generating a pre-given texture xe. In the provided other semantic segmentation SEMt, a region be, for example a rectangular region be, is provided that does not intersect the segmentation boundary such that all pixels within this region are assigned a unique class. In the other training images xt corresponding to the other semantic segmentation SEMt, the region bex corresponding to this region be is identified (this can be given, for example, by pixel-by-pixel correspondence). Then, the texture xe is obtained as a copy of the pixel values of the other training images xt. It is readily understandable that: by changing the region be for each other training image xw, multiple textures xe can be provided. It is also readily understandable that: by providing multiple training images xw, even more textures xe can be provided.

[0074] Figure 10Figure b illustrates a method for assigning new textures according to the training image xs. This method uses the provided textures xe1, xe2, xe3. These textures are randomly assigned to the regions b1, b2, b3 of the provided semantic segmentations SEMs, that is, each region is exactly surrounded by a segmentation boundary. It is also possible that one of the regions of the provided semantic segmentations SEMs is not assigned a texture. Subsequently, in the training image xs, these regions are occupied by the provided textures xe1, xe2, xe3 assigned to them, for example, by tiling. If a region is not assigned a texture, the content of the training image xs within this region is adopted unchanged. The image thus obtained is the new training image xn.

[0075] Figure 11 Schematically shows a possible configuration of the robustness evaluator 141 for evaluating the robustness of the machine learning system 60. The configuration and function of the machine learning system are illustrated as in Figure 7 as shown. The input image x and the semantic segmentation SEM are fed to the texture exchanger 77, which determines another input image x2 by covering the provided textures (not shown) from the input image x using the method illustrated in Figure 10 b). This other input image is also fed to the machine learning system 60, and the machine learning system determines another output signal y2 based on this. This other output signal is fed to the robustness estimator 76 together with the output signal y, and the robustness estimator determines the robustness value b based on this. The robustness evaluator 141 can, for example, be integrated into the control system 40 (not shown), where the robustness value b can be fed to the modification unit 80 so that the modification unit selects the control signal A accordingly. Alternatively or additionally, the robustness evaluator 141 can also be integrated into the training system 140 (not shown), where the training of the machine learning system 60 is continued or ended based on the robustness value b.

[0076] The term "computer" includes any device for running pre-given calculation rules. These calculation rules can exist in the form of software, or in the form of hardware, or also in a hybrid form of software and hardware.

[0077] List of reference numerals

[0078] A Control signal

[0079] P Parameter memory

[0080] S Sensor signal

[0081] SEM Semantic segmentation

[0082] SEMs Provided semantic segmentations

[0083] Other semantic segmentations provided by SEMt

[0084] b robustness value

[0085] be region

[0086] bex region

[0087] x input image

[0088] x2 other input images

[0089] xe pre - given texture

[0090] xn new training image

[0091] xs training image

[0092] xt other training images

[0093] xw additional training image

[0094] y output signal

[0095] y2 other output signals

[0096] yw rated output signal

[0097] φ parameter

[0098] φ' new parameter

[0099] 10 actuator

[0100] 10a display unit

[0101] 11 production machine

[0102] 12 finished product

[0103] 20 surrounding environment

[0104] 30 sensor

[0105] 40 control system

[0106] 45 processor

[0107] 46 machine - readable storage medium

[0108] 50 receiving unit

[0109] 60 machine learning system, neural network

[0110] 61 front - subsystem, first neural network

[0111] 62 texture exchanger

[0112] 71 Post - subsystem, Second neural network

[0113] 74 Comparator

[0114] 76 Robustness estimator

[0115] 77 Texture exchanger

[0116] 80 Retrofit unit

[0117] 100 Motor vehicle

[0118] 140 Training device

[0119] 141 Robustness evaluator

[0120] 145 Processor

[0121] 146 Machine - readable storage medium

[0122] 200 Production system

[0123] 249 User

[0124] 250 Personal assistant

[0125] 300 Access system

[0126] 400 Monitoring system

[0127] 401 Door.

Claims

1. A method for training a machine learning system (60) to create a semantic segmentation and / or classification (y) of an input image (x) detected by means of an imaging sensor (30), wherein at least one training image (xs) is provided, the training image having an associated per-pixel semantic segmentation (SEMs), and wherein a new training image (xn) is generated based on the training image (xs) by covering at least one predefined region (b1, b2, b3) corresponding to a segmentation region of the provided semantic segmentation (SEMs) with a predefined texture (xe) in the training image (xs). Moreover, the machine learning system determines a semantic segmentation (SEM) for the new training image (xn), and a parameter (φ) for parameterizing the machine learning system is adapted based on the determined semantic segmentation (SEM) and the provided semantic segmentation (SEMs).

2. The method according to claim 1, wherein the machine learning system (60) is a neural network.

3. The method according to claim 1, wherein the predefined texture (xe) is cut from other provided training images (xt).

4. The method according to claim 3, wherein the predefined texture is cut from a region (bex) of the other provided training images (xt) that is completely within a single segmentation region belonging to another semantic segmentation (SEMt) of the other provided training images (xt).

5. The method according to any one of claims 1 to 4, wherein the predefined region is randomly selected.

6. The method according to claim 5, wherein the predefined region is randomly selected from a plurality of segmentation regions of the provided semantic segmentation (SEMs).

7. The method according to any one of claims 1 to 4, wherein the provided training images (xs) are generated according to a predefined list of geometric structures.

8. The method according to claim 7, wherein the provided training images (xs) are generated textureless according to a predefined list of geometric structures.

9. The method according to claim 7, wherein the machine learning system to be trained consists of a front subsystem (61) and a rear subsystem (71), and when adapting the parameter (φ) based on the determined semantic segmentation (SEM) and the provided semantic segmentation (SEMs), only the parameter (φ) parameterizing the front subsystem (61) is adapted.

10. The method according to claim 9, wherein if the new training image (xn) is fed to the machine learning system (60), the determined semantic segmentation (SEM) is provided at the output of the front subsystem (61).

11. The method according to claim 9, wherein after training the front subsystem (61), at least one additional training image (xw) captured by the imaging sensor (30) and the rated output signal (yw) assigned to one or more of the training images (xw) are provided to the machine learning system (60), wherein the machine learning system (60) provides an output signal (y) based on the at least one additional training image (xw) and the parameter (φ) for parameterizing the rear subsystem is adapted according to the consistency between the rated output signal (yw) and the output signal (y).

12. The method according to any one of claims 1 to 4, wherein a robustness value (b) characterizing the robustness of the machine learning system (60) is determined by using the method according to claim 14 and the training is continued or not continued according to the determined robustness value (b).

13. A training system (14) configured to execute the method according to any one of claims 1 to 12.

14. A method for determining the robustness of a machine learning system (60) for creating a semantic segmentation and / or classification (y) of an input image (x) detected by means of an imaging sensor (30), wherein a respective pixel-wise semantic segmentation (SEM) of the detected input image (x) is provided, wherein other input images (x2) are generated from the input image (x) by covering at least one pre-given region corresponding to a segmentation region of the provided semantic segmentation (SEM) with a pre-given texture in the other input images (x), and wherein the machine learning system determines an output signal (y) based on the detected input image (x) and a second output signal (y2) based on the generated other input images (x2), wherein a robustness value (b) characterizing the robustness is determined based on the determined output signal (y) and the determined second output signal (y2).

15. The method according to claim 14, wherein the machine learning system (60) is a neural network.

16. A method for providing a control signal (A) for controlling an actuator (10), wherein in a first stage, a machine learning system is trained by using the method according to any one of claims 1 to 12 and in a second stage, the control signal (A) is provided based on the semantic segmentation (y) determined by the machine learning system.

17. The method according to claim 16, wherein the robustness value (b) is determined by means of the method according to claim 14 or 15, and the control signal (A) is provided correspondingly according to the value of the robustness value (b).

18. A control system (40) configured to implement the method according to any one of claims 16 or 17.

19. A computer program product, the computer program product comprising a computer program which is configured to implement the method according to any one of claims 1 to 12 or 14 to 17.

20. A machine-readable storage medium (46, 146) which is configured to implement the method according to any one of claims 1 to 12 or 14 to 17.

Images