Method and apparatus for improving robustness against adversarial examples

By generating data signal interference to evaluate and enhance the robustness of the machine learning system, the problem of difficult to identify against sample attacks is solved, and the security and robustness of the system is improved, especially in the actuator control system.

CN108734289BActive Publication Date: 2024-11-01ROBERT BOSCH GMBH
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN201810348276.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Priority Date
2018-01-17
Filing Date
2018-04-18
Publication Date
2024-11-01
Estimated Expiration
2038-04-18

AI Technical Summary

Technical Problem

Existing machine learning systems have difficulty distinguishing semantic segmentation when facing manipulated data signals, making it difficult to identify adversarial sample attacks, which may lead to misleading and security threats from autonomous robots.

Method used

Confusing the machine learning system by generating data signal interference, simulating adversarial samples, evaluating the robustness of the system, and generating manipulated data signals, especially periodic interference through data signal interference, to suppress the effectiveness of the adversarial samples.

Benefits of technology

It effectively improves the robustness of machine learning systems for semantic segmentation, reduces the effectiveness of attacks against samples, and enhances the security of the system, especially in the executor control system, which can identify and defend against potential attacks.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN108734289B_ABST
    Figure CN108734289B_ABST
Patent Text Reader

Abstract

The present invention relates to a method and apparatus for improving robustness against adversarial examples. adv ) to confuse a first machine learning system (60), the first machine learning system being configured to determine semantic segments (y_cls) of a received one-dimensional or multi-dimensional data signal (x), the method comprising the following steps: a) determining a training data set (D train ), the training data set includes data signals (x (k) ) and the desired semantic segment to which it belongs (y target,k ) pairs, b) according to the training data set (D train ) data signal (x (k) ), the desired semantic segment (y target,k ) and the data signal (x (k) ) of the estimated semantic segmentation (f θ ) to generate the data signal interference (#imgabs2#).
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to a method for generating data signal interference and, based on this, to a method for generating a manipulated data signal, a method for evaluating the robustness of an actuator control system, a method for operating an actuator control system, a method for training an actuator control system, an actuator control system trained using the method, a computer program, a machine-readable storage medium and a computer, wherein the computer program includes instructions that are designed to carry out one of the methods when the computer program is executed on a computer, the computer program is stored on the machine-readable storage medium, and the computer is designed to carry out one of the methods. Background Art

[0002] A neural network for controlling a device is known from DE 10 2005 050 577 A1. The invention tests a neural network 1 for controlling a device. The neural network has a plurality of first neurons N1, N2, ..., Nn in a first layer and second neurons M in a second layer following the first layer. Each test signal combination is selected from a plurality of predefined test signal combinations. Each test signal combination assigns a test input signal vector ut1, ut2, ..., utk to each first neuron N1, N2, ..., Nn, which is either a zero signal or saturates the associated first neuron N1, N2, ..., Nn, so that the first neuron N1, N2, ..., Nn outputs a lower saturation value , or saturate the corresponding first neurons N1, N2, ..., Nn so that the first neurons N1, N2, ..., Nn output an upper saturation value. A test signal combination is applied to the first neurons N1, N2, ..., Nn and the output signal p of the second neuron M is detected. When the output signal p is greater than a predetermined threshold value, a partial test signal is stored. After applying each of the test signal combinations and when the partial test signal is not stored, a positive total test signal is output, wherein a plurality of predetermined test signal combinations are stored for the partial test signal.

[0003] Advantages of the invention

[0004] In contrast, the method having the features of independent claim 1 has the advantage that it makes it possible to render semantic segmentations determined by means of machine learning methods particularly robust against misleading examples (“adversarial examples” in English). Adversarial examples are slightly manipulated input data (which in the case of image data are similar to unmanipulated input data so that they are practically indistinguishable to a human expert) which can lead to significant changes in the determined semantic segmentations. It is conceivable, for example, that a malicious attacker uses such misleading examples in order to mislead an autonomous robot by, for example, suppressing semantic segmentations that label a flow that is actually present as a “flow”, which can lead to harming the autonomous robot which performs its route planning based on the semantic segmentations. The effectiveness of such attacks can be reduced using the method according to the invention.

[0005] Advantageous developments are the subject matter of the independent claims. Summary of the invention

[0006] In a first aspect, the invention relates to a method for generating a generic data signal disturbance in order to generate a manipulated data signal for confusing a first machine learning system, the first machine learning system being configured to determine semantic segments of a received one-dimensional or multi-dimensional data signal, the data signal characterizing states of an agent system, in particular an actuator system comprising an actuator (10) and an environment (20) of the actuator (10).

[0007] In this case, semantic segmentation of the data signal means, as is conventional, that a semantic value from a predefined selection of possible semantic values ​​is assigned to a region of the data signal.

[0008] The method comprises the following steps:

[0009] a) determining a training data set comprising pairs of data signals and associated desired semantic segments,

[0010] b) Generating a data signal interference as a function of the data signal of the training data set, the associated desired semantic segmentation and the estimated semantic segmentation of the data signal loaded with the data signal interference.

[0011] The data signal disturbance determined in this way is universal in the sense that a plurality of data signals not contained in the training data set can be transformed into misleading examples by means of the data signal disturbance (i.e., loaded with the data signal disturbance). If the semantic segmentation of the misleading examples is determined by means of the first machine learning system, a clear difference is obtained with respect to the corresponding semantic segmentation of the respective corresponding unloaded data signal according to the expected semantic segmentation.

[0012] The estimated semantic segmentation can in this case be determined by the first machine learning system or by a simulation of the first machine learning system.

[0013] The semantic segmentation is preferably performed “pixel by pixel” in this case, ie the semantic segmentation has the same dimensionality as the one-dimensional or multi-dimensional data signal and preferably assigns a semantic value to each data point in the one-dimensional or multi-dimensional data signal.

[0014] With such a data signal perturbation, in particular a manipulated data signal can be generated by applying the data signal perturbation to the data signal (e.g. by addition). By generating such a manipulated data signal, it is in particular possible to improve and reliably assess the robustness of the first machine learning system for semantic segmentation with respect to possible attackers. In particular, due to the independence of the received data signal and the data signal of the training data set, it is possible to detect a wide range of misleading examples with the help of the generated data signal perturbation.

[0015] In one refinement, it can be provided that the data signal disturbance is generated in such a way that it is periodic. In this way, an overmatching of the manipulated data signal to the training data set can be effectively suppressed. Otherwise, such an overmatching would be expected based on the dimensions of the data signal disturbance, which has the same dimensions as the data signal.

[0016] Alternatively or additionally, provision may be made for the desired semantic segmentation to be selected as a function of an estimated semantic segmentation of the associated data signal.

[0017] In this way, the desired semantic segmentation can be generated with particularly little effort, since no actual “true” semantic segmentation of the data signal of the training data set is necessary.

[0018] In a further development of this aspect, it can be provided that the desired semantic value is selected depending on whether the estimated semantic value assumes a predeterminable value. In this way, attacks can be particularly well simulated, which aim to suppress information about data points whose semantic values ​​have predeterminable values.

[0019] In this case, it can be provided that the expected semantic value at the location of the predefinable value is adopted instead of the estimated semantic value by a substitute value, which in this case is a value different from the estimated semantic value.

[0020] Advantageously, the replacement value is the estimated semantic value at a replacement position in the vicinity of the position. The replacement position can be, for example, the nearest position whose estimated semantic value differs from the predefinable value.

[0021] Furthermore, it can be provided that the expected semantic value is selected identically to the estimated semantic value if the estimated semantic value does not assume the predefinable value.

[0022] With each of the two measures mentioned above, it is possible to simulate an attacker who causes a particularly coordinated transition of suppressed data points into the semantic segmentation of the background and thus carries out a particularly targeted attack.

[0023] This can occur in the following way, that the data signal interference is generated according to the function value of the cost function of the first machine learning system, wherein the cost function adopts the function value in the case of a data signal loaded with the data signal interference and for the desired semantic segmentation. That is, the cost function usually has as an independent variable the output value of the first machine learning system (the determined semantic segmentation), which the first machine learning system determines as a function of its input signal (data signal). In addition, the cost function has as a common method the desired semantic segmentation (i.e. the desired semantic value) as a further independent variable. Here it is now provided that, as an input signal, the data signal is replaced by a data signal loaded with the data signal interference.

[0024] Specifically, the data signal interference can be selected from a set of allowed data signal interferences so that the data signal interference minimizes the cost function numerically for the desired semantic value when the data signal loaded with the data signal interference is loaded. Such numerical minimization can be performed, for example, by an iterative method, wherein the iterative method is terminated if a predeterminable termination criterion is reached (for example, after a predeterminable number of iterative steps). It is understood by those skilled in the art that in this case, the theoretically achievable minimum value will usually be reached inaccurately but only approximately.

[0025] In a flexible improvement, it can be provided that the cost function comprises two parts, one of the two parts representing a part of a position on the cost function whose estimated semantic value assumes a predefined value, and the other part representing a part of a position on the cost function whose estimated semantic value does not assume a predefined value, wherein the two parts are weighted relative to each other in a predefined manner.

[0026] In this way, it is possible in a particularly simple way to flexibly simulate not only attacks whose goal is to suppress information about data points whose semantic segments have predefined values ​​as efficiently as possible (which can be achieved by heavily weighting the first part), but also attacks whose goal is to leave the semantic background as intact as possible so as to make it particularly difficult to recognize (which can be achieved by heavily weighting the second part).

[0027] In another improved aspect, it is provided that the present invention is applied to a method for evaluating the robustness of an actuator control system, the actuator control system being used to control an actuator, the actuator control system comprising a first machine learning system, the first machine learning system being configured to determine semantic segments of a received one-dimensional or multi-dimensional data signal, the data signal characterizing a state of an actuator system including the actuator and an environment of the actuator, wherein the actuator control system is configured to determine a control signal based on the determined semantic segments and to control the actuator based on the determined control signal, wherein a manipulated data signal is generated based on the data signal and a data signal disturbance (wherein the data signal disturbance is determined using one of the previously mentioned methods), and a second control signal is determined based on a determined second semantic segment, the second semantic segment being determined by means of the first machine learning system by replacing the data signal with a manipulated data signal (i.e., the manipulated data signal is fed to the first machine learning system instead of the data signal). In this case, it is then determined based on the control signal and also based on the second control signal whether the actuator control system is robust.

[0028] In particular, such a method can also be run during operation of the actuator control system in order to assess continuously or at regular time intervals whether the actuator control system will behave robustly with respect to possible attacks.

[0029] Based on this, a method for operating an actuator control system can be provided, wherein the actuator control system is used to control an actuator, wherein an evaluation is performed according to the previously mentioned method as to whether the actuator control system is robust, and the actuator control system controls the actuator according to the evaluation, wherein if the evaluation shows that the actuator control system is not robust, the actuator control system can be switched, in particular, to a defense mode.

[0030] In yet another aspect, the present invention relates to a method for training an actuator control system, wherein the actuator control system comprises in particular a first machine learning system, which is configured to determine semantic segmentation of a received one-dimensional or multi-dimensional data signal, wherein the data signal represents a state of an agent system, in particular an actuator system including an actuator and an environment of the actuator, wherein the actuator control system is further configured to determine an output value of a second machine learning system, wherein the output value represents the probability that the semantic segmentation is erroneous (falsch), in particular whether the data signal is a misleading example, wherein the actuator can be controlled based on the determined semantic segmentation and, if an error is determined, the actuator can be controlled based on an error prevention reaction.

[0031] The training method comprises the following steps:

[0032] a) selecting a subset of data signals from a training set for the second machine learning system, the training set comprising data signals,

[0033] b) determining whether the data signal of the subset should be manipulated,

[0034] c) as long as it is determined that the data signal should be manipulated, determining a manipulated data signal as a function of the respective data signal and a data signal disturbance determined by one of the above-mentioned methods and replacing the data signal by the respective determined manipulated data signal,

[0035] d) if the data signal is not manipulated, setting the desired output value to the value of a predefinable first value, or if the data signal is manipulated, setting the desired output value to a second predefinable second value different from the first value, and

[0036] e) Training parameters of the second machine learning system using the (optionally manipulated) subset of observation values ​​and the associated desired output values.

[0037] In yet another aspect, the present invention relates to an actuator control system comprising a first machine learning system configured to determine semantic segmentation of a received one-dimensional or multi-dimensional data signal and a second machine learning system trained using the training method. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The embodiments of the present invention are described in more detail below with reference to the accompanying drawings. In the accompanying drawings:

[0039] Figure 1 schematically illustrates the interaction between an actuator and an actuator control system;

[0040] Figure 2 schematically illustrates the interaction between the training system and the actuator control system;

[0041] Figure 3 An implementation of the training method is shown;

[0042] Figure 4 An embodiment of a method for determining data signal interference is shown;

[0043] Figure 5 An embodiment of a method for determining a manipulated data signal is shown;

[0044] Figure 6 An embodiment of a method for evaluating the robustness of an actuator control system is shown;

[0045] Figure 7 An embodiment of a method for operating an actuator control system is shown. DETAILED DESCRIPTION

[0046] Figure 1 An actuator 10 is shown, which is in interaction with an actuator control system 40 in its environment 20. The actuator 10 and the environment 20 are also collectively referred to below as the actuator system. The state of the actuator system is detected by means of a sensor 30, which can also be given by a plurality of sensors. The output signal S of the sensor 30 is transmitted to the actuator control system 40. The actuator control system 40 determines therefrom an actuation signal A, wherein the actuator 10 receives the actuation signal.

[0047] The actuator 10 may be, for example, a (semi-)autonomous robot, such as a (semi-)autonomous motor vehicle. The sensor 30 may be, for example, one or more video sensors and / or one or more radar sensors and / or one or more ultrasound sensors and / or one or more position sensors (e.g. GPS). Alternatively or additionally, the sensor 30 may also include an information system that determines information about the state of the actuator system, such as a weather information system that determines the current or future state of the weather in the environment 20.

[0048] In another embodiment, the actuator 10 may be a manufacturing robot, and the sensor 30 is then, for example, an optical sensor that detects properties of a manufactured product of the manufacturing robot.

[0049] In another embodiment, the actuator 10 may be an authorization system, which is configured to authorize or not authorize the activity of the device. The sensor 30 may be, for example, an optical sensor (for example, for detecting image data or video data), which is configured to detect a face. The actuator 10 determines an authorization signal based on the control signal A, which can be used to authorize the device based on the value of the authorization signal. The device may be, for example, a physical or logical access control device. Based on the value of the control signal A, the access control device can then be set to grant or not grant access.

[0050] The actuator control system 40 receives the output signal S of the sensor in an optional receiving unit 50, which converts the output signal S into a data signal x (alternatively, the output signal S can also be directly taken over as data signal x). The data signal x can be, for example, a fragment or further processed output signal S. The output signal x is fed to a first machine learning system 60, for example a neural network.

[0051] In the preferred embodiment described below, the data signal x is a two-dimensional image signal, and the pixels of the image signal are represented by one or three values. However, the data signal x may also be other one-dimensional or multi-dimensional data signals.

[0052] The first machine learning system 60 determines the associated semantic segment y_cls from the image signal x. In this case, a semantic value is assigned to each range of the image signal x. In this embodiment, a pixel-by-pixel segmentation is performed, ie the semantic segment y_cls has the same dimensionality as the data signal x.

[0053] In this case, the semantic segment y_cls characterizes the probability that the pixel is classified as belonging to a semantic category (from a plurality of semantic categories). The semantic segment y_cls can, for example, be a vector-valued parameter for each pixel, which indicates, for each of the semantic categories, by means of an assigned number in the value interval [0; 1], how high the probability is that the pixel can be assigned to the corresponding semantic category. The semantic segment y_cls can also, for example, be a scalar parameter for each pixel, such as an identifier of the semantic category for which the above-mentioned probability assumes the maximum value.

[0054] The actuator control system 40 further comprises a second machine learning system 70, for example a neural network. The second machine learning system 70 also receives the image signal x and determines therefrom an output value y_det, which can be, for example, a number in the value range [0; 1] and can represent the probability that the observation value B has been manipulated so that the semantic segment y_cls incorrectly represents the image data x.

[0055] In this embodiment, this is achieved in that the second machine learning system 70 is set up such that the output value y_det characterizes the probability that the image signal x is a misleading example.

[0056] The semantic segment y_cls and the output value y_det are transmitted to an output unit 80, which determines therefrom the control signal A. For example, it is possible that the output unit first checks whether the output value y_det is less than a predefinable threshold value. If this is the case, the control signal A is determined based on the semantic segment y_cls. This is the normal situation. If, on the other hand, it is determined that the output value y_det is not less than a predefinable threshold value, it can be provided that the control signal A is constructed so that it causes the actuator A to switch to a defensive mode.

[0057] In one specific embodiment, the actuator control system 40 includes a computer and a machine-readable storage medium (not shown) on which a computer program is stored, which, when executed by the computer, causes the computer to implement the described functionality of the actuator control system 40. The first machine learning system 60 and the second machine learning system 70 can be implemented here in particular as separate or common computer programs.

[0058] Figure 2The interaction between the training system 90, the first machine learning system 60 and the second machine learning system 70 is illustrated. The training system 90 temporarily provides a training data set, which includes image signals x. One or more image signals x are selected from the training data set and are provided to the first machine learning system 60. This can be individual image signals x, i.e. such image signals which are also fed to the first machine learning system 60 when the actuator control system 40 interacts with the actuator 10 and the sensor 30. However, it can also be a batch, i.e. a plurality of such image data x.

[0059] The first machine learning system 60 determines the semantic segments y_cls from the image data x fed to it. Similarly, the second machine learning system 60 is similar to Figure 1 The output value y_det is determined in a certain way. The semantic segment y_cls and the output value y_det are in turn supplied to the training system 90. The training system 90 determines therefrom a parameter matching signal P, which encodes which parameters of the second machine learning system 70, such as their values, should be changed. The desired matching is performed, for example, by presetting the desired value for the output value y_det and backpropagation. For this purpose, the training system 90 supplies the parameter matching signal P to a matching block 95, which matches the parameters in the first machine learning system 60 and in the second machine learning system 70 accordingly. In this case, it is possible to match only the parameters of the first machine learning system 60 or only the parameters of the second machine learning system 70.

[0060] In one embodiment, the training system 90 includes a computer and a machine-readable storage medium (not shown) on which is stored a computer program that, when executed by the computer, causes the computer to implement the described functionality of the learning system 90 .

[0061] Figure 3 One specific embodiment of a method for training parameters of second machine learning system 70 by means of training system 90 is shown in a flow chart.

[0062] In this case, the parameters of the first machine learning system 60 are frozen and the parameters of the second machine learning system 60 are trained.

[0063] First ( 1110 ), for each image signal x supplied to first machine learning system 60 in a first training phase, it is determined whether the image signal has been manipulated. This can be done, for example, randomly with a predeterminable probability, for example 50%.

[0064] Next ( 1120 ), the desired second output value y_det of the image value x to be manipulated is set to the value “1”, otherwise to the value “0”.

[0065] In the following step 1130, a common data interference signal is provided . You can either read in the general data interference signal , or the general data interference signal may be generated . Used to generate a common data interference signal The method in Figure 4 The illustration is given in the figure.

[0066] Then (1140), the image value x to be manipulated is converted into its manipulated form x adv Instead, the generic data interference signal Add to x.

[0067] In step 1150 , which now follows, the trained parameters of the first machine learning system 60 are frozen and the parameters of the second machine learning system 70 are trained.

[0068] Using the described cost function J of the second machine learning system 70 det The parameters are trained using back propagation.

[0069] The second phase 1100 thus ends.

[0070] Figure 4 A flowchart is shown for generating a general data signal interference method.

[0071] In a first step 2000, an image signal x is provided. (k) The training data set D train , where k=l...m, with a fixed number m.

[0072] Next is step 2010, in which the image signal x (k) For each of the above, the associated semantic segment y_cls is determined with the aid of the first machine learning system 60 (instead of the first machine learning system 60, an equivalent system can be used). The semantic segment y_cls is also referred to below by the symbol f θ (x (k) ) represents, where θ represents the parameters of the first machine learning system 60, and f represents the imaging criterion, which is implemented by the machine learning system 60. Therefore, for each point (i, j), the semantic value is estimated The estimated semantic segment is also denoted by y pred To express.

[0073] In step 220, a predefinable class o is defined, ie o is a predefinable value which corresponds to one of the permissible semantic values. Now, the foreground set I o (k) Defined as the corresponding image signal x (k) The set of all the following points (i, j): that is, the estimated semantic value y of the point pred ij The predeterminable value o is adopted. In addition, the background set I is determined bg (k) , the background set includes all the following points: that is, the estimated semantic value y of the point pred ij The predeterminable value o is not used. (k) o and Background Set I bg (k) The resulting union is the same for every value of k and is also called the total set I.

[0074] In step 2030, for the image signal x (k) Each of the desired semantic segment y target,k , the semantic value y of the desired semantic segment target,k ij For the background set I bg (k) Each point (i, j) in is associated with the corresponding estimated semantic value y pred ij In addition, for the foreground set I o (k) For each point (i, j) in , we first determine the replacement value. (k) The alternative position (Ersatzstelle) (i', j') in

[0075] to be sure.

[0076] Then, as the replacement value, the semantic value at the replacement position (i', j') is selected. target,k For point (i, j), select the semantic value, i.e., y target,k ij =y pred i'j' .

[0077] Now (2040), the values ​​h, w are predefined. h, w are selected so that the image signal x (k) The height of each of is an integer multiple of h, and the image signal x (k) The width of each of them is an integer multiple of w, so that the image signal x (k) The dimensions of are Rh×Sw, where R and S are integers. (0) = 0 Initialize the prototype data signal interference ξ with a dimension of h×w points (0) . The number n is set to the value n=0.

[0078] Then (2050), calculate the parameter

[0079] .

[0080] In this case, [r,s] means according to

[0081]

[0082] A collection of points.

[0083] x (k) [r,s] Represents the image signal x (k) The corresponding fragment, y target,k [r,s] Indicates the desired semantic segment y target,k The corresponding fragment.

[0084] Parameters Indicates based on

[0085]

[0086] The weighted cost function of the first machine learning system (60) is: cls is the cost function of the first machine learning system 60, such as cross entropy.

[0087] The variable ω is a predefinable variable which can be predefined in the value range [0; 1], for example to a value of 0.5.

[0088] In step 2050, the (weighted) cost function is thus determined The gradient of averaging over the training data set.

[0089] Next (2060), according to the formula

[0090]

[0091] Calculate the prototype data signal interference ξ (n+1) In this case, the function Clipɛ (x) Normalizes the values ​​of the variable x to the ɛ-sphere around the origin. The norm in this case can be L 2 Standard, or it can be L ∞ Standardization. α is set to a predeterminable value, for example α = 0.25. The value of the variable n is incremented by 1.

[0092] Now (2070) it is checked whether the convergence criterion is met. For example, it can be checked whether n exceeds a predeterminable limit value nmax, for example nmax=10. If this is not the case, branch back to step 2050. Otherwise, in step 2080, the prototype data signal interference ξ is generated. (n) (Among them, for i <h,j<w, ) and periodic edge conditions Iteratively determine the data signal interference The method thus ends.

[0093] Figure 5 A flow chart showing the method for determining the manipulated data signal x adv In a first step 3000, a data signal x is received. In a next step 3010, optionally by means of Figure 4 The method illustrated in the figure determines the data signal interference Alternatively, the data signal interference may have been detected at an earlier time by means of Figure 4 The method illustrated in FIG. 3 is determined, stored in a storage medium, and is now called in step 3010. Next (3020), as the data signal x and the data signal interference The manipulated data signal x is determined by adding adv The method ends here. The method can be implemented as a computer program, which can be stored on a storage medium of the actuator control system 40 and / or the training system 90 or implemented by the actuator control system and / or the training system. The computer program can also be stored on a general computer and can be implemented by the computer.

[0094] Figure 6 An exemplary embodiment of a method for evaluating the robustness of an actuator control system 40 is shown in a flow chart. In a first step 4000, a data signal x is received. Subsequently (4010), the actuator control system 40 determines therefrom an associated control signal A in order to control the actuator 10. Then (4020), using the Figure 5 The method illustrated in the figure determines the manipulated data signal x from the data signal x adv Similarly, the manipulated data signal x advis fed to the first machine learning system 60 and determines the second semantic segment y_cls adv Similar to the determination of the control signal A, from the determined second semantic segment y_cls adv The second control signal A is determined adv Now (4030), the control signal A and the second control signal A are determined. adv and checks (4040) whether the magnitude of the difference (Betrag) exceeds a predefinable limit value. If this is the case (4050), it is determined that the actuator control system 40 is not robust, otherwise (4060) it is determined that the actuator control system 40 is robust. The method ends here. The method can be implemented as a computer program, can be stored on a storage medium of the actuator control system 40 or can be implemented by the actuator control system. The computer program can also be stored on a general computer and can be implemented by the computer, which can be coupled to the actuator control system 40.

[0095] Figure 7 An exemplary embodiment of a method for operating an actuator control system is shown in a flow chart. In a first step 5000, a Figure 6 The method illustrated in the figure determines whether the actuator control system 40 is robust. Then in step 5010, it is checked whether the actuator control system 40 is determined to be robust. If this is the case (5020), the actuator control system 40 is operated normally and the actuator 10 is actuated normally. If this is not the case (5030), it is checked whether the actuator control system 40 is switched to a defensive mode, for example, by limiting the absolute value of the actuation signal A to a value less than a critical limit. The method ends here. The method can be implemented as a computer program, can be stored on a storage medium of the actuator control system 40 or can be implemented by the actuator control system.

Claims

1. A method for generating a universal data signal interference Ξ to generate a manipulated data signal x adv A method for deceiving a first machine learning system (60), the first machine learning system being configured to determine a semantic segmentation y_cls of a received one-dimensional or multi-dimensional data signal x, the method comprising the following steps: a) Determine the training data set D train , the training data set includes data signal x (k) and the desired semantic segment y target,k Yes, b) Based on the training data set D train The data signal x (k) , the desired semantic segment y target,k and a data signal x loaded with data signal interference Ξ (k) The estimated semantic segmentation f θ To generate the data signal interference Ξ, wherein, According to the corresponding image signal x (k) The estimated semantic value y at position (i, j) in pred ij Whether to use the predefined value o to select the desired semantic value y target,k ij , wherein the estimated semantic value y is replaced by a replacement value pred ij The desired semantic value y at the position (i, j) at which the predeterminable value o is adopted target,k ij , wherein, according to the weighted cost function J of the first machine learning system (60) ss ω The data signal x loaded with the data signal interference Ξ (k) For the desired semantic segment y target,k The function value used is used to generate the data signal interference Ξ, wherein the cost function J ω ss is based on , wherein θ represents a parameter of the first machine learning system (60), and f represents an imaging criterion, wherein the foreground set I o Defined as the corresponding image signal x (k) The set of all the following points (i, j): that is, the estimated semantic value y of the point pred ij Using the predeterminable value o, I bg is defined as a background set, which includes all the following points: that is, the estimated semantic value y of the point pred ij The predeterminable value o is not adopted, and the foreground set I o and the background set I bg The union of the components is defined as the total set I, where J cls is the cost function of the first machine learning system (60), and ω is a predeterminable parameter.

2. The method according to claim 1, wherein: The data signal interference Ξ is generated so that the data signal interference Ξ is periodic.

3. The method according to claim 1, wherein: The replacement value is the estimated semantic value y of the replacement position (I', j') pred i'j' , the alternative position is near the position (i, j).

4. The method according to claim 1, wherein: When the estimated semantic value y pred ij When the predeterminable value o is not used, the estimated semantic value y pred ij Similarly, select the desired semantic value y target,k ij .

5. The method according to claim 1, wherein: The data signal interference Ξ is selected from a set of allowed data signal interferences Ξ, so that the data signal interference Ξ is added to the data signal x loaded with the data signal interference Ξ. (k) For the desired semantic value y target,k Let the cost function J ss ω Minimize numerically.

6. The method according to claim 1 or 5, wherein: The cost function J ss ω It includes two parts, one of which represents the cost function J. ss ω Its estimated semantic value y pred ij A portion of the position (i, j) of the predeterminable value o is adopted, and another portion characterizes the cost function J ss ω Its estimated semantic value y pred ij A portion of the position (i, j) of the predeterminable value o is not used, wherein the two portions are weighted relative to one another in a predeterminable manner.

7. A method for generating a manipulated data signal x adv A method for confusing a first machine learning system (60) configured to determine a semantic segmentation y_cls of a received one-dimensional or multi-dimensional data signal x, wherein Generate a common data signal disturbance Ξ using the method according to any one of claims 1 to 6, and generate the manipulated data signal x according to the data signal x and the generated data signal disturbance Ξ adv .

8. A method for evaluating the robustness of an actuator control system (40), the actuator control system being used to control an actuator (10), the actuator control system comprising a first machine learning system (60), the first machine learning system being configured to determine a semantic segment y_cls of a received one-dimensional or multi-dimensional data signal x, the data signal characterizing a state of the actuator system including the actuator (10) and an environment (20) of the actuator (10), in, The actuator control system (40) is configured to determine an actuation signal A as a function of the determined semantic segment y_cls and to actuate the actuator (10) as a function of the determined actuation signal A. wherein the manipulated data signal x is generated based on the data signal x and the data signal interference Ξ determined by the method according to any one of claims 1 to 6 adv , and according to the determined second semantic segment y_cls adv Determine the second control signal A adv The second semantic segmentation is performed by the first machine learning system (60) when the manipulated data signal x is adv When the data signal x is replaced, According to the control signal A and the second control signal A adv It is determined whether the actuator control system (40) is robust.

9. A method for operating an actuator control system (40) for controlling an actuator (10), wherein: By means of the method according to claim 8, it is evaluated whether the actuator control system (40) is robust, and the actuator control system (40) controls the actuator (10) according to the evaluation, wherein if the evaluation shows that the actuator control system (40) is not robust, the actuator control system (40) is switched to a defensive mode.

10. A method for training an actuator control system (40), the actuator control system comprising a first machine learning system (60), the first machine learning system being configured to determine a semantic segmentation y_cls of a received one-dimensional or multi-dimensional data signal x, wherein: The actuator control system (40) is further configured to determine an output value y_det of a second machine learning system (70), the method comprising the following steps: a) selecting a subset of data signals x from a training set for the second machine learning system (70), the training set comprising the data signals x, b) determining whether the data signal x of the subset should be manipulated, c) determining the manipulated data signal x as a function of the respective data signal x and of the data signal disturbance Ξ determined by the method according to any one of claims 1 to 6, if it is determined that the data signal x is to be manipulated adv and through the respective determined manipulated data signal x adv To replace the data signal x, d) setting the desired output value to the value of a predefinable first value if the data signal x is not manipulated, or setting the desired output value to a second predefinable second value different from the first value if the data signal x is manipulated, and e) Using data signals x, x adv The parameters of the second machine learning system (70) are trained using the manipulated subset and the corresponding desired output values. 11 . A machine-readable storage medium having a computer program stored thereon, wherein the computer program is designed to carry out the method according to claim 1 . 12 . A computer configured to carry out the method according to claim 1 .

13. An actuator control system (40), comprising a first machine learning system (60) and a second machine learning system (70), wherein the first machine learning system is configured to determine a semantic segment y_cls of a received one-dimensional or multi-dimensional data signal x, and the second machine learning system is trained using the method for training an actuator control system (40) according to claim 10.