DEFENSE GENERATOR, METHOD FOR PREVENTING AN ATTACK ON AN AI UNIT AND COMPUTER-READABLE STORAGE MEDIUM

Abstract

The present invention relates to an attack-deterrent generator (20) for dynamically generating at least one AI defense module (16). A core of the invention lies in the fact that a distribution function is determined for model data. The assumption of the invention is that the model data belong to a multiplicity of models or have a similar statistic behavior. It is thus possible to determine for an input data set whether the data of the input data set is to be associated with an adversarial attack. This is for example the case when statistic anomalies are determined in the input data set.

Description

[0001] Defense generator, method for preventing an attack on an AI unit and computer-readable storage medium

[0002] DESCRIPTION

[0003] The invention relates to a defense generator for the dynamic generation of at least one AI defense module, a defense system against an adversarial attack on an AI unit, a method for the dynamic generation of AI defense modules, a method for preventing an attack on an AI unit and a computer-readable storage medium.

[0004] Most state-of-the-art machine learning methods are susceptible to adversarial perturbations. Therefore, robustness against adversarial perturbations is a major challenge in the development of machine learning methods.

[0005] An adversarial error occurs when data to be classified has been altered in such a way that the change is not noticeable to a human observer, but correct classification by an AI unit is no longer possible. This can lead to multiple misclassifications.

[0006] In the context of image classification, for example in the segmentation of image data, adversarial interference can be caused by superimposing noise onto the input images. This noise can be inserted into the input image in such a way that it is imperceptible to the human observer.

[0007] Adversarial interference, however, does not occur in the natural environment, i.e., in commonly available sensor data. One way in which adversarial interference can occur is through an attack on an AI unit. This means that an attacker modifies the data passed to the AI ​​unit for classification in such a way that it cannot be classified correctly. This poses significant safety risks, particularly with regard to safety-critical applications such as highly automated driving. For example, if a highly automated driving system misclassifies data, a vehicle might fail to recognize a stop sign or misinterpret it. It is also possible that a vehicle might not stop at a red light, thus posing a significant danger to other road users.

[0008] One goal is therefore to reduce the risk of adversarial attacks. While there are some isolated solutions, these are reactive and tailored to a specific attack. This means that, so far, a countermeasure can only be developed once the attack has occurred.

[0009] However, it is known that there is an unlimited number of possible attacks on AI units using adversarial jamming.

[0010] It is therefore an object of the invention to provide a means of protecting AI units from adversarial attacks. In particular, it is an object of the invention to identify an adversarial attack. It is further, in particular, an object of the invention to design an AI unit to be robust against adversarial attacks. It is also, in particular, an object of the invention to provide a means of generating an unlimited number of defense modules for AI units.

[0011] This problem is solved by a defense generator for the dynamic generation of at least one AI defense module according to claim 1, a defense system against an adversarial attack on an AI unit according to claim 10, a method for the dynamic generation of AI defense modules according to claim 11, a method for preventing an attack on an AI unit according to claim 12 and a computer-readable storage medium according to claim 14.

[0012] In particular, the problem is solved by a defense generator for the dynamic generation of at least one AI defense module, comprising the following: a tile unit configured to determine at least one tile for model data, wherein the model data are assigned to an AI unit and the at least one tile specifies at least a subset of the model data; an aggregation unit configured to determine aggregated data, wherein the aggregated data assign at least one, in particular mathematical, characteristic to the at least one tile; a distribution unit configured to determine a distribution function for the aggregated data; an inference unit configured to determine at least one inference configuration using the distribution function;a data transformation unit trained to generate at least one AI defense module for the AI ​​unit using the at least one inference configuration, wherein the at least one AI defense module is trained to determine, for an input data set of the AI ​​unit: whether an attack on the AI ​​unit can be attributed to the input data set; and / or, using a data transformation, a second

[0013] To determine an input data set that cannot be attributed to an attack on the AI ​​unit.

[0014] A key aspect of the invention is the determination of a distribution function for model data. The invention is based on the assumption that the model data belongs to a model manifold. In other words, that the model data exhibits similar statistical behavior. Thus, it is possible to determine, for an input dataset comprising, for example, images from an RGB camera, whether the data in the input dataset can be attributed to an adversarial attack. This is the case, for example, if statistical anomalies are observed in the input dataset.

[0015] Furthermore, the invention also includes the possibility of determining a second input data set for the input data set, which exhibits no statistical anomalies and therefore cannot be attributed to any attack on the AI ​​unit. Thus, the noise introduced into the original data by the adversarial attack can be filtered out again.

[0016] The invention is not limited to a specific type of AI unit. Thus, the AI ​​unit can be an artificial neural network, a Bayesian classifier, a Markov logic unit, or any other type of classifier. Overall, the invention provides a means of generating countermeasures against any adversarial attacks. These countermeasures are implemented by the defense module for the AI ​​unit. The defense module for the AI ​​unit can therefore be executable program code. This program code can be in compiled or uncompiled form. It is also conceivable that the defense module for the AI ​​unit is used as an add-on for the AI ​​unit.Thus, prior to a classification or regression task, the AI ​​unit can use the defense module to determine whether there is an attack on the AI ​​unit and neutralize it through data transformation.

[0017] The model data can, for example, be in the form of a tensor. The at least one tile can be a subset of the tensor. For example, the at least one tile can represent a one-dimensional tensor that encompasses a subset of one dimension of the tensor in the model data. The model data can be one-dimensional or multi-dimensional.

[0018] In one embodiment, the model data can be formed as an image with at least one color channel, in particular three color channels, preferably red-green-blue.

[0019] If the model data is formatted as an image, then at least one tile can specify a section of the image from a given color channel.

[0020] The described embodiment exploits local dependencies. This allows model data to be divided into a multitude of tiles, which together form a tile set.

[0021] For example, dependencies exist between the values ​​of closely spaced pixels in an image. If the distance between two pixels is very large, there is almost no dependency. Similarly, there is a strong dependency between letters of a word. In contrast, there is hardly any dependency between letters of different words or even different paragraphs. Thus, it is possible to process different tiles of the input data separately. This enables parallelization and therefore more efficient execution. In one embodiment, the tiling unit can be configured to receive a tile specification, wherein the tile specification can be user-definable, preferably specifying a kernel size, a stride, and / or an offset, and wherein the tiling unit can be configured to determine the at least one tile using the tile specification.

[0022] The kernel size can specify the size of at least one tile; in particular, the kernel size can include a value for the x-axis, a value for the y-axis, and / or a value for the z-axis. The stride, or step size, can specify the interval at which a new tile is generated when scanning the input dataset. The offset can be defined such that an offset of the step size does not generate a tile smaller than the other tiles. This could be the case, for example, if an input image with a size of 10 x 10 pixels is scanned with a kernel of size 3 x 3 pixels for which a step size of three pixels is defined.

[0023] In this case, an offset of two pixels is necessary.

[0024] In one embodiment, the model data can specify image data, audio data, sensor data, text data and / or at least one feature detector of an artificial neural network.

[0025] The model data can encompass a wide variety of data. In addition to the image data already mentioned, speech data, sensor data, or text data are also possible. However, it is also possible to use parts of an AI unit as model data instead of the data to be classified.

[0026] It has been shown that adversarial disturbances lead to unusual patterns in the activation functions of an artificial neural network.

[0027] Thus, in one embodiment, it is provided that feature detectors of an artificial neural network, in particular a Convolutional Neural Network, can form the model data.

[0028] In one embodiment, the aggregation unit can be configured to receive an aggregation specification, in particular a user-definable one, wherein the aggregation specification and / or the at least one identifier can each specify a singular value decomposition, a convolution, a mean, a median and / or a variance for the at least one tile, and wherein the aggregation unit can be configured to determine the aggregated data taking into account the aggregation specification.

[0029] With the described embodiment, it is possible for the aggregation unit to be configured to convolve the data of the input data set, e.g., with a filter, in order to determine the aggregated data. This can be implemented by a discrete Fourier transform followed by multiplication and subsequent integration, such that the convolution corresponds to a multiplication followed by integration in the frequency spectrum.

[0030] In one embodiment, the aggregation unit can be configured to perform a singular value decomposition for the at least one tile. In such cases, where a tile has more than one dimension, the tile can be represented as a block matrix, with each block of the block matrix corresponding to one dimension of the at least one tile.

[0031] It has been shown that adversarial perturbations generate unusual patterns, particularly in the lower singular values ​​of a matrix. Therefore, an adversarial perturbation in the model data can be detected by singular value decomposition.

[0032] In one embodiment, it is also possible to determine the mean, median, and / or variance for the at least one tile. In this case, the aggregated data can be represented as a vector of statistical values.

[0033] In one embodiment, the aggregation unit can be configured to receive a tile reference from the tile unit, wherein the aggregation unit can be configured to subtract the tile reference from the at least one tile in order to determine the aggregated data.

[0034] The at least one tile can represent a multitude of values. For example, a tile can represent a multitude of due date values. It is also possible for a tile to represent a multitude of letters, thus representing a text passage. In the embodiment described above, it is also conceivable to subtract a median, variance, or mean from the at least one tile. If more than one tile is provided, then subtracting the aforementioned values ​​from a tile can mean that, if the at least one tile is configured as a tensor, the mean and / or median is subtracted from each data element of the tensor across all tiles / tensors. The median, variance, and / or mean can be determined across all data elements of the at least one tile or tensor, or across all tiles / tensors.

[0035] Subtracting statistical values ​​has the advantage of making tiles from different regions comparable.

[0036] In one embodiment, the defense generator can be configured to receive a target definition, in particular a user-definable one, and the inference unit can be configured to determine the inference configuration taking into account the target definition.

[0037] The data transformation unit can be trained to select, taking into account the inference configuration, whether to determine if an attack on the AI ​​unit can be attributed to an input data set, or to perform a determination using the data transformation of a second input data set to which no attack on the AI ​​unit can be attributed.

[0038] Defining objectives thus makes it possible to control the behavior of the AI ​​defense modules. This provides a simpler way for users to define the behavior of the AI ​​defense modules.

[0039] In one embodiment, the distribution unit can be configured to receive a distribution specification, in particular a user-definable one, and the distribution unit can further be configured to determine the distribution function taking into account the distribution specification, wherein the distribution specification can be an explicit or implicit distribution function.

[0040] It is therefore possible that the distribution specification indicates a calculation method for a probability distribution, so that the data distribution itself represents a probability distribution. This distribution specification can be either implicit or explicit.

[0041] With an explicit distribution function, a transition matrix can be determined. The transition matrix can, for example, represent a Markov chain. It is also conceivable that a Gibbs distribution can be determined by an explicit distribution function.

[0042] The matrix elements of a transition matrix can each correspond to a data point in the aggregated data. A transition matrix can specify a probability or a probability distribution for each matrix element that the corresponding matrix element will assume a certain value. For example, if the input data set contains image data, each pixel could be assigned a probability that a neighboring pixel has a specific brightness value.

[0043] An implicit distribution function can be implemented, for example, using quilts or natural aggregation. In the quilt method, a distance between the input data and the model data is determined. For example, the L 2 A distance metric or another metric can be used to determine the distance. If the distance is greater than a predefined threshold, then an attack on the AI ​​unit can be inferred. Furthermore, it is possible to replace a tile of the input data, based on the metric used, with the nearest tile of the model data to determine the second input data set. In this case, the model data no longer exhibits adversarial interference, allowing an unaffected tile in the input data to be replaced.

[0044] In one embodiment, the inference unit can be configured to receive at least one threshold, particularly a user-definable one, and the determination of the inference configuration can be performed taking into account the at least one threshold. The at least one threshold can specify that, if the at least one threshold is exceeded, a data transformation is performed by the AI ​​defense module. In one embodiment, the threshold can, for example, be compared to a distance, as described above. This means that the AI ​​defense module can be configured to compare the threshold to a distance between input data or a tile (which can represent a subset of the input data) and the at least one tile.

[0045] In one embodiment, the AI ​​defense module can further be configured to perform data transformation based on the comparison, e.g., when the distance is greater than the threshold.

[0046] In one embodiment, the AI ​​defense unit can be trained to determine, based on the comparison, whether an attack on the AI ​​unit can be attributed to the input data set, e.g., if the distance is greater than the threshold.

[0047] Besides distance, it's also conceivable that the AI ​​defense unit is trained to determine a so-called Overall Probability Value for the model data. The comparison can then be performed using the Overall Probability Value.

[0048] In one embodiment, the AI ​​defense unit for generating the second input data can be configured to perform a smoothing of the input data or of the at least one tile based on the comparison. Smoothing the input data or of the at least one tile can result in the adversarial disturbance no longer having any effect on the AI ​​unit.

[0049] In one embodiment, the inference unit can be configured to receive a reconstruction specification, wherein the inference unit can be configured to determine the inference configuration taking into account the reconstruction specification, wherein the reconstruction specification can indicate whether a data transformation affects the entire model data, a random subset of the model data and / or an importance-based selection of the model data.

[0050] With the described embodiment, it is therefore also possible to reconstruct only a portion of the input data. This allows the device to operate more efficiently overall. Errors can also be avoided if only a portion of the input data needs to be considered. For example, in one embodiment, the AI ​​defense unit can be configured to divide input data, as described above, into at least one tile. During reconstruction, only specific tiles can be considered. For instance, with image data, individual image areas can be examined separately, so that key figures, particularly mathematical ones, are calculated for each of these image areas by the AI ​​defense unit. This embodiment takes into account the fact that, for example, a sky with a homogeneous blue color has different properties in its mathematical-visual average than, for example, an image area showing a street. The image areas, or rather...The tiles corresponding to these image areas can be statically defined. It is also conceivable that the image areas or tiles are dynamically determined by the AI ​​defense unit, taking image properties into account.

[0051] In one embodiment, the second input data set exhibits no adversarial interference. This means that an AI unit functions as intended with the second input data.

[0052] In one embodiment, the data transformation can be designed as a sampling method.

[0053] Using a sampling method, it is possible to determine the second input data set so that no adversarial disturbance is present in the second input data set.

[0054] The problem is further solved in particular by a defense system against an adversarial attack on an AI unit, comprising: an input unit trained to receive input data and / or an input model as model data; a defense generator, in particular as described above, trained to receive the model data and generate at least one AI defense module; an AI unit trained to use the at least one AI defense module before performing a regression and / or classification to determine whether an input data set of the AI ​​unit can be associated with an attack on the AI ​​unit; and / or, using a data transformation, to generate a second

[0055] to determine the input data set to which no attack on the AI ​​unit can be attributed and to use the second input data set in the regression and / or classification.

[0056] It is therefore also planned that at least one AI defense module generated by the defense generator will be used to defend against an attack on an AI unit within a defense system. This will efficiently and securely prevent attacks on AI units.

[0057] The task is also solved in particular by a method for the dynamic generation of AI defense modules, comprising the following steps:

[0058] • Determining at least one tile for model data, where the model data is assigned to an AI unit and the at least one tile specifies at least a subset of the model data;

[0059] • Determining aggregated data, whereby key figures, in particular mathematical ones, are assigned to the aggregated data of at least one tile;

[0060] • Determining a distribution function for the aggregated data;

[0061] • Determine at least one inference configuration using the distribution function;

[0062] • Generating at least one AI defense module for the AI ​​unit using the at least one inference configuration, wherein the at least one AI defense module is specifically configured to: o determine for an input data set of the AI ​​unit whether an attack on the AI ​​unit can be attributed to the input data set; and / or o generate a second AI defense module using a data transformation.

[0063] To determine an input data set that cannot be attributed to an attack on the AI ​​unit.

[0064] In one embodiment, the method can include receiving a tile specification, particularly a user-defined one, wherein the tile specification can preferably specify a kernel size, a stride, and / or an offset. The determination of the at least one tile can then be carried out using the tile specification. The kernel size can be chosen taking into account the functionality of an AI unit. For example, the kernel size can be chosen such that the functionality of the AI ​​unit is not impaired by replacing a tile in the input data of the AI ​​unit with an AI defense module.

[0065] In one embodiment, the method may include receiving an aggregation specification, wherein the aggregation specification may specify at least one identifier for the at least one tile, wherein the at least one identifier may each specify a singular value decomposition, a convolution, a mean, a median and / or a variance for the at least one tile.

[0066] In one embodiment, determining the aggregated data may involve subtracting one or more mathematical identifiers from the at least one tile.

[0067] In one embodiment, the method may include:

[0068] Receiving a target definition, especially a user-defined one;

[0069] Determining the inference configuration, taking into account the target definition.

[0070] In one embodiment, the method may include receiving an aggregation specification, in particular a user-defined one, wherein the at least one identifier and / or the aggregation specification may each specify a singular value decomposition, a convolution, a mean, a median and / or a variance for the at least one tile, wherein the determination of the aggregated data may be carried out taking into account the aggregation specification.

[0071] In one embodiment, determining the aggregated data may involve subtracting one or more of the aggregation specifications from the at least one tile.

[0072] In one embodiment, the method can include receiving a target definition, in particular a user-defined one, and determining the inference configuration taking the target definition into account.

[0073] Furthermore, the procedure may include selections taking into account the inference configuration, in particular whether a determination is made whether the input data set can be associated with an attack on the AI ​​unit and / or whether a determination is made using the data transformation of a second input data set to which no attack on the AI ​​unit can be associated.

[0074] In one embodiment, the method can include receiving a distribution specification, particularly a user-defined one, which specifies an explicit or implicit distribution function. The determination of the distribution function can then be performed taking the distribution specification into account.

[0075] In one embodiment, the method may include receiving at least one, in particular user-defined, threshold value, wherein the at least one threshold value may indicate that, upon exceeding the at least one threshold value, a data transformation is performed by the AI ​​defense module.

[0076] In one embodiment, the method can include receiving a reconstruction specification, particularly a user-defined one, which can indicate whether a data transformation affects all model data, a random subset of the model data, and / or an importance-based selection of the model data. The determination of the inference configuration can then be performed taking the reconstruction specification into account.

[0077] In one embodiment, the data transformation can be designed as a sampling method.

[0078] The problem is further solved in particular by a method for preventing an attack on an AI unit, comprising the following steps: generating at least one AI defense module, in particular according to a method as described above, in particular using a defense generator as described above; determining, using the at least one AI defense module, whether an input data record of the AI ​​unit can be attributed to an attack on the AI ​​unit; and / or

[0079] Determine, using at least one AI defense module, a second input data set that cannot be associated with an attack on the AI ​​unit, and use the second input data set in a regression and / or classification.

[0080] In one embodiment, the method may include:

[0081] Receiving a target definition, especially a user-defined one;

[0082] Determining the inference configuration, taking into account the target definition;

[0083] Selections, taking into account the target definition, whether o a determination of whether an attack on the AI ​​unit can be attributed to the input data set; and / or o a determination is carried out using the data transformation of a second input data set to which no attack on the AI ​​unit can be attributed.

[0084] The problem is further solved in particular by a computer-readable storage medium which contains instructions that cause at least one processor to implement a method as described above when the instructions are executed by the at least one processor.

[0085] The advantages are similar or identical to those already described in connection with the above-described method, the above-described defense generator and the above-described system.

[0086] It is hereby expressly pointed out that all process steps described in connection with the system are also disclosed as essential to the invention with regard to the process and the computer-readable storage medium (and vice versa). In particular, details described with regard to the defense generator and / or the system are also applicable to the process and the computer-readable storage medium, and vice versa.

[0087] Further embodiments are described in the dependent claims.

[0088] The invention will now be explained in more detail using exemplary embodiments. These will show:

[0089] Figure 1: a representation of the principles of classification of an example image;

[0090] Figure 2: a schematic representation of the superimposition of the example image with an adversarial disturbance;

[0091] Figure 3: a representation of the principles of the classification of the example image with an adversarial disturbance;

[0092] Figure 4: a schematic representation of the AI ​​defense system;

[0093] Figure 5: a schematic representation of an artificial neural

[0094] network;

[0095] Figure 6: a schematic representation of the subdivision of a

[0096] Opening image;

[0097] Figure 7: a schematic representation of the subdivision of a

[0098] Entrance image divided into different tiles; and

[0099] Figure 8: a schematic representation of the determination of a

[0100] Probability distribution for a large number of tiles.

[0101] Figure 1 shows a schematic representation of the classification of an input image 1. In the embodiment shown in Figure 1, the input image 1 is a schematic representation of a stop sign. The input image 1 could therefore have been captured, for example, by a vehicle's camera system, such as in highly automated driving. To enable highly automated driving, it is necessary to classify the individual components of the input image 1. This means that each pixel must be assigned whether it belongs to a specific object, which object it represents, or whether it shows the background, for example, the sky.

[0102] In the illustrated embodiment, four simplified classes 3, 3', 3", 3'" are possible for each pixel of the input image 1. A classification unit 2 is provided for classifying the pixels; this unit can, for example, be implemented as an artificial neural network. However, it is also conceivable that the classification unit 2 is implemented as a support vector machine or as a Bayesian network.

[0103] Classification unit 2 is trained for a specific classification task during a training phase. During training, a cost function is minimized using a training dataset. This dataset includes sample images and their corresponding classifications. The training dataset thus represents the so-called ground truth.

[0104] Normally, a classification unit 2 can perform image segmentation or classification of individual pixels with corresponding reliability.

[0105] Figure 2 illustrates the case of an adversarial interference. An input image 1 is superimposed with an attack image 4. The result of the superimposition is a distorted input image 5. The distorted input image 5 is indistinguishable from the input image 1 to the human observer.

[0106] Figure 3 illustrates the behavior of classification unit 2 when the distorted input image 5 is to be classified. Classification unit 2 will not assign the distorted input image 5 to class 3' as in Figure 1, but rather to class 3'". This results in an incorrect classification.

[0107] It is therefore possible to deliberately interfere with classification units 2 without a human user being able to recognize the reason for this in the data.

[0108] There is therefore a risk that highly autonomous driving could lead to attacks on the classification units used in the vehicle, potentially causing an accident. However, similar problems arise in all areas of artificial intelligence, such as text recognition, audio recognition, or speech recognition.

[0109] Figure 4 shows a defense system 10 which includes a target definition unit 11, an input data unit 13 and a defense generator 20.

[0110] The target definition unit 11 is designed to receive user input, for example through a graphical user interface or via a communication interface, which specifies the target to be achieved by the defense system 20.

[0111] User input allows the system to specify that an adversarial attack detection should be performed. This means that the system should only determine whether input data 14 exhibits adversarial interference.

[0112] Another way to define the goal is to eliminate the adversarial disturbances.

[0113] In the embodiment shown in Figure 4, the target definition unit 11 determines a target definition 12, taking into account the user input, which specifies whether detection, filtering, or reconstruction should be performed.

[0114] The target definition 12 is fed to an input data unit 13, which also receives an input data set 14. In the illustrated embodiment, the input data set 14 comprises a plurality of image data. In one possible embodiment, the image data of the input data set 14 are preprocessed such that they have an identical size. In another embodiment, the image data of the input data set 14 each have a different size. In one embodiment, the input data unit 13 is configured to preprocess the image data as described. Of course, the input data set 14 is not limited to image data and can comprise any possible form of data.

[0115] In general, and not specific to any particular embodiment, the input data set 14 can comprise data that does not exhibit adversarial interference. In one embodiment, the input data set 14 comprises a plurality of feature detectors 33, 35, 37 of an artificial neural network 30 (see Fig. 5). A feature detector 33, 35, 37 can be configured as a matrix to which an activation function is assigned. The weights of the activation function can be stored in the matrix. Possible activation functions are, for example, sigmoid, relu (Rectified Linear Unit), or tanh (Hyperbolic Tangent).

[0116] Input data set 14 can also be selected by a user via a graphical user interface. It is equally conceivable that input data set 14 is received via a communication interface, for example, via the internet using an API.

[0117] Furthermore, input data set 14 can have range restrictions. Range restrictions specify that only parts of the remaining input data set 14 should be considered. For example, input data set 14 can specify that only the lower part of an image should be viewed. This is particularly useful if important features are only expected to be found in certain areas of the image. The user thus has the option of storing prior knowledge about input data set 14 within the input data set 14 itself.

[0118] The input data unit 13 determines model data 15 using the target definition 12 and the input data set 14. The model data 15 is received by the defense generator 20. The defense generator 20 comprises a tiling unit 21, an aggregation unit 22, a distribution unit 23, an inference unit 24, and a data transformation unit 25.

[0119] The tile unit 21 is configured to divide the model data 15 into at least one tile 26 and thus determine a tile set 53. As already explained, local dependencies can be exploited by dividing the model data 15 into individual tiles 26, 53. The model data 15 can be scanned to create the at least one tile 26 or the tile set 53. For example, the input data set 14 and / or the model data 15 can be stored as a multidimensional tensor.

[0120] The tile unit 21 can be configured to scan the model data 15. This means that the model data 15 is sequentially divided into tiles 26. For this purpose, the tile unit 21 can be configured to receive a tile specification 211. The tile specification 211 can provide a kernel definition that specifies the size of the tiles 26. In addition to the kernel definition, the tile specification can also specify a stride and an offset. The stride specifies how many data elements, for example, pixels, of the model data 15 are exceeded between the individual scan steps. The offset, as already described in detail, ensures that the tile size and the step size do not cause the kernel to extend beyond the end of the data during scanning.

[0121] The at least one tile 26 is then received by the aggregation unit 22. The aggregation unit 22 is fundamentally configured to determine aggregated data 27 that map at least one tile 26 to a finite-dimensional vector space with real numbers.

[0122] The aggregation unit 22 is configured to receive an aggregation instruction 221, which specifies how the at least one tile 26 is to be processed to determine the aggregated data 27. The aggregation unit 22 can perform different operations to determine the aggregated data 27. In the illustrated embodiment, the aggregation instruction 221 specifies that the identity of the at least one tile 26 is to be used to determine the aggregated data 27. This means that the values ​​of the model data 15 themselves are used as the aggregated data 27.

[0123] The aggregation unit 22 outputs the aggregated data 27 to the distribution unit 23. The distribution unit 23 is configured to determine a distribution function 28 (see also Figure 8) from the aggregated data 27. The distribution function 28 indicates the distribution of different characteristics of the input data set 14 or the model data 15. The distribution unit 23 is configured to receive a distribution specification 231. The distribution specification 231 indicates whether the distribution function 28 is to be determined implicitly or explicitly.

[0124] Explicitly, this means that a specific distribution function exists which can be used to calculate distribution function 28. Implicitly, this means that distribution function 28 can only be determined indirectly, for example via an L. 2 Distance between input data and model data 15.

[0125] The inference unit 24 then receives the distribution function 28 and is configured to determine an inference configuration 29. Additionally, the inference unit receives at least one threshold value 241 and a reconstruction specification 242. The threshold value 241 specifies, during a comparison, whether a detection or a reconstruction should be performed. The reconstruction specification 242 indicates whether a data transformation to be performed involves the entire model data 15, a random subset of the model data 15, and / or an importance-based selection of the model data 15.

[0126] The inference unit 24 is trained to determine an inference configuration 29, i.e., how an AI defense module 16 works or is designed, either to determine whether there are adversarial disturbances in the model data 15 or in the input data set 14, or to determine a second input data set 14 that does not include any adversarial disturbances.

[0127] The inference configuration 29 is received by the data transformation unit 25, which uses the inference configuration 29 to generate at least one AI defense module 16. This at least one

[0128] AI Defense Module 16 can then be used to detect adversarial interference in input data or to determine a second input data set without adversarial interference.

[0129] The at least one AI defense module 16 can therefore be used, for example, as an add-on for an AI unit 30, so that before each classification or regression it is checked whether an adversarial disturbance is present in the input data of the AI ​​unit.

[0130] The AI ​​defense unit 16 processes input data from an AI unit according to the results of the defense generator 20. This means that the AI ​​unit 16 is trained to divide input data into at least one tile according to the tile specification 211. Furthermore, the AI ​​defense unit 16 is trained to determine aggregated data for the at least one tile according to the aggregation specification 221 and subsequently to determine a distribution function using the aggregated data and the distribution specification 231.

[0131] The AI ​​defense unit 16 checks, for example, to what extent the distribution function of the input data deviates from the distribution function 28 determined by the distribution unit 23. The AI ​​defense unit 16 is further configured to compare the determined deviation with the threshold value 241. Based on this comparison, it can be determined whether the input data set of the AI ​​unit 30 exhibits an adversarial disturbance.

[0132] Naturally, the defense generator 20 makes it possible to generate a large number of AI defense modules 16 with different configurations, enabling the detection and elimination of a large number of adversarial disturbances. Multiple AI defense modules can process input data from an AI unit 30 in parallel, thus enabling efficient execution.

[0133] Furthermore, the AI ​​defense unit 16 is designed to determine a second input data set for input data using the distribution function 28 and a data transformation. This second set should correspond to the input data without adversarial disturbances. Sampling-based reconstruction methods, for example, can be used here.

[0134] Figure 5 describes the structure of an artificial neural network 30, which is designed as a convolutional neural network (CNN) and represents a possible AI unit 30. The data to be processed is read into an input layer 31. For example, the RGB values ​​of a pixel in an image. In a first convolution 32, a kernel scans the input data layer 31 to create the first feature detectors 33. The feature detectors 33 can each be configured to recognize specific data patterns, such as edges or corners in an image.

[0135] It is possible to use the feature detectors, or the weights and neurons of a feature detector 33, as model data 15 for the defense system 10 or the defense generator 20. In a second convolution 34, a reduction of the feature detectors is carried out by means of so-called pooling or subsampling, so that second feature detectors 35 are generated. In a further step, third feature detectors 37 are generated in a further convolution 36. The last layer 38 of the CNN is fully connected. In the output layer 39, or soft-max layer, each value is assigned a probability of belonging to a class.

[0136] Figure 6 shows another way to improve the functionality of the defense system 20. Figure 6 shows an input image 40, which can be part of the input data 14. The input image 40 shows a stop sign 46, lane markings 44, 44', a center line 45, and clouds 43. In autonomous driving applications, it is now possible to further exploit the fact that information relevant to the task of autonomous driving is only found in certain image areas.

[0137] In the embodiment shown in Figure 40, all relevant information is arranged in a lower image section 42. Above a dividing line 47, i.e., in the upper image section 41, no information relevant to driving a vehicle can be found.

[0138] This division can be exploited in such a way that in the input data unit 13 the input data 14 is preprocessed in such a way that irrelevant image sections are deleted, i.e. the upper image section.

[0139] Figure 7 illustrates once again, by way of example, the division of an input image 50 or the model data 15 into a multitude of tiles 52, 52', 52”, 52'”.

[0140] Figure 7 shows that an input image 50, or the model data 15, is divided into four tiles 52, 52', 52", and 52'" using a tile specification 51. When scanning the input image 50, the scan proceeds row by row from the top left to the bottom right. This means that tile 52 is created first, followed by tile 52', then tile 52", and finally tile 52'". Thus, at least one tile 26, or the set of tiles 53, can be created efficiently. Figure 8 illustrates the determination 60 of the distribution function 28, 65. Figure 8 shows a set of tiles 61, where each data field of a tile in the set of tiles 61 contains a brightness value 62, which is an integer in the range of 0 to 255. Using the set of tiles 61 and a distribution specification 66, a distribution function 63 can now be determined in order to determine a distribution 65.

[0141] In the illustrated embodiment, the transition probability that a value in the data field to the right of the current data field assumes a specific brightness value is determined using the tile set 61. This value is also called the Overall Probability Value. This means that 256 values ​​are stored in each data field of distribution 65. Distribution 65 thus represents a probability distribution.

[0142] Using distribution 65, it can now be determined whether input data 5 contains an adversarial perturbation. This involves checking whether the transition probabilities of distribution 65 explain the input data 5. A threshold can be set to indicate the minimum probability that the values ​​of the input data are explained by the tile set 61. If the probability is above the threshold, there is no adversarial perturbation. If the probability is below the threshold, there is an adversarial perturbation.

[0143] It is pointed out that all the parts described above are considered individually and in any combination, especially the details shown in the drawings, as essential to the invention, but not indispensable.

[0144] Reference symbol list

[0145] 1 Entrance image

[0146] 2 Classification unit

[0147] 3, 3', 3", 3'" Image class

[0148] 4 Attack pattern

[0149] 5 Test data set / disturbed input image

[0150] 10 Defense system

[0151] 11. Target definition unit

[0152] 12. Target definition: Input data unit

[0153] Input data set

[0154] Model data

[0155] AI defense module

[0156] Defense generator

[0157] Tile unit

[0158] Aggregation unit 1 Aggregation specification

[0159] Distribution unit 1 Distribution information

[0160] Inference unit 1 Threshold 2 Reconstruction specification

[0161] Data transformation unit, 53, 61 tile / tile set of aggregated data

[0162] Distribution function

[0163] Inference configuration

[0164] AI Unit / Convolutional Neural Network (CNN)

[0165] Input layer, first fold, first feature detectors, second fold, second feature detectors, third fold, third feature detectors

[0166] Soft-Max layer

[0167] Output layer

[0168] Opening image

[0169] Upper / First image section Lower / Second image section Cloud, 44' Road marking

[0170] center line

[0171] Street sign

[0172] dividing line

[0173] Input image, 211 tile specification, 52', 52", 52"" Image tile Distribution calculation Brightness value Distribution function Transition matrix Probability distribution Distribution selection

Claims

REQUIREMENTS 1. Defense generator (20) for the dynamic generation of at least one AI defense module (16), comprising the following: a tile unit (21) configured to determine at least one tile (26, 53, 61) for model data (15), wherein the model data (15) are assigned to an AI unit (30) and the at least one tile (26, 53, 61) specifies at least a subset of the model data (15); an aggregation unit (22) configured to determine aggregated data (27), wherein the aggregated data (27) assign at least one, in particular mathematical, identifier to the at least one tile (26, 53, 61); a distribution unit (23) configured to determine a distribution function (28, 65) for the aggregated data (27); an inference unit (24) which is designed to determine at least one inference configuration (29) using the distribution function (28, 65);a data transformation unit (25) configured to generate at least one AI defense module (16) for the AI ​​unit (30) using the at least one inference configuration (29), wherein the at least one AI defense module (16) is configured to determine, for an input data set (5, 14) of the AI ​​unit (30): o whether an attack on the AI ​​unit (30) can be attributed to the input data set (5, 14); and / or o using a data transformation to determine a second input data set (14) to which no attack on the AI ​​unit (30) can be attributed.

2. Defense generator (20) according to claim 1, d ad u rc hge kenn ch net, that the tile unit (21) is configured to receive a tile specification (51, 211), wherein the tile specification (51, 211) is in particular user-definable, wherein the tile specification (51, 211) preferably specifies a kernel size, a stride and / or an offset, and wherein the tile unit (21) is designed to include at least one tile (26, 53, 61) using the tile reference (51, 211).

3. Defense generator (20) according to one of the preceding claims, which is characterized by the fact that the aggregation unit (22) is configured to receive an aggregation specification (221), in particular a user-definable one, wherein the aggregation specification (221) and / or the at least one identifier specifies a singular value decomposition, a convolution, a mean, a median and / or a variance for the at least one tile (26, 53, 61) and wherein the aggregation unit (22) is configured to determine the aggregated data (26) taking into account the aggregation specification (221).

4. Defense generator (20) according to one of the preceding claims, in particular according to claim 2, d ad u rc hge kenn zei ch net that the aggregation unit (22) is configured to receive a tile specification (211, 51) from the tile unit (21), wherein the aggregation unit (22) is configured to determine the aggregated data (27) by subtracting the tile specification (51, 221) from the at least one tile (26, 53, 61).

5. Defense generator (20) according to one of the preceding claims, wherein the defense generator (20) is configured to receive a target definition (12), in particular a user-definable one, wherein the inference unit (24) is configured to determine the inference configuration (29) taking into account the target definition (12), wherein the data transformation unit (25) is configured to select, taking into account the inference configuration (29), whether o a determination as to whether an attack on the AI ​​unit (30) can be attributed to the input data set (5, 14), and / or o a determination, using the data transformation, of a second input data set (14) to which no attack on the AI ​​unit (30) can be attributed, is carried out.

6. Defense generator (20) according to one of the preceding claims, characterized in that the distribution unit (23) is configured to receive a distribution specification (231), in particular a user-definable one, wherein the distribution unit (23) is further configured to determine the distribution function (65) taking into account the distribution specification (231), wherein the distribution specification (231) specifies an explicit or implicit distribution function (28).

7. Defense generator (20) according to one of the preceding claims, characterized in that the inference unit (24) is configured to receive at least one, in particular user-definable, threshold value (241) and to perform the determination of the inference configuration taking into account the at least one threshold value (241), wherein the at least one threshold value (241) indicates that if the at least one threshold value (241) is exceeded, a data transformation is performed by the AI ​​defense module (16).

8. Defense generator (20) according to one of the preceding claims, characterized in that the inference unit (24) is configured to receive a reconstruction specification (242), wherein the inference unit (24) is configured to determine the inference configuration (29) taking into account the reconstruction specification (242), wherein the reconstruction specification (242) indicates whether a data transformation concerns the entire model data (15), a random subset of the model data (15) and / or an importance-based selection of the model data (15).

9. Defense generator (20) according to one of the preceding claims, characterized in that a / the data transformation is designed as a sampling method.

10. Defense system (10) against an adversarial attack on an AI unit (30), comprising: an input unit (13) trained to, to receive input data (5, 14) and / or an input model (30) as model data (15); a defense generator (20), in particular according to one of claims 1 to 9, which is configured to receive the model data (15) and to generate at least one AI defense module (16); an AI unit (30) which is configured to use the at least one AI defense module (16) prior to performing a regression and / or classification in order to: o determine whether an attack on the AI ​​unit (30) can be attributed to an input data set (5, 14); and / or o determine a second input data set (14) to which no attack on the AI ​​unit (30) can be attributed using a data transformation and to use the second input data set (14) in the regression and / or classification.

11. Method for the dynamic generation of AI defense modules (16), comprising the following steps: Determining at least one tile (26, 53, 61) for model data (15), wherein the model data (15) are assigned to an AI unit (30) and the at least one tile (26, 53, 61) specifies at least a subset of the model data (15); Determining aggregated data (27), wherein the aggregated data (27) are assigned key figures, in particular mathematical ones, to at least one tile (26, 53, 61); Determining a distribution function (28, 65) for the aggregated data (27); Determine at least one inference configuration (29) using the distribution function (28, 65); Generating at least one AI defense module (16) for the AI ​​unit (30) using the at least one inference configuration (29), wherein the at least one AI defense module (16) is in particular configured to provide, for an input data set (5, 14) of the AI ​​unit (30): o to determine whether an attack on the AI ​​unit (30) can be associated with the input data set (5, 14); and / or o to determine a second input data set (14) that cannot be associated with an attack on the AI ​​unit (30) using a data transformation.

12. Method for preventing an attack on an AI unit (30), comprising the following steps: Generating at least one AI defense module (16), in particular according to claim 11; Determine, using the at least one AI defense module (16), whether an input data set (5, 14) of the AI ​​unit (30) can be attributed to an attack on the AI ​​unit (30); and / or Determine, using the at least one AI defense module (16), a second input data set (14) to which no attack on the AI ​​unit (30) can be attributed, and use the second input data set (14) in a regression and / or classification.

13. Method according to claim 12, characterized by Receiving a, especially a custom, Goal definition (12); Determining the inference configuration (29) taking into account the target definition (12); Selections, taking into account the target definition (12), whether o a determination can be made as to whether an attack on the AI ​​unit (30) can be attributed to the input data set (5, 14); and / or o a determination is made using the data transformation of a second input data set (14) to which no attack on the AI ​​unit (30) can be attributed.

14. Computer-readable storage medium containing instructions that cause at least one processor to implement a method according to any one of claims 11 to 13 when the instructions are executed by the at least one processor.