Defence generator, method and computer readable storage medium for preventing attacks on an ai unit

Abstract

The present application relates to a defense generator (20) for dynamically generating at least one AI defense module (16). A core feature of the present application is to determine a distribution function of model data. The present application is based on the assumption that model data belongs to a model manifold or has a similar statistical behavior. Thus, it can be determined for an input data set whether the data of the input data set can be associated with an adversarial attack. For example, if a statistical anomaly is found in the input data set, it can be determined that the data of the input data set can be associated with an adversarial attack.

Description

Technical Field

[0001] This application relates to a defense generator for dynamically generating at least one AI defense module, a defense system against adversarial attacks against AI units, a method for dynamically generating AI defense modules, a method for preventing attacks on AI units, and a computer-readable storage medium. Background Technology

[0002] Most machine learning methods in the present technology are known to be susceptible to adversarial interference. Therefore, robustness to adversarial interference is a major challenge in the development of machine learning methods.

[0003] Adversarial perturbation occurs when the data to be classified changes in a way that is imperceptible to human observers, making it impossible for the AI ​​unit to classify it correctly. Therefore, multiple misclassifications may occur.

[0004] In the context of image classification, such as in image data segmentation, adversarial interference can be caused by superimposed noise on the input image. This noise can be inserted into the input image in a way that is imperceptible to a human observer.

[0005] However, adversarial interference does not occur in the natural environment, i.e., in the sensor data that is normally provided. One scenario where adversarial interference occurs is when the AI ​​unit is attacked. This means that an attacker modifies the data provided to the AI ​​unit for classification, preventing it from classifying correctly. This can lead to significant security risks, especially in safety-critical applications such as highly automated driving. For example, if misclassification occurs in highly automated driving, the vehicle may fail to recognize stop signs or misrecognize stop signs, or it may fail to stop at red lights, posing a significant danger to other road users.

[0006] Therefore, one of the goals is to reduce the risk of adversarial attacks. There are isolated methods that can achieve this, but they are tailored to specific attacks. This means that, until now, countermeasures could only be developed after an attack has occurred.

[0007] However, it is well known that there are an infinite number of possibilities for attacking AI entities using adversarial interference.

[0008] Therefore, the purpose of this application is to provide a method for preventing AI units from adversarial attacks. Specifically, the purpose of this application is to identify adversarial attacks. More specifically, the purpose of this application is to design an AI unit that is robust against adversarial attacks. More concretely, the purpose of this application is to make it possible to generate an unlimited number of defense modules for the AI ​​unit. Summary of the Invention

[0009] The objective of this application is achieved through a defense generator for dynamically generating at least one AI defense module, a defense system against adversarial attacks against AI units, a method for dynamically generating AI defense modules, a method for preventing attacks against AI units, and a computer-readable storage medium.

[0010] Specifically, the above objective is achieved through a defense generator that dynamically generates at least one AI defense module, the defense generator comprising the following:

[0011] - A block unit for determining at least one block of model data, wherein the model data is associated with an AI unit, and at least one block represents at least one subset of the model data;

[0012] - Aggregation unit, used to determine aggregated data, wherein the aggregated data assigns at least one, in particular a mathematical key value, to at least one block;

[0013] - Distribution unit, used to determine the distribution function of aggregated data;

[0014] - An inference unit for determining at least one inference configuration using a distribution function;

[0015] - A data transformation unit for generating at least one AI defense module for an AI unit using at least one inference configuration, wherein the at least one AI defense module is used to perform the following steps on the input dataset of the AI ​​unit:

[0016] o Determine whether an attack on the AI ​​unit could be linked to the input dataset; and / or

[0017] o Use data transformation to determine the second input dataset, where an attack on the AI ​​unit cannot be associated with the second input dataset.

[0018] The core of this application is determining the distribution function of the model data. This application is based on the assumption that the model data belongs to the model manifold. In other words, the model data exhibit similar statistical behavior. Therefore, for an input dataset including, for example, images from an RGB camera, it is possible to determine whether the data in the input dataset is subject to adversarial attacks. For example, if statistical anomalies can be detected in the input dataset, it indicates that the data in the input dataset is subject to adversarial attacks.

[0019] Furthermore, this application also includes the possibility of determining a second input dataset that is free of statistical anomalies and therefore cannot be assigned to attacks against the AI ​​unit. Thus, noise inserted into the original data by adversarial attacks can be filtered out again.

[0020] This application is not limited to any particular type of AI unit. For example, an AI unit can be an artificial neural network, a Bayesian classifier, a Markov logic unit, or any other type of classifier.

[0021] In summary, this application provides a method for generating defenses against any adversarial attack. These defenses are implemented by a defense module of an AI unit. Therefore, the defense module of the AI ​​unit can be executable program code. This program code can be in compiled or uncompiled form. It is also conceivable that the defense module of the AI ​​unit can be implemented as an additional component of the AI ​​unit. Thus, before performing classification or regression tasks, the AI ​​unit can use its defense module to determine whether the AI ​​unit is under attack and render it harmless through data transformation.

[0022] For example, model data can be formed as tensors. At least one block can form a subset of a tensor. For example, at least one block can represent a one-dimensional tensor that includes a subset of the dimensions of the model data tensor. Model data can be formed in one-dimensional or multi-dimensional form.

[0023] In one embodiment, the model data can be formed as an image with at least one color channel, particularly three color channels, preferably red-green-blue.

[0024] If the model data is formed as an image, then at least one block can represent a portion of the image corresponding to a color channel.

[0025] According to the described embodiments, local dependencies can be utilized. Therefore, the model data can be divided into many chunks, which together form a chunk set.

[0026] For example, there are dependencies between closely spaced pixel values ​​in an image. If the distance between two pixels is large, there is almost no dependency. Similarly, there are significant dependencies between letters in a word. On the other hand, there are almost no dependencies between letters in different words or even different paragraphs. Therefore, different chunks of input data can be processed separately. This enables parallelization and thus more efficient execution.

[0027] In one embodiment, the segmentation unit can be used to receive a segmentation indication, wherein, in particular, the segmentation indication can be user-definable, preferably, the segmentation indication can represent kernel size, stride and / or offset, and the segmentation unit can be used to determine at least one segment by using the segmentation indication.

[0028] The kernel size can represent the size of at least one block; specifically, the kernel size can include an indication of the x-axis, y-axis, and / or z-axis. The stride or step size specifies the step size at which new blocks are generated when scanning the input dataset. An offset can be defined such that an offset of the step size does not create blocks smaller than other blocks. For example, if a 3×3 pixel kernel is used to scan an input image of size 10×10 pixels, and a three-pixel stride is defined for that kernel, blocks smaller than other blocks will appear. In this case, an offset of two pixels is required.

[0029] In one embodiment, the model data may specify image data, audio data, sensor data, text data, and / or at least one artificial neural network feature detector.

[0030] Model data can include a wide variety of data. In addition to image data already mentioned, it may also include voice data, sensor data, or text data. However, it is also possible to use a portion of the AI ​​unit as model data, rather than the data to be classified.

[0031] Therefore, it has been demonstrated that adversarial interference can lead to abnormal activation patterns in artificial neural networks.

[0032] Therefore, in one embodiment, the feature detector of an artificial neural network, especially a convolutional neural network, can form model data.

[0033] In one embodiment, the aggregation unit may be used to receive an aggregation instruction, particularly a user-defined aggregation instruction, wherein the aggregation instruction and / or at least one key value may respectively represent the singular value decomposition, convolution, mean, median and / or variance of at least one block, and wherein the aggregation unit may be used to determine aggregated data based on the aggregation instruction.

[0034] According to the described embodiment, the aggregation unit may be used, for example, to convolve the data of the input dataset using a filter to determine aggregated data. This can be achieved by a discrete Fourier transform, followed by multiplication and then integration, such that the convolution corresponds to multiplication in the spectrum, followed by integration.

[0035] In one embodiment, the aggregation unit can be used to perform singular value decomposition on at least one block. When a block has multiple dimensions, the block can be represented as a block matrix, where each block of the block matrix corresponds to one dimension of at least one block.

[0036] It has been demonstrated that adversarial perturbations produce anomalous patterns, particularly in the lower singular values ​​of the matrix. Therefore, adversarial perturbations in model data can be detected using singular value decomposition.

[0037] In one embodiment, the mean, median, and / or variance of at least one block can also be determined. In this case, the aggregated data can be formed as a vector of statistical values.

[0038] In one embodiment, the aggregation unit may be used to receive a block indication from the block unit, wherein the aggregation unit may be used to subtract the block indication from at least one block to determine aggregated data.

[0039] At least one chunk can specify multiple values. For example, a chunk can specify multiple brightness values. A chunk can also specify multiple letters, allowing a chunk to specify a piece of text.

[0040] In the above embodiments, it is also possible to subtract the median, variance, or mean from at least one block. If multiple blocks are provided, subtracting the above value from one block may mean, if the at least one block is formed as a tensor, subtracting the mean and / or median of all blocks / tensors from each data element of the tensor. All data elements of at least one block or tensor, or the median, variance, and / or mean of all blocks / tensors, can be determined.

[0041] The advantage of subtracting statistical values ​​is that it makes segments from different regions comparable.

[0042] In one embodiment, the defense generator may be used to receive target definitions, particularly user-definable target definitions, and the inference unit may be used to determine the inference configuration based on the target definitions.

[0043] In this regard, the data transformation unit can be used to select and perform the following operations based on the inference configuration: determine whether an attack on the AI ​​unit can be associated with the input dataset; and determine a second input dataset by using data transformation, wherein an attack on the AI ​​unit cannot be associated with the second input dataset.

[0044] Therefore, the behavior of the AI ​​defense module can be controlled through target definition. This provides a simpler method for users to define the behavior of the AI ​​defense module.

[0045] In one embodiment, the distribution unit can be used to receive a distribution indication, particularly a user-defined distribution indication, wherein the distribution unit can also be used to determine a distribution function based on the distribution indication, wherein the distribution indication can indicate an explicit or implicit distribution function.

[0046] Therefore, a distribution indicator can be used to indicate the method of calculating the probability distribution, thus allowing the data distribution to indicate a probability distribution. In this case, a distribution indicator can be used to indicate either an implicit or explicit distribution function.

[0047] The transition matrix can be determined using an explicit distribution function. The transition matrix can, for example, indicate a Markov chain. It is also conceivable that the Gibbs distribution is determined by an explicit distribution function.

[0048] The elements of the transition matrix can each correspond to a data point in the aggregated data. The transition matrix can specify the probability or probability distribution of a particular value for each element. Therefore, if the input dataset specifies image data, it's conceivable to assign a probability that each pixel has a specific brightness value to its neighboring pixels.

[0049] For example, implicit distribution functions can be implemented using Quilt or natural aggregation. In the Quilt approach, the distance from the input data to the model data is determined. For example, L... 2 Distance is determined using metrics or other methods. If the distance exceeds a predetermined threshold, an attack on the AI ​​unit can be inferred. Furthermore, a second input dataset can be determined by replacing blocks of the input data with the closest blocks from the model data, based on the metrics used. In this case, the model data no longer has adversarial perturbations, so undisturbed blocks in the input data can be replaced.

[0050] In one embodiment, the inference unit may be used to receive at least one threshold, particularly a user-defined threshold, and to determine an inference configuration based on the at least one threshold, wherein the at least one threshold may indicate that the AI ​​defense module performs data transformation when the at least one threshold is exceeded.

[0051] In one embodiment, for example as described above, the threshold can be compared to a distance. This means that the AI ​​defense module can be used to compare the threshold with the distance between the input data or a subset of the input data and at least one of the blocks.

[0052] In one embodiment, the AI ​​defense module can also be used to perform data transformation based on the above comparison (e.g., when the distance is greater than a threshold).

[0053] In one embodiment, the AI ​​defense unit can be used to determine whether an attack on the AI ​​unit can be associated with the input dataset based on the comparisons described above (e.g., if the distance is greater than a threshold).

[0054] Besides distance, one can also think of using AI defense units to determine a so-called overall probability value for the model data. A threshold can then be compared to this overall probability value.

[0055] In one embodiment, the AI ​​defense unit used to generate the second input data can be used to perform smoothing processing on the input data or at least one block based on the comparison described above. Smoothing of the input data or at least one block may result in adversarial interference no longer affecting the AI ​​unit.

[0056] In one embodiment, the inference unit may be configured to receive a reconstruction instruction, wherein the inference unit may be configured to determine an inference configuration based on the reconstruction instruction, wherein the reconstruction instruction may indicate whether the data transformation involves the entire model data, a random subset of the model data, and / or an importance-based selection of the model data.

[0057] Therefore, using the described embodiments, it is also possible to reconstruct only a portion of the input data. This makes the overall efficiency of the device higher, and errors can be avoided if only a portion of the input data needs to be considered.

[0058] Therefore, in one embodiment, the AI ​​defense unit can be used to decompose the input data into at least one block as described above. Only certain blocks can be considered during the reconstruction process. Thus, in the case of image data, individual image regions can be examined separately so that the AI ​​defense unit can calculate, in particular, mathematically critical values ​​for these image regions in each case. This embodiment takes into account the fact that, for example, a sky with a uniform blue color has different characteristics in a mathematical-visual average than, for example, an image region showing a road. Image regions or blocks corresponding to these image regions can be statically determined. Furthermore, it is conceivable that image regions or blocks can be dynamically determined by the AI ​​defense unit, taking into account image properties.

[0059] In one embodiment, the second input dataset is free from adversarial interference. This means that the AI ​​unit with the second input data functions as expected.

[0060] In one embodiment, data transformation can be formulated as a sampling method.

[0061] Using sampling methods, a second input dataset can be determined so that adversarial interference is no longer present in the second input dataset.

[0062] The above objective is further achieved through a defense system against adversarial attacks on AI units, the defense system comprising:

[0063] - Input unit, used to receive input data and / or input model as model data;

[0064] - A defense generator, specifically as described above, for receiving model data and generating at least one AI defense module;

[0065] -AI unit, used to employ at least one AI defense module before performing regression and / or classification, so that

[0066] o Determine whether an attack on the AI ​​unit can be correlated with the AI ​​unit's input dataset; and / or

[0067] o Use data transformation to determine a second input dataset, where attacks on AI units cannot be correlated with the second input dataset, and use the second input dataset in regression and / or classification.

[0068] Therefore, it is also conceivable that at least one AI defense module generated by the defense generator could be used to target attacks on AI units within the defense system. This would allow for attacks on AI units in an effective and secure manner.

[0069] Specifically, the above objective is also achieved through a method for dynamically generating AI defense modules, which includes the following steps:

[0070] - Identify at least one block of model data, wherein the model data is associated with an AI unit, and at least one block indicates at least one subset of the model data;

[0071] - Identify aggregated data, where at least one block of aggregated data is allocated, particularly mathematically key values;

[0072] - Determine the distribution function of the aggregated data;

[0073] - Use the distribution function to determine at least one inference configuration;

[0074] - Generate at least one AI defense module for the AI ​​unit using at least one inference configuration, wherein at least one AI defense module is particularly well-suited for the input dataset of the AI ​​unit, so as to:

[0075] o Determine whether an attack on the AI ​​unit can be correlated with the input dataset; and / or

[0076] o Use data transformation to determine the second input dataset, where an attack on the AI ​​unit cannot be associated with the second input dataset.

[0077] In one embodiment, the method may include receiving a chunking instruction, particularly a user-defined chunking instruction, wherein the chunking instruction may be user-definable, and preferably, the chunking instruction may specify a kernel size, stride, and / or offset.

[0078] In this regard, at least one block can be determined using a block indicator. The kernel size can be selected based on the operation of the AI ​​unit. For example, the kernel size can be selected so that the operation of the AI ​​unit is not affected by replacing blocks in the AI ​​unit's input data with the AI ​​defense module.

[0079] In one embodiment, the method may include receiving an aggregation indication, wherein the aggregation indication may indicate at least one key value of at least one block, wherein the at least one key value may represent a singular value decomposition, convolution, mean, median and / or variance of at least one block.

[0080] In one embodiment, determining aggregated data may include subtracting a mathematical ratio from at least one chunk.

[0081] In one embodiment, the method may include:

[0082] - Receive target definitions, especially user-defined target definitions;

[0083] - Determine the inference configuration based on the target definition.

[0084] In one embodiment, the method may include receiving an aggregation indication, particularly a user-defined aggregation indication, wherein at least one key value and / or the aggregation indication may represent the singular value decomposition, convolution, mean, median, and / or variance of at least one block, wherein aggregated data can be determined based on the aggregation indication.

[0085] In one embodiment, determining aggregated data may include subtracting aggregated data from at least one chunk.

[0086] In one embodiment, the method may include receiving a target definition, particularly a user-defined target definition, wherein an inference configuration can be determined based on the target definition.

[0087] Furthermore, the method may include selecting, based on the inference configuration, whether to perform the following operations: determining whether an attack on the AI ​​unit can be associated with an input dataset; and / or using data transformation to determine a second input dataset, wherein the second input dataset cannot be associated with an attack on the AI ​​unit.

[0088] In one embodiment, the method may include receiving a distribution indication, particularly a user-defined distribution indication, representing an explicit or implicit distribution function. In this case, the distribution function can be determined based on the distribution information.

[0089] In one embodiment, the method may include receiving at least one threshold, particularly a user-defined threshold, wherein the at least one threshold may indicate that the AI ​​defense module performs data transformation when the at least one threshold is exceeded.

[0090] In one embodiment, the method may include receiving a refactoring instruction, particularly a user-defined refactoring instruction, wherein the refactoring instruction may indicate whether the data transformation involves the entire model data, a random subset of the model data, and / or a selection of model data based on importance. In this regard, the inference configuration can be determined based on the refactoring instruction.

[0091] In one embodiment, data transformation can be formulated as a sampling method.

[0092] In particular, the above objective is further achieved by a method for preventing attacks on AI units, the method comprising the following steps:

[0093] - Specifically, using the defense generator described above, at least one AI defense module is generated according to the method described above;

[0094] - Determine whether the input dataset of the AI ​​unit can be associated with an attack on the AI ​​unit by using at least one AI defense module; and / or;

[0095] - A second input dataset is determined by using at least one AI defense module, wherein an attack on the AI ​​unit cannot be associated with the second input dataset, and the second input dataset is used in regression and / or classification.

[0096] In one embodiment, the method may include:

[0097] - Receive target definitions, especially user-defined target definitions;

[0098] - Determine the inference configuration based on the target definition;

[0099] -Based on the target definition, choose whether to perform the following operations:

[0100] o Determine whether an attack on the AI ​​unit can be correlated with the input dataset; and / or

[0101] o Determine the second input dataset by using data transformation, where an attack on the AI ​​unit cannot be associated with the second input dataset.

[0102] In particular, the above objective is further achieved by a computer-readable storage medium containing instructions that, when executed by at least one processor, cause at least one processor to perform the method as described above.

[0103] It can achieve similar or the same advantages as those described in the above-mentioned methods, defense generators and systems.

[0104] It should be clearly pointed out in this regard that all the method steps disclosed in this application in conjunction with the system description are also necessary steps for the above-described method and computer-readable storage medium, and vice versa. In particular, the details regarding the defense generator and / or system description also apply to the above-described method and computer-readable storage medium, and vice versa.

[0105] Further embodiments will be apparent from the dependent claims. Attached Figure Description

[0106] The present application will be explained in more detail below with reference to exemplary embodiments, wherein:

[0107] Figure 1 A diagram illustrating the classification principle of an example image is shown.

[0108] Figure 2 A schematic diagram of the overlay principle for example images with adversarial interference is shown.

[0109] Figure 3 A diagram illustrating the classification principle of an example image with adversarial interference is shown.

[0110] Figure 4 A schematic diagram of an AI defense system is shown.

[0111] Figure 5 A schematic diagram of an artificial neural network is shown.

[0112] Figure 6 A schematic diagram of the subdivision of the input image is shown.

[0113] Figure 7 This diagram illustrates the process of dividing the input image into different blocks.

[0114] Figure 8 A schematic diagram is shown to determine the probability distribution of multiple blocks. Detailed Implementation

[0115] Figure 1 The diagram illustrates the classification principle of input image 1. Figure 1 In an exemplary embodiment, input image 1 shows a schematic diagram of a stop sign. Therefore, for example, in the case of highly automated driving, input image 1 may have been captured, for example, by the vehicle's camera system.

[0116] To achieve highly automated driving, the individual image components of input image 1 need to be classified. This means that each pixel must be assigned a classification, determining whether it belongs to a specific object or to which object, or whether the pixel displays the background, such as the sky.

[0117] In the exemplary embodiment shown, for each pixel of the input image 1, there may be four simplified categories 3, 3', 3'', 3''''. A classification unit 2 for classifying the pixels is provided, which can be designed, for example, an artificial neural network. However, it is also conceivable that the classification unit 2 can be designed as a support vector machine or a Bayesian network.

[0118] Classification Unit 2 is trained during the training phase for a specific classification task. When training Classification Unit 2, the cost function is minimized using the training dataset. The training dataset includes example images and their corresponding relevant classifications. Therefore, the training dataset provides what is known as the ground truth.

[0119] Typically, classification unit 2 can segment or classify individual pixels with appropriate reliability.

[0120] Figure 2 This illustrates an adversarial interference scenario. Input image 1 is superimposed with attack image 4. The result of the superposition is the interfered input image 5. To a human observer, the interfered input image 5 is indistinguishable from input image 1.

[0121] Figure 3 The behavior of classification unit 2 is shown when classifying a disturbed input image 5. Therefore, classification unit 2 will not behave like... Figure 1 Instead of assigning the disturbed input image 5 to category 3' as in the previous case, it was assigned to category 3'''. Therefore, an incorrect classification occurred.

[0122] Therefore, it is possible to deliberately interfere with Classification Unit 2 when the cause cannot be identified by human users in the data. Thus, during highly automated driving, there is a risk of attacking Classification Unit 2 used in the vehicle, potentially causing an accident.

[0123] However, similar problems exist in all areas of AI, such as text recognition, audio recognition, or speech recognition.

[0124] Figure 4 A defense system 10 is shown, comprising a target definition unit 11, an input data unit 13, and a defense generator 20.

[0125] The target definition unit 11 is used, for example, to receive user input via a graphical user interface or a communication interface, indicating the target of the defense system 10 to be achieved.

[0126] User input can indicate that adversarial attack detection is required. This means that it is only necessary to determine whether input data 14 contains adversarial interference.

[0127] Another way to define a goal is to eliminate adversarial interference.

[0128] exist Figure 4 In an exemplary embodiment, the target definition unit 11 determines the target definition 12 based on user input, which details whether detection, filtering, or reconstruction should be performed.

[0129] Target definition 12 is provided to input data unit 13, which further receives input dataset 14. In the illustrated exemplary embodiment, input dataset 14 includes multiple image data. In one possible exemplary embodiment, the image data of input dataset 14 are preprocessed to have the same size. In another exemplary embodiment, the image data of input dataset 14 each have a different size. In one exemplary embodiment, input data unit 13 is used to preprocess the image data as described. It should be understood that input dataset 14 is not limited to image data and may include data of any possible form.

[0130] Generally, and regardless of any particular exemplary embodiment, input dataset 14 may include data that does not exhibit adversarial interference.

[0131] In one exemplary embodiment, the input dataset 14 includes an artificial neural network 30 (see [link to example]). Figure 5 Multiple feature detectors 33, 35, and 37 are used. In this case, feature detectors 33, 35, and 37 can be formed as a matrix assigned activation functions. This matrix can store the weights of the activation functions. For example, the activation functions could be the sigmoid function, the ReLU function, or the hyperbolic tangent function (tanh).

[0132] Input dataset 14 can also be selected by the user through a graphical user interface. Similarly, it is conceivable that input dataset 14 could be received via a communication interface, such as the internet via an API.

[0133] Furthermore, the input dataset 14 may have range constraints. Range constraints indicate that only the remaining portion of the input dataset 14 should be viewed. For example, input dataset 14 could represent viewing only the lower part of an image. This is particularly useful if you want to find important features only in certain regions of an image. Therefore, the user is able to store prior knowledge about the input dataset 14 within the input dataset 14 itself.

[0134] Input data unit 13 uses target definition 12 and input dataset 14 to determine model data 15. Model data 15 is received by defense generator 20. Defense generator 20 has a block unit 21, an aggregation unit 22, a distribution unit 23, an inference unit 24, and a data transformation unit 25.

[0135] The segmentation unit 21 is used to divide the model data 15 into at least one segment 26, thereby determining a set of segments 53. As described above, local dependencies can be utilized by dividing the model data 15 into individual segments 26, 53. To create at least one segment 26 or a set of segments 53, the model data 15 can be scanned. For example, the input dataset 14 and / or the model data 15 can be stored as a multidimensional tensor.

[0136] Blocking unit 21 can be used to scan model data 15. This means that model data 15 is divided into blocks 26 sequentially. For this purpose, blocking unit 21 can be used to receive block indication 211. Block indication 211 can indicate a kernel definition representing the size of block 26. In addition to the kernel definition, the block indication can also indicate stride and offset. Stride specifies how many data elements, such as pixels, of model data 15 are spanned between each scan step. As described in detail above, offset ensures that the block size and stride do not cause the kernel to exceed the end of the data during scanning.

[0137] Subsequently, at least one block 26 is received by aggregation unit 22. Aggregation unit 22 is essentially used to determine aggregated data 27 that maps at least one block 26 to a finite-dimensional vector space containing real numbers.

[0138] Aggregation unit 22 is configured to receive aggregation instruction 221, which indicates how at least one chunk 26 will be processed to determine aggregate data 27. At this point, aggregation unit 22 can perform various operations to determine aggregate data 27. In the illustrated exemplary embodiment, aggregation instruction 221 indicates that the identifier of at least one chunk 26 will be used to determine aggregate data 27. This means that the value of model data 15 itself is used as aggregate data 27.

[0139] Aggregation unit 22 passes aggregated data 27 to distribution unit 23. Distribution unit 23 is used to determine distribution function 28 from aggregated data 27 (see also...). Figure 8 Distribution function 28 represents the distribution of different features of the input dataset 14 or model data 15. Distribution unit 23 is used to receive distribution instruction 231. Distribution instruction 231 indicates whether the distribution function 28 should be determined implicitly or explicitly.

[0140] Explicit means that there exists a distribution function that can be used to compute distribution function 28. Implicit means that distribution function 28 can only be determined indirectly, for example, through the L-axis between the input data and model data 15. 2 distance.

[0141] Subsequently, the inference unit 24 receives the distribution function 28 and uses it to determine the inference configuration 29. Additionally, the inference unit receives at least one threshold 241 and a reconstruction instruction 242. The at least one threshold 241 indicates whether detection or reconstruction is performed during comparison. The reconstruction instruction 242 indicates whether the data transformation to be performed involves the entire model data 15, a random subset of model data 15, and / or an importance-based selection of model data 15.

[0142] The inference unit 24 is used to determine the inference configuration 29, i.e. how the AI ​​defense module 16 operates or is configured, to determine whether adversarial interference exists in the model data 15 or the input dataset 14, or to determine a second input dataset 14 that does not include adversarial interference.

[0143] The inference configuration 29 is received by the data conversion unit 25, which uses the inference configuration 29 to generate at least one AI defense module 16. The at least one AI defense module 16 can then be used to detect adversarial interference in the input data or to determine a second input dataset without adversarial interference.

[0144] Therefore, at least one AI defense module 16 can be used as an additional component, such as AI unit 30, to enable an examination to determine whether adversarial interference exists in the input data of the AI ​​unit before any classification or regression.

[0145] AI defense unit 16 processes the input data of the AI ​​unit based on the results of defense generator 20. This means that AI defense unit 16 is used to divide the input data into at least one block according to block instruction 211. Further, AI defense unit 16 is used to determine the aggregated data corresponding to aggregation instruction 221 for at least one block, and then use the aggregated data and distribution instruction 231 to determine the distribution function.

[0146] For example, AI defense unit 16 checks the degree to which the distribution function of the input data deviates from the distribution function 28 determined by distribution unit 23. AI defense unit 16 is also used to compare the determined deviation with a threshold 241. Based on the above comparison, it can be determined whether the input dataset of AI unit 30 has adversarial interference.

[0147] It is understandable that the defense generator 20 can generate multiple AI defense modules 16 with different configurations, thereby enabling the detection or elimination of a large number of adversarial interferences. In this regard, multiple AI defense modules can process the input data from the AI ​​unit 30 in parallel, thereby achieving efficient execution.

[0148] Furthermore, the AI ​​defense unit 16 is used to determine a second input dataset that corresponds to the input data and is free from adversarial interference, using the distribution function 28 and the data transformation. For example, a sampling-based reconstruction method can be used here.

[0149] Figure 5The structure of an artificial neural network 30 is described. This artificial neural network 30 is designed as a convolutional neural network (CNN) and represents a possible AI unit 30. In the input layer 31, data to be processed is read in, such as the RGB values ​​of image pixels. In the first convolution 32, a kernel scan is performed on the input data layer 31 to create a first feature detector 33. Each feature detector 33 can be used to detect specific data patterns, such as edges or corners in the image.

[0150] The weights and neurons of the feature detector or feature detector 33 can be used as model data 15 for the defense system 10 or the defense generator 20.

[0151] In the second convolution 34, feature detector reduction is performed through so-called pooling or subsampling, resulting in a second feature detector 35. In the next step, a third feature detector 37 is generated in a further convolution 36. The final layer 38 of the CNN is fully connected. In the output layer 39, or Softmax layer, each value is assigned a probability of belonging to a certain class.

[0152] Figure 6 This demonstrates another way to improve the operation of the defense system 20. Figure 6 Input image 40 is shown, which may be a part of input data 14. Input image 40 shows a stop sign 46, lane markings 44, 44', a center line 45, and a cloud 43. In autonomous driving applications, this fact can now be taken advantage of more, namely that information relevant to the autonomous driving task can only be found in certain areas of the image.

[0153] exist Figure 6 In the exemplary embodiment shown, all relevant information is arranged in the lower image portion 42. Above the dividing line 47, i.e., in the upper image portion 41, there is no information related to driving the vehicle.

[0154] This division can be utilized in such a way that, in the input data unit 13, the input data 14 is preprocessed by deleting irrelevant image portions, namely the upper image portions.

[0155] Figure 7 To illustrate again, the input image 50 or model data 15 is divided into multiple blocks 52, 52', 52'', and 52'''.

[0156] Figure 7The diagram illustrates the use of block instruction 51 to divide the input image 50 or model data 15 into four blocks 52, 52', 52'', and 52'''. When scanning the input image 50, scanning is performed row-by-row from top left to bottom right. This means that block 52 is created first. Then, block 52' is created, followed by block 52'', and finally block 52'''. Therefore, at least one block 26 or a group of blocks 53 can be created efficiently.

[0157] Figure 8 The determination of distribution functions 28 and 65 is shown in 60. Figure 8 A block set 61 is shown, in which each data field of a block in the block set 61 stores a brightness value 62, which is an integer ranging from 0 to 255. Using the block set 61 and the distribution indicator 66, the distribution function 63 can now be determined to determine the distribution 65.

[0158] In the exemplary embodiment shown, block set 61 is used to determine the conversion probability, i.e., the value in the data field to the right of the current data field represents a specific brightness value. This value is also called the overall probability value. This means that 256 values ​​are stored in each data field of distribution 65. Therefore, distribution 65 specifies the probability distribution.

[0159] Now, using distribution 65, we can determine whether input data 5 contains adversarial interference. We can check whether the transition probabilities of distribution 65 can explain input data 5. In this case, a threshold can be set that indicates the minimum probability that the value of the input data can be explained by the block set 61. If the probability is higher than the threshold, there is no adversarial interference. If the probability is lower than the threshold, there is adversarial interference.

[0160] It should be noted that all of the foregoing portions are necessary to this application, but not essential, when considered individually or in any combination, especially the details shown in the accompanying drawings.

[0161] List of reference numerals

[0162] 1. Input image

[0163] 2 taxonomic units

[0164] 3, 3', 3'', 3''' Image class

[0165] 4 attack images

[0166] 5. Test dataset / Disturbed input image

[0167] 10 defense systems

[0168] 11 Target Definition Unit

[0169] 12 Target Definition

[0170] 13 Input Data Units

[0171] 14 Input Dataset

[0172] 15 model data

[0173] 16 AI defense modules

[0174] 20 Defense Generator

[0175] 21 block units

[0176] 22 polymerization units

[0177] 221 Aggregation Indicator

[0178] 23 distribution units

[0179] 231 distribution indication

[0180] 24 Reasoning Units

[0181] 241 threshold

[0182] 242 Refactoring Instructions

[0183] 25 data conversion units

[0184] 26, 53, 61 blocks / block sets

[0185] 27 Aggregated Data

[0186] 28 Distribution Function

[0187] 29 Reasoning Configuration

[0188] 30 AI units / Convolutional Neural Network (CNN)

[0189] 31 Input Layer

[0190] 32 First convolution

[0191] 33 First Feature Detector

[0192] 34 Second Convolution

[0193] 35 Second Feature Detector

[0194] 36 Third convolution

[0195] 37 Third Feature Detector

[0196] 38 Soft Max layers

[0197] 39 Output Layer

[0198] 40 input images

[0199] 41 Upper / First Image Section

[0200] 42 Lower / Second Image Section

[0201] 43 Clouds

[0202] Lane markings 44, 44'

[0203] 45 center line

[0204] 46 Road Sign

[0205] 47 dividing line

[0206] 50 input images

[0207] 51, 211 block instructions

[0208] Image blocks of 52, 52', 52'', 52''''

[0209] 60 distribution calculation

[0210] 62 brightness value

[0211] 63 Distribution Function

[0212] 64 Transition Matrix

[0213] 65 probability distribution

[0214] 66 distribution selection

Claims

1. A defense generator (20) for dynamically generating at least one AI defense module (16), comprising: - Block unit (21) for determining at least one block (26, 53, 61) of model data (15), wherein the model data (15) is associated with AI unit (30), and the at least one block (26, 53, 61) represents at least one subset of the model data (15); - Aggregation unit (22) for determining aggregated data (27), wherein the aggregated data (27) assigns at least one key value to the at least one block (26, 53, 61). - Distribution unit (23) is used to determine the distribution function (28, 65) of the aggregated data (27); -Inference unit (24) for determining at least one inference configuration (29) using the distribution function (28, 65); - Data transformation unit (25) for generating at least one AI defense module (16) of the AI ​​unit (30) using the at least one inference configuration (29), wherein the at least one AI defense module (16) is used to perform the following steps on the input dataset (5, 14) of the AI ​​unit (30): o Determine whether an attack on the AI ​​unit (30) can be associated with the input dataset (5, 14); and / or o Use data transformation to determine the second input dataset (14), wherein an attack on the AI ​​unit (30) cannot be associated with the second input dataset (14). The inference unit (24) is used to receive a reconstruction instruction (242), wherein the inference unit (24) is used to determine the inference configuration (29) according to the reconstruction instruction (242), wherein the reconstruction instruction (242) indicates whether the data transformation involves the entire model data (15), a random subset of the model data (15) and / or an importance-based selection of the model data (15).

2. The defense generator (20) according to claim 1, characterized in that, The segmentation unit (21) is used to receive segmentation instructions (51, 211), wherein the segmentation instructions (51, 211) are user-definable, wherein the segmentation instructions (51, 211) represent kernel size, stride and / or offset, and wherein the segmentation unit (21) is used to determine the at least one segment (26, 53, 61) using the segmentation instructions (51, 211).

3. The defense generator (20) according to any one of claims 1 to 2, characterized in that, The aggregation unit (22) is used to receive an aggregation instruction (221), wherein the aggregation instruction (221) and / or the at least one key value in each case represent the singular value decomposition, convolution, mean, median and / or variance of the at least one block (26, 53, 61), and wherein the aggregation unit (22) is used to determine the aggregated data (27) according to the aggregation instruction (221).

4. The defense generator (20) according to any one of claims 1 to 2, characterized in that, The aggregation unit (22) is used to receive the block indication (211, 51) from the block unit (21), wherein the aggregation unit (22) used to determine the aggregated data (27) is used to subtract the block indication (51, 221) from the at least one block (26, 53, 61).

5. The defense generator (20) according to any one of claims 1 to 2, characterized in that, The defense generator (20) is used to receive a target definition (12), wherein the inference unit (24) is used to determine the inference configuration (29) based on the target definition (12), wherein - The data conversion unit (25) is used to select whether to perform the following operations based on the inference configuration (29): o Determine whether the input datasets (5, 14) can be associated with an attack on the AI ​​unit (30), and / or o Determines a second input dataset (14) by using data transformation, wherein an attack on the AI ​​unit (30) cannot be associated with the second input dataset (14).

6. The defense generator (20) according to any one of claims 1 to 2, characterized in that, The distribution unit (23) is used to receive a distribution instruction (231), wherein the distribution unit (23) is also used to determine the distribution function (65) based on the distribution instruction (231), wherein the distribution instruction (231) represents an explicit or implicit distribution function (28).

7. The defense generator (20) according to any one of claims 1 to 2, characterized in that, The inference unit (24) is used to receive at least one threshold (241) and determine the inference configuration based on the at least one threshold (241), wherein the at least one threshold (241) indicates that when the at least one threshold (241) is exceeded, the AI ​​defense module (16) performs the data transformation.

8. The defense generator (20) according to any one of claims 1 to 2, characterized in that, The data transformation forms a sampling method.

9. A defense system (10) against adversarial attacks on an AI unit (30), the defense system comprising: - Input unit (13) is used to receive input data and / or input model as model data (15). - A defense generator (20) according to any one of claims 1 to 8, for receiving the model data (15) and generating at least one AI defense module (16). -AI unit (30), used to employ at least one AI defense module (16) before performing regression and / or classification, so that o Determine whether an attack on the AI ​​unit (30) can be associated with the input dataset (5, 14) of the AI ​​unit (30); and / or o determines a second input dataset (14) by using data transformation, wherein an attack on the AI ​​unit (30) cannot be associated with the second input dataset (14), and the second input dataset (14) is used in the regression and / or classification.

10. A method for dynamically generating an AI defense module (16), the method comprising the following steps: - Determine at least one block (26, 53, 61) of model data (15), wherein the model data (15) is associated with AI unit (30) and the at least one block (26, 53, 61) represents at least one subset of the model data (15); - Determine aggregate data (27), wherein the aggregate data (27) assigns at least one key value to the at least one block (26, 53, 61). - Determine the distribution function (28, 65) of the aggregated data (27); - Use the distribution functions (28, 65) to determine at least one inference configuration (29); - Generate at least one AI defense module (16) of the AI ​​unit (30) using the at least one inference configuration (29), wherein the at least one AI defense module (16) is adapted for the input dataset (5, 14) of the AI ​​unit (30) so as to: o Determine whether an attack on the AI ​​unit (30) can be associated with the input dataset (5, 14); and / or o Use data transformation to determine the second input dataset (14), wherein an attack on the AI ​​unit (30) cannot be associated with the second input dataset (14). The method further includes receiving a reconstruction instruction (242), wherein the inference configuration (29) is determined based on the reconstruction instruction (242), wherein the reconstruction instruction (242) indicates whether the data transformation involves the entire model data (15), a random subset of the model data (15), and / or an importance-based selection of the model data (15).

11. A method for preventing attacks on an AI unit (30), the method comprising the following steps: - Generate at least one AI defense module (16) according to claim 10. - By using the at least one AI defense module (16), determine whether the input dataset (5, 14) of the AI ​​unit (30) can be associated with an attack on the AI ​​unit (30); and / or - A second input dataset (14) is determined by using the at least one AI defense module (16), wherein an attack on the AI ​​unit (30) cannot be associated with the second input dataset (14), and the second input dataset (14) is used in regression and / or classification.

12. The method according to claim 11, characterized in that, - Receive target definition (12); - Determine the inference configuration (29) based on the target definition (12); - Select whether to perform the following operations based on the target definition (12): o Determine whether an attack on the AI ​​unit (30) can be associated with the input dataset (5, 14); and / or o Determine the second input dataset (14) by using data transformation, wherein an attack on the AI ​​unit (30) cannot be associated with the second input dataset (14).

13. A computer-readable storage medium comprising instructions that, when executed by at least one processor, cause the at least one processor to perform the method of any one of claims 10 to 12.

Images