Apparatus and method for training a classifier and for evaluating the robustness of a classifier - Patents.com

Abstract

To provide a computer-implemented method for training a classifier (60), in which the classifier (60) is configured to classify an input signal of digital image data and / or audio data, and the step of training the classifier is performed on the basis of a perturbed input signal obtained by applying perturbation provided from a plurality (ξ) of perturbation to an input signal provided from a training data set.SOLUTION: A method includes the steps of: providing a plurality of initial perturbation; adapting perturbation from the plurality of initial perturbation to an input signal, in which the input signal is randomly drawn from a training data set, and the perturbation is adopted to the input signal so that adaptation of the perturbation to the input signal generates a second input signal classified differently from a first input signal; providing a sub-set of the plurality of initial perturbation as a plurality (ξ) of perturbation; and training a classifier (60) on the basis of the plurality (ξ) of perturbation.SELECTED DRAWING: Figure 7

Description

[Technical field]

[0001] The present invention relates to a method for training a classifier, a method for determining a robustness value of a classifier, a method for operating an actuator, a computer program, a machine-readable storage medium, a classifier, a control system and a training system. [Background technology]

[0002] Background technology In DE 102018200724 A1 a method for generating universal adversarial perturbations is disclosed. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] DE 102018200724 A1 Summary of the Invention [Problem to be solved by the invention]

[0004] Advantages of the invention Classifiers, especially those based on machine learning models, are susceptible to adversarial examples, i.e., input signals that are modified in a way that causes misclassification by the respective classifier. Therefore, enhancing classifiers to be robust against adversarial examples is an important task for many applications.

[0005] For example, consider an autonomous vehicle that uses a classifier to detect pedestrians around the vehicle and plan a trajectory that avoids colliding with any of the detected pedestrians. It is conceivable that an attacker could create adversarial examples that trick the classifier into not detecting certain pedestrians, causing the vehicle to plan a trajectory that would result in a collision with the pedestrians that were missed.

[0006] A classifier can be made robust against adversarial examples by training it using adversarial training methods, such as PGD-based adversarial training or training with universal adversarial examples. Adversarial training methods may require initial perturbations to create adversarial examples during the training process. Obtaining a suitable set of initial perturbations is paramount for adversarial training methods that adapt the initial perturbations to the data used for training, for example, to obtain perturbations of increasing strength. If the initial perturbations are not suitable, it may take a long time to adapt the initial perturbations to the data. [Means for solving the problem]

[0007] The method with the features of independent claim 1 can advantageously provide a variety of perturbations as initialization for the adversarial training. Due to the way in which multiple perturbations are provided, these perturbations require less time to adjust to the training data, which also speeds up the training. This leads to a more robust classifier, since it can be trained with more data in a given period of time.

[0008] Disclosure of the Invention In a first aspect, the present invention relates to a computer implemented method for training a classifier, the classifier being configured to classify an input signal of digital image data and / or audio data, and the step of training the classifier is based on a perturbed input signal obtained by applying a perturbation provided from a plurality of perturbations to the input signal provided from a training data set, the method comprising the steps of: providing a plurality of initial perturbations; - fitting a perturbation from a plurality of initial perturbations to the input signal, the input signal being drawn randomly from a training data set, and the perturbation being fitted to the input signal such that application of the perturbation to the input signal generates a second input signal that is classified differently from the first input signal; providing a subset of the plurality of initial perturbations as the plurality of perturbations; training a classifier based on a plurality of perturbations; Includes.

[0009] The term "classifier" is understood to mean a device that accepts an input signal and provides an output signal that characterizes a classification of the input signal. For this purpose, the classifier can obtain the output signal by feeding the input signal to a machine learning model, in particular a neural network. Furthermore, the classifier can adapt the input signal before feeding it to the classifier, for example before extracting features from the input signal. Additionally, the classifier can also post-process the output signal of the machine learning model.

[0010] The classifier can accept input signals of different modalities, in particular images such as video images, radar images, LIDAR images and / or ultrasound images, as well as infrared camera images, etc. For images, the classifier may preferably include a convolutional neural network.

[0011] Alternatively, the classifier may accept as input audio data in the form of a feature representation of the audio signal, such as a quantized signal or MFCCs. For audio data, the classifier may preferably include a Transformer network or a recurrent neural network, such as an LSTM.

[0012] Alternatively, the input signal may also include data that is a combination of multiple sensor signals and / or sensor signals of different modalities. When both image data and audio data are provided in the input signal, the classifier may preferably include different types of neural networks for processing respective portions of the input signal, for example a combination of one or more convolutional neural networks and one or more transformer networks.

[0013] The classification characterized by the output signal may assign one or more class labels to the input signal. Alternatively or additionally, the output signal may be envisaged to characterize the classification by a form of object detection, particularly when image data is used as the input signal. Alternatively or additionally, the output signal may be envisaged to characterize a semantic segmentation of the input signal.

[0014] For audio signals, the classification may involve assigning a class label to the entire sequence of audio data. Alternatively or additionally, the classification may be in the form of detection of the start and end of particular audio events in the audio sequence (e.g., a siren in a microphone recording).

[0015] Applying a perturbation to an input signal can be understood as overlaying the input signal with the perturbation. For example, when using an image as the input signal, the perturbation may be provided by a small image patch that may be overlaid on a given input image to form a perturbation image. In this case, the location of the patch in the input image may be selected randomly. Alternatively or additionally, the patch may be transformed by an affine transformation before being applied to the input image. For audio data, the perturbation may be overlaid on a random portion of the audio sequence to form the perturbation data.

[0016] It may further be assumed that the perturbation and the input signal have the same dimensions, and applying the perturbation to the input signal may be accomplished by adding the perturbation and the input signal together. For example, when using image data, the perturbation may have a size comparable to the image used for training the classifier. In this case, the pixel values ​​of the perturbation and the image may be added together to form the adversarial examples. Additionally, the perturbation may be scaled by a scalar value before being applied to the input image. A similar approach may be used mutatis mutandis to apply the respective perturbation to the audio signal.

[0017] When a classifier provides an output signal for an input signal, the output signal may include a number of logit values, where each of these logit values ​​corresponds to a class. If the logit value corresponding to the desired class is the largest among the number of logit values, the input signal may be assigned the desired class and the output signal may be deemed to characterize the correct classification.

[0018] For evaluating whether the second perturbation is stronger, i.e. easier to use than the first perturbation to fool the classifier, the second perturbation may be applied to the input signal to obtain a corresponding second perturbed input signal, and the first perturbation may be applied to the same input signal to obtain the first perturbed input signal. The classifier can then use the second perturbed input signal as an input signal to obtain a second output signal, and the first perturbed input signal as an input signal to obtain a first output signal. Thus, the second output signal corresponds to the second perturbation, while the first output signal corresponds to the first perturbation.

[0019] Then, if the logit value of the desired class is smaller in the second output signal than in the first output signal, the second perturbation may be considered stronger than the first perturbation. Similarly, if the maximum logit value of the undesired class in the second output signal is larger than the maximum logit value of the undesired class in the first output signal, the second perturbation may be considered stronger than the first perturbation. Either option may be understood to make the classifier more confident of an incorrect prediction using the second perturbation than the first perturbation.

[0020] Alternatively or additionally, the output signal may include a plurality of probability values, where each of these probability values ​​corresponds to a class, and the first and second output signals may be calculated as before, in which case the second perturbation may be considered stronger than the first perturbation if the probability value of the desired class is smaller in the second output signal than in the first output signal.

[0021] In other aspects, it may further be envisaged that at least one perturbation from the plurality of initial perturbations is provided by randomly drawing a noise signal and providing the noise signal as the perturbation.

[0022] The process of training a classifier benefits from randomly initializing multiple perturbations so that they are highly diverse in structure. The advantage of this is that the variety of perturbations results in the classifier being robust to a wide range of perturbations, making it very difficult to fool the classifier after training.

[0023] In another aspect, it may be further envisaged that at least one perturbation from the plurality of initial perturbations is provided by randomly sampling an input signal from the training data set or the second data set, adapting a plurality of values ​​contained in the input signal, and providing the adapted input signal as the perturbation.

[0024] To obtain an initial set of strong perturbations, the classifier may be pre-trained without defense methods against adversarial examples. Then, one or more second adversarial examples may be obtained from the classifier, for example, by adapting images in the training dataset with known adversarial attack methods, and the perturbations required to induce the one or more second adversarial examples may then be used as the initial set of perturbations for training.

[0025] The advantage of this approach is that the perturbations used for training initialization can already be used for forming adversarial examples. Thus, the step of forming the second perturbation is significantly faster than in gradient-based approaches such as Iterative Fast Gradient Sign Method (I-FGSM) or Projected Gradient Descent (PGD), and the steps required to obtain the second perturbation from the first are also significantly reduced. As a result, this leads to faster training, which in turn leads to the classifier being able to extract information from more training samples during training. This has the effect of further improving the performance and robustness of the classifier.

[0026] If a second data set is used, the type of data may preferably be the same as the type of data in the training data set.

[0027] In another aspect, it may further be envisaged that in the step of adapting the perturbation, the perturbation is applied to a domain of the input signal to obtain a perturbed input signal.

[0028] The advantage of this approach is that the classifier becomes robust to perturbations that do not target the entire input signal. For example, one can imagine using stickers on objects to fool a classifier that receives input image data from a camera. To be robust to these perturbations, the classifier can be trained with perturbations that are applied only to smaller regions of the image. Before application, the perturbations may be transformed by an affine transformation, for example a rotation or a scaling.

[0029] For audio data, a particular period of the audio signal used as the input signal may be perturbed to obtain a perturbed input signal.

[0030] In another aspect, the step of training the classifier comprises the further steps of: a. selecting a first perturbation from a plurality of perturbations and selecting an input signal and a corresponding desired output signal from a training data set; b. obtaining a second perturbation that is stronger than the first perturbation by adapting the first perturbation based on the input signal, the corresponding desired output signal, and the classifier; c. Obtaining a first adversarial example by applying a second perturbation to the input signal; d. adapting the classifier by training the classifier based on the first adversarial examples and corresponding desired output signals to harden the classifier against a second perturbation; e. replacing a first perturbation in the plurality of perturbations by a linear combination of the first perturbation and a second perturbation; f. repeating steps a through e; It can be assumed that the

[0031] Training a classifier with the proposed method can be seen as a form of meta-training, where a set of perturbations and a classifier are trained. The classifier is trained with the goal of achieving the highest possible classification accuracy (also known as performance), but the method also seeks to find a set of perturbations that are as strong as possible.

[0032] Therefore, training the classifier with perturbations of increasing strength allows strengthening the classifier against strong adversarial attacks, making it highly robust.

[0033] The training can be performed iteratively. In each training step, the method can select a batch of training data including one or more tuples of an input signal and a desired output signal. Then, for each input signal, a first perturbation can be randomly drawn with replacement from a set of perturbations. Each such drawn first perturbation can then be adapted to its corresponding input signal to determine a stronger second perturbation.

[0034] The classifier may then be trained by providing adversarial examples obtained from applying the obtained second perturbation to each of those input signals and minimizing a classification loss that characterizes the difference between the obtained output signal and a desired output signal for each input signal.

[0035] Furthermore, each first perturbation may be replaced in the set of perturbations by a weighted sum of itself and the second perturbation. Again, the perturbations are preferably trained to be universally strong. The advantage of this training method is that the classifier has to learn to protect itself against perturbations of increasing strength. As a result, this helps the classifier to become universally robust against adversarial attacks.

[0036] In another aspect, the present invention relates to a computer implemented method for determining a robustness value for a classifier, the classifier being configured to classify an input signal of digital image data and / or audio data, the method comprising the steps of: g. providing a plurality of initial perturbations; h. selecting a first perturbation from the plurality of perturbations and selecting an input signal and a corresponding desired output signal from the test data set; i. obtaining a second perturbation that is stronger than the first perturbation by adapting the first perturbation based on the input signal, the corresponding desired output signal, and a classifier (60); j. replacing a first perturbation in the plurality of perturbations by a linear combination of the first perturbation and a second perturbation; k. repeating steps h to j a predetermined number of times; l. determining a strongest perturbation from the plurality of perturbations on the test data set after completing a predetermined number of iterations; m. determining a proportion of input signals in the test data set for which the strongest perturbation may cause a misclassification by the classifier (60) and providing the determined proportion as a robustness value; Includes.

[0037] The advantage of the proposed method is that after fitting of perturbations to fool the classifier, the classifier is evaluated against the hardest perturbations. As a result, this allows a very accurate estimation of the vulnerability of the classifier to perturbations applied to its potential input signals. For this purpose, the test data set is preferably of the same data type as the training data set used to train the classifier.

[0038] The robustness value may be used as a criterion when deciding whether to use the classifier in a (potentially safety-critical) device. For example, it may be assumed that the classifier should be used to detect pedestrians in an at least partially autonomous vehicle in order to plan collision-free routes. It may be assumed that the classifier must meet a robustness criterion, e.g., that the classifier must be robust to small changes (i.e., perturbations) of the input signal. This robustness criterion may be checked simply by determining whether the obtained robustness value exceeds a predefined robustness threshold. In this case, the classifier may be used in the vehicle. In other cases, it may be trained, for example, with additional training data and / or different perturbations.

[0039] Embodiments of the invention will now be explained in more detail with reference to the following drawings, in which: [Brief description of the drawings]

[0040] [Figure 1] FIG. 1 illustrates a control system that includes a classifier that controls actuators in the environment. [Diagram 2] FIG. 1 illustrates a control system for controlling an at least partially autonomous robot. [Diagram 3] FIG. 1 illustrates a control system for controlling an automated personal assistant. [Figure 4] FIG. 2 illustrates a control system for controlling the access control system. [Diagram 5] FIG. 2 is a diagram showing a control system that controls the monitoring system. [Figure 6] FIG. 1 shows a flowchart of a method for training a classifier. [Figure 7] FIG. 1 illustrates a training system for training a classifier. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0041] Description of the embodiments In Fig. 1, an embodiment of an actuator 10 is shown in its environment 20. The actuator 10 interacts with a control system 40. The actuator 10 and its environment 20 together are referred to herein as the actuator system. Preferably, at equally spaced time points, a sensor 30 senses a state of the actuator system. The sensor 30 may comprise a plurality of sensors. Preferably, the sensor 30 is an optical sensor that takes an image of the environment 20. An output signal S of the sensor 30 (or of each sensor, if the sensor 30 comprises a plurality of sensors) encoding the sensed state is transmitted to the control system 40.

[0042] Thereby, the control system 40 receives a stream of sensor signals S. In dependence on the stream of sensor signals S, a sequence of actuator control commands A is then calculated, which are then transmitted to the actuator 10.

[0043] The control system 40 receives the stream of sensor signals S of the sensors 30 at an optional receiving unit 50. The receiving unit 50 converts the sensor signals S into an input signal x. Alternatively, if there is no receiving unit 50, each sensor signal S may be directly taken as an input signal x. The input signal x may, for example, be provided as an excerpt from the sensor signals S. Alternatively, the sensor signals S may be processed to generate the input signal x.

[0044] The input signal x is then passed to a classifier 60 .

[0045] The classifier 60 is parameterized by a set of parameters Φ, which are stored in and provided by a parameter storage St1.

[0046] The classifier 60 determines an output signal y from an input signal x. This output signal y includes information that assigns one or more labels to the input signal x. The output signal y is transmitted to an optional transformation unit 80, which transforms the output signal y into control commands A. These actuator control commands A are then transmitted to the actuator 10 for controlling the actuator 10 accordingly. Alternatively, the output signal y may be directly taken as the actuator control commands A.

[0047] The actuator 10 receives the actuator control command A and is controlled accordingly to perform an action corresponding to the actuator control command A. The actuator 10 may include control logic that converts the actuator control command A into a further control command that is then used to control the actuator 10.

[0048] In further embodiments, the control system 40 may include the sensor 30. In yet other embodiments, the control system 40 may alternatively or additionally include the actuator 10.

[0049] In one embodiment, the classifier 60 may be configured to identify lanes on the road ahead, for example by classifying the road surface and markings on the road and identifying the lanes as patches of road surface between the markings. A suitable lane for following the selected path may then be selected based on the output of the navigation system, and then depending on the current lane and the target lane, it may be determined whether the vehicle 100 should switch lanes or stay in the current lane. The actuator control command A may then be calculated, for example by retrieving a predefined movement pattern from a database corresponding to the identified action.

[0050] Additionally or alternatively, the classifier 60 may also detect road signs and traffic lights in the environment 20. Upon identifying a road sign or a traffic light, then depending on the identified type of road sign or the identified state of said traffic light, corresponding constraints on the possible movement patterns of the vehicle 100 may be retrieved, for example from a database, a planned trajectory of the vehicle 100 may be calculated according to these constraints, and actuator control commands A may be calculated for maneuvering the vehicle 100 to execute the planned trajectory.

[0051] Additionally or alternatively, the classifier may detect pedestrians and / or vehicles in the environment 20. Upon identifying a pedestrian and / or vehicle, a predicted future behavior of the pedestrian and / or vehicle may be estimated, and then a trajectory may be selected based on the estimated future behavior to avoid a collision with the identified pedestrian and / or vehicle, and actuator control commands A may be calculated to steer the vehicle 100 to execute the trajectory.

[0052] In a further embodiment, it may be envisaged that the control system 40 controls the display 10 a instead of or in addition to the actuator 10 .

[0053] Additionally, the control system 40 may include a processor 45 (or multiple processors) and at least one machine-readable storage medium 46 having stored thereon a number of instructions which, when executed, cause the control system 40 to perform methods in accordance with aspects of the present invention.

[0054] FIG. 2 illustrates an embodiment in which the control system 40 is used to control an at least partially autonomous robot, such as an at least partially autonomous vehicle 100 .

[0055] The sensors 30 may include one or more video sensors and / or one or more radar sensors and / or one or more ultrasonic sensors and / or one or more LiDAR sensors and / or one or more location sensors (e.g., GPS, etc.), some or all of which are preferably, but not necessarily, integrated into the vehicle 100.

[0056] Alternatively or additionally, the sensor 30 may include an information system for determining the state of the actuator system. One example of such an information system is a weather information system that determines the current or future state of the weather in the environment 20.

[0057] For example, using the input signal x, the classifier 60 can detect, for example, an object that is at least partially in the vicinity of the autonomous robot. The output signal y can include information characterizing where the object is located at least partially in the vicinity of the autonomous robot. The actuator control command A can then be determined according to this information, for example to avoid a collision with the detected object.

[0058] Advantageously, the actuator 10 integrated in the vehicle 100 may be provided by the brakes, the propulsion system, the engine, the drive train or the steering of the vehicle 100. An actuator control command A may be determined such that the vehicle 100 controls the actuator (or actuators) 10 such that it avoids a collision with the detected object. The detected objects may be classified according to which the classifier 60 is most likely to recognize them as an object, e.g. pedestrian or tree, and the actuator control command A may be determined depending on the classification.

[0059] In further embodiments, the at least partially autonomous robot may be provided by another kind of mobile robot (not shown), which can move, for example, by flying, swimming, diving or stepping. The mobile robot may in particular be an at least partially autonomous lawnmower or an at least partially autonomous cleaning robot. In all the above embodiments, the actuator control command A may be determined such that the propulsion unit and / or steering and / or braking of the mobile robot are controlled so that the mobile robot can avoid a collision with the identified object.

[0060] In a further embodiment, the at least partially autonomous robot may be provided by a gardening robot (not shown) that uses sensors 30, preferably optical sensors, to determine the state of the plants in the environment 20. The actuator 10 may control a nozzle for spraying a liquid and / or a cutting device, e.g. a blade. Depending on the identified species and / or the identified state of the plant, an actuator control command A may be determined to cause the actuator 10 to spray an appropriate amount of an appropriate liquid towards the plant and / or to cause the actuator 10 to cut the plant.

[0061] 3 shows an embodiment in which the control system 40 is used to control an automated personal assistant 250. The sensor 30 may be an optical sensor, for example an optical sensor for receiving video images of the gestures of the user 249. Alternatively, the sensor 30 may be an audio sensor, for example an audio sensor for receiving voice commands of the user 249.

[0062] The control system 40 then determines an actuator control command A for controlling the automated personal assistant 250. This actuator control command A is determined depending on the sensor signal S of the sensor 30. The sensor signal S is transmitted to the control system 40. For example, the classifier 60 may be configured to execute a gesture recognition algorithm for identifying, for example, a gesture made by the user 249. The control system 40 may then determine an actuator control command A for transmission to the automated personal assistant 250. The actuator control command A is then transmitted to the automated personal assistant 250.

[0063] For example, actuator control command A may be determined according to a user gesture recognized and identified by classifier 60. It may then include information that causes automated personal assistant 250 to retrieve information from a database and output the retrieved information in a form suitable for receipt by user 249.

[0064] In Fig. 4 an embodiment is shown in which the control system 40 controls an access control system 300. This access control system 300 may be designed to physically control access. It may for example include a door 401. The sensor 30 may be configured to detect a scene relevant for deciding whether to grant access. It may for example be an optical sensor for providing image or video data, for example for detecting a person's face. The classifier 60 may be configured to interpret this image or video data, for example by performing an identity match with known persons stored in a database, thereby determining the identity of the person. An actuator control signal A may then be determined depending on the interpretation of the classifier 60, for example according to the determined identity. The actuator 10 may be a lock which is opened or closed depending on the actuator control signal A.

[0065] In Fig. 5 an embodiment is shown in which the control system 40 controls the surveillance system 400. This embodiment is almost identical to the embodiment shown in Fig. 4. Therefore, only the different aspects will be described in detail. In this embodiment, the sensor 30 is configured to detect the scene under surveillance. The control system 40 does not necessarily have to control the actuator 10, but can alternatively control the display 10a. For example, the classifier 60 can determine a classification of the scene, for example whether the scene detected by the optical sensor 30 is suspicious. The actuator control signal A sent to the display 10a can then be configured to cause the display 10a to adjust the displayed content depending on the determined classification, for example to highlight objects deemed suspicious by the classifier 60.

[0066] 6 is a flow chart outlining an embodiment of a method for training a classifier 60 of a control system 40 to be robust against adversarial attacks. The method for training is provided with a classifier 60, preferably a neural network that is pre-trained with a pre-training data set, and a training data set of input signals and corresponding desired output signals.

[0067] In a first step 601, a number of perturbations are initialized. For this initialization, the input signals from the training data set can each be scaled by a random number to form a perturbation. Then, each perturbation is applied to each input signal of the training data set to obtain a number of new input signals. Then, for each perturbation, a first value is obtained by calculating the proportion of new input signals corresponding to the adapted input signals such that the classifier provides a misclassification for all new input signals corresponding to the adapted input signals.

[0068] Alternatively, the input signals can be downsampled and / or cropped to obtain perturbations. These perturbations can be applied to some of the input signals of the training data set to obtain a number of new input signals. For example, when images are used as input signals, these perturbations can be applied to the input signals of the training data set as patches.

[0069] Alternatively, the perturbations may be initialized as noise applied to either the entire input signal or a portion of the input signal, where each of these perturbations is further adapted to an individual input signal from the training dataset to subsequently fool the classifier. This can be achieved, for example, using I-FGSM or PGD attack type algorithms to create adversarial examples.

[0070] Then, in a second step 602, a subset of the training data set is selected to serve as the training batch. For this purpose, a desired amount of input signals and corresponding desired output signals are randomly selected from the training data set.

[0071] In a third step 603, perturbations from the multiple perturbations are randomly drawn by pairwise replacement of input signals from the training batches with corresponding desired output signals.

[0072] In a fourth step 604, each of the derived perturbations is matched to their corresponding input signal and desired output signal, thereby creating adversarial examples for the classifier 60 when the perturbation is applied to each input signal. This can be achieved, for example, by running an I-FGSM or PGD attack type algorithm. For each perturbation, this step returns to a second perturbation matched to the respective input signal.

[0073] In a fifth step 605, each second perturbation is applied to each of the input signals to obtain a number of adversarial examples for the classifier 60.

[0074] In a sixth step 606, a classifier may be trained to predict a desired output signal for the adversarial examples obtained in the previous step. For this purpose, the classifier can be trained using (stochastic) gradient descent or an adapted form thereof (such as Adam). When training the classifier 60 in this way, the classifier predicts an output signal based on the adversarial examples. The predicted output signal is then compared to the desired output signal using a loss function. The classifier 60 is then adapted such that the loss value obtained from the loss function is small when the adversarial examples are reprocessed by the classifier and then the predicted output signal is compared to the desired output signal. For example, when a neural network is used as the classifier 60, for example, a number of parameters of the neural network are adapted according to the negative gradient of the parameters with respect to the loss value.

[0075] Additionally, the classifier may be trained using the input signal and the corresponding desired output signal without modification.

[0076] In a seventh step 607, each randomly drawn perturbation is replaced in the plurality of perturbations by a linear combination of itself and its corresponding second perturbation, for example, by the following relationship: l = (1-σ)·α1+σ·α2, where l is a linear combination, α1 is a perturbation, α2 is a second perturbation, and σ is a pre-determined value. This formulation can be understood as training the perturbation through meta-learning, where σ is analogous to the learning rate in gradient-based learning algorithms.

[0077] In an eighth step 608, steps 2 through 7 are repeated. These steps may be repeated, for example, until the loss value falls below a predetermined threshold. Alternatively, training may be performed for a predetermined number of iterations. When the desired number of iterations has been completed, the classifier 60 may be used in the control system 40.

[0078] In a further embodiment (not shown), multiple perturbations may be obtained to test the robustness of the classifier 60. This method is largely similar to the training process shown in Figure 6, so only the differences are noted here.

[0079] Instead of training the classifier 60 with a set of perturbations (ξ), testing the classifier 60 does not require adapting the classifier's parameters Φ. Instead, the set of perturbations ξ, as is done during training of the classifier, is adapted to a test data set of the same data type as the training data set used to train the classifier 60. After adapting the perturbations for a predetermined number of steps, the robustness of the classifier 60 is determined by applying each perturbation to an input signal from the test data set and providing the thus obtained perturbed input signal to the classifier 60.

[0080] For each perturbation, the number of perturbed input signals misclassified by the classifier 60 may then be determined. The robustness value of the classifier 60 may then be defined as the maximum proportion of perturbed input signals for a single perturbation, across all input signals in the test data set.

[0081] Fig. 7 shows an embodiment of a training system 140 for carrying out the method shown in Fig. 6. A training data unit 150 is provided for the adapted input signal χ i To this end, the training data unit 150 accesses a computer-implemented training database St2 in which at least one set T of training data is stored. The set T includes pairs of input signals and corresponding desired output signals y i The set of training data T may be a full set of training data or a selected batch of training data if training is performed in batches. Additionally, the training data unit 150 receives a set of perturbations ξ from a computer-implemented perturbation database St3. The training data unit 150 then calculates a set of perturbations ξ for the input signal and the desired output signal y i , selecting a perturbation from the set of perturbations ξ, and applying the selected perturbation to the input signal, i Determine.

[0082] Then, the obtained adapted input signal χ i is the adapted input signal χ i Output signal from

number

number

[0083] After the desired number of iterations, the output signal

number

[0084] Then, the output signal

number

[0085] Then, in a further embodiment, the training process is repeated for a desired number of iterations, where an updated set of parameters Φ' is provided in each iteration as a set of parameters Φ by parameter storage St1, and an updated set of perturbations ξ' is provided as a set of perturbations ξ by perturbation database St3.

[0086] Further, training system 140 may include a processor 145 (or multiple processors) and at least one machine-readable storage medium 146 having stored thereon a plurality of instructions that, when executed, cause training system 140 to perform a training method according to an aspect of the present invention.

[0087] In yet another embodiment, the training unit 150 randomly selects at each iteration whether a perturbation should be applied to the input signal. If it determines that a perturbation should not be applied to the input signal, the classifier 60 selects the adapted input signal χ i and no perturbation updates are computed.

Claims

1. A computer-implemented method for training a classifier (60), the classifier (60) being configured to classify an input signal of digital image data and / or audio data, the step of training the classifier (60) being based on a perturbed input signal obtained by applying a perturbation provided from a plurality (ξ) of perturbations to the input signal provided from a training data set, the method comprising: - providing a number of initial perturbations; - fitting a perturbation from the plurality of initial perturbations to an input signal, the input signal being randomly drawn from the training data set, the perturbation being fitted to the input signal such that applying the perturbation to the input signal generates a second input signal that is classified differently from the first input signal; providing a subset of the plurality of initial perturbations as a plurality of (ξ) perturbations; training the classifier (60) based on the plurality (ξ) of perturbations; Including, The step of training the classifier (60) comprises the following further steps: a. selecting a first perturbation from a plurality of perturbations and selecting an input signal and a corresponding desired output signal from the training data set (603); b. Obtaining (604) a second perturbation that is stronger than the first perturbation by adapting the first perturbation based on the input signal, the corresponding desired output signal, and the classifier (60); c. Obtaining 605 a first adversarial example by applying the second perturbation to the input signal; d. adapting (606) the classifier (60) by training the classifier (60) based on the first adversarial examples and the corresponding desired output signals to harden the classifier (60) against the second perturbation; e. replacing the first perturbation in the plurality of perturbations by a linear combination of the first perturbation and the second perturbation (607); f. Repeating (608) the steps from the selecting (603) to the replacing (607); A computer-implemented method comprising:

2. The method of claim 1 , wherein at least one perturbation from the plurality of initial perturbations is provided by randomly drawing a noise signal and providing the noise signal as a perturbation.

3. 2. The method of claim 1 , wherein at least one perturbation from the plurality of initial perturbations is provided by randomly sampling an input signal from the training data set or a second data set, fitting a plurality of values ​​contained in the input signal, and providing the fitted input signal as a perturbation.

4. 4. The method according to claim 2 or 3, wherein in the step of adapting the perturbation, the perturbation is applied to a region of the input signal to obtain a perturbed input signal.

5. 1. A computer-implemented method for providing an output signal (y) characterizing a classification of an input signal (x), comprising: - training a classifier (60) according to the method of any one of claims 1 to 4; - providing said classifier (60) to a control system (40); - obtaining the output signal (y) from the control system (40), the control system (40) providing the input signal (x) to the classifier (60) to obtain the output signal (y); - providing said output signal (y) to control said control system (40); A computer-implemented method comprising:

6. 6. The method according to claim 5, wherein the input signal (x) is obtained based on a signal (S) of a sensor (30) and / or an actuator (10) is controlled based on the output signal (y) and / or a display device (10a) is controlled based on the output signal (y).

7. 1. A computer-implemented method for determining a robustness value for a classifier (60), comprising: The classifier (60) is configured to classify an input signal of digital image data and / or audio data, and the method comprises the steps of: g. providing a plurality of initial perturbations; h. selecting a first perturbation from the plurality of perturbations and selecting an input signal and a corresponding desired output signal from the test data set; i. obtaining a second perturbation, the second perturbation being stronger than the first perturbation, by adapting the first perturbation based on the input signal, the corresponding desired output signal and the classifier (60); j. replacing the first perturbation in the plurality of perturbations by a linear combination of the first perturbation and the second perturbation; k. Repeating the steps from the selecting step h to the replacing step j a predetermined number of times; l. determining a strongest perturbation from said plurality of perturbations on a test data set after completing a predetermined number of iterations; m. determining a proportion of input signals in the test data set for which the strongest perturbation may cause a misclassification by the classifier (60) and providing the determined proportion as a robustness value; A computer-implemented method comprising:

8. The method of claim 7 , wherein at least one perturbation from the plurality of initial perturbations is provided by randomly drawing a noise signal and providing the noise signal as a perturbation.

9. 8. The method of claim 7, wherein at least one perturbation from the plurality of initial perturbations is provided by randomly sampling an input signal from a training data set or a second data set, fitting a plurality of values ​​contained in the input signal, and providing the fitted input signal as a perturbation.

10. A computer program arranged to cause a computer to carry out all the steps of the method according to any one of claims 1 to 9, when said computer program is executed by a processor (45, 145).

11. A machine-readable storage medium (46, 146) having stored thereon a computer program according to claim 10.

12. a control system (40) configured to control an actuator (10) and / or a display device (10a) based on an output signal (y) of the classifier (60), comprising: A control system (40) wherein the classifier is trained by the method of any one of claims 1 to 4.

13. A training system (140) configured to perform the method of any one of claims 1 to 4.

Images