Method and system for classifying sensor data with improved training robustness

Abstract

Some embodiments relate to a method (500) for classifying sensor data with improved robustness against label noise. A predicted label for a novel input can be computed with improved robustness against label noise by estimating the most likely label given repeated application of a base training function to training labels incorporating noise according to the noise level and then applying a base classifier configured according to the base prediction function to the novel input.

Description

Technical Field

[0001] The presently disclosed subject matter relates to a computer-implemented method of classifying sensor data with improved robustness against label noise, a classification system configured to classify sensor data with improved robustness against label noise, and a transitory or non-transitory computer-readable medium. Background Art

[0002] Deep neural networks, despite their widespread empirical success, are notoriously vulnerable to adversarial attacks. Research in this topic has focused on so-called "test-time attacks" (also known as exploratory attacks), where the attacker adversarially manipulates the inputs of a classifier at inference time so that examples are misclassified. Data poisoning attacks - where the attacker manipulates some aspect of the training data so as to cause the learning algorithm to output an incorrect classifier - are also notoriously a problem. For example, one can consider so-called label flipping attacks, where the labels of the training set are adversarially manipulated to degrade the performance of the trained classifier. For example, one can attempt to manipulate the training set so that specific input patterns trigger certain classifications at test time. However, unlike the test-time adversarial setting, relatively little work has been done on building deep classifiers that are robust to data poisoning attacks, where there are reasonably effective defenses to build adversarially robust deep neural networks.

[0003] The performance of any machine learning method ultimately depends on the dataset it is trained on. To improve the quality of machine learning methods (e.g., classification or regression), large amounts of data are often collected. If parts of the data are corrupted, this can lead to compromised machine learning equipment. This corruption can be adversarial. For example, an attacker might purposefully alter parts of the dataset to corrupt the final training product. However, corruption can also occur without malicious intent. For example, a sensor at a data source might be faulty. The fault might even be intermittent.

[0004] To avoid working with corrupted data, sanitation methods can be employed. For example, samples that violate statistical assumptions about the data can be removed. However, this is known to be insufficient, especially in the context of adversarial attacks. Summary of the Invention

[0005] It would be advantageous to have a method for classifying sensor data with improved robustness against label noise (e.g., corrupted training data). Label noise can include adversarial label noise, e.g., labels that are intentionally altered in an attempt to change the behavior of a classifier device trained on the corrupted training data. Label noise can also include non-adversarial label noise, e.g., caused by measurement problems (e.g., caused by a faulty sensor).

[0006] Disclosed herein is a computer-implemented method for classifying sensor data.

[0007] For example, a classification method can predict a label for novel input data based on training data, where multiple training input data are associated with similar labels. The training data may be corrupted. This prediction is made based on a base classifier, e.g., a prediction function that can be trained at least in part on the training data. Interestingly, the base classifier can include two parts: an encoder and a classifier.

[0008] The encoder can be configured to map input data to a potential representation, while the classifier can be configured to be applied to the potential representation. Both the encoder and the classifier can be defined by parameters. Training the base classifier can include optimizing the parameters so that the base classifier fits the training data. Interestingly, in an embodiment, the encoder portion of the base classifier can be considered fixed, while the classifier portion can be retrained.

[0009] For example, the basic training function can be configured to optimize the parameters defining the classifier based on the training data while keeping the parameters of the encoder unchanged. Training only the classifier is faster than training the combination of the encoder and the classifier; the combination has more parameters than its parts.

[0010] Novel input data can be provided. Typically, novel input data is not represented in the training data. However, this is not required, as the labels assigned to novel input data may be suspected to be corrupted.

[0011] Labels for novel inputs with improved robustness to label noise can be computed by repeatedly: 1) training a base classifier on the training data, but with label noise added, and 2) predicting the classification based on the trained base classifier. Based on the repeated predictions, an improved prediction can be determined by determining which label appears most frequently. Training in the case of 1) can be performed efficiently because the base classifier has two parts. Only the classifier part needs to be trained in the case of 1); the encoder part can remain fixed.

[0012] This method can be used in situations with varying noise levels, such as those determined by the expected amount of damage in the tag. For example, a noise level can be determined that flips the tag with a certain probability. For example, with a probability of 10%, the tag might be flipped to add tag noise. Flipping a tag means assigning a different value to the tag. In the case of binary tags, this means assigning the opposite value. For example, if the tag is bit b, which is either 1 or 0, the flipped bit b might be 1-b.

[0013] Binary labels work well in many embodiments. However, this is not necessary, for example, the label may also carry some information in its magnitude. For example, a ranged label may have a value in an interval, for example, the interval [0-1], or [0-2], etc. For example, a positive hazard label may indicate the hazard classification in the sensor data, while the value of the range label indicates the severity. For example, a positive range label may indicate pedestrians indicated in the sensor data, while its magnitude may indicate the number and / or proximity of pedestrians, etc. In an embodiment, discrete (e.g., binary) labels are used, such as on / off labels. Such non-range labels have the advantage that good robustness bounds can be computed with clear semantics, for example, a radius indicating the number of label flips in the training data that will not change the label.

[0014] Input data (e.g., training input data or novel input data) is sometimes referred to as input points. This does not mean that the training inputs necessarily refer to geographic points, but rather that the inputs can be represented as points in a vector space.

[0015] For example, the input data may be a collection of sensor values, e.g., represented as a vector. For example, the input may include one or more of an image, temperature, pressure, etc. The output may classify the input, e.g., determining whether the input represents a specific object, e.g., a pedestrian, a car, a bicycle, etc.; or determining whether certain conditions apply, e.g., whether a machine is in a safe condition, etc.

[0016] Classifying data can also include actions such as semantically segmenting sensor data and detecting objects in the sensor data. For example, segmenting sensor data can be used to identify objects therein, such as traffic participants, such as pedestrians or machine parts. Semantic segmentation can correspond to pixel-by-pixel classification, and detecting objects can correspond to classification into one of two categories, which can be described, for example, as "object present" and "object absent."

[0017] Interestingly, unlike some conventional uses of randomized smoothing for test-time guarantees, the randomization process does not need to be applied to the input of the classifier (e.g., to novel input data), but instead the entire training process of the classifier is randomized. By randomizing the labels during the training process, a robust overall classification can be obtained even when a certain number of labels in the training set are adversarially manipulated (e.g., their predictions are resistant to changes in the presence of label noise).

[0018] Since the final prediction is obtained from multiple predictions obtained from training data that differ by a certain amount of noise, the final prediction is less sensitive to noise, whether it is adversarial or not. However, the prediction is still obtained from a trained classifier and is therefore likely to correctly classify the input. In fact, experiments have confirmed that if there is no label corruption, the robust predictor as in the embodiment is hardly less accurate than the conventional classifier. However, once label noise is added, especially adversarial label noise, the accuracy of the conventional classifier will drop rapidly, while the robust predictor resists this.

[0019] Certain specific choices of base classifiers can provide additional advantages. For example, the classifier portion of the base classifier can include a linear operation configured to be applied to the latent representation (h(x)β). For example, the linear operation can be applied to the latent representation vector via a dot product. The entries in the vector can be considered as parameters to be trained.

[0020] In any case, the basic training function can be configured to apply least squares optimization to derive parameters, however, least squares optimization can be performed particularly efficiently if the classifier includes linear operations. For example, predictions can be obtained as dot products between labels with or without noise and vectors computed from the training data and / or novel input data.

[0021] Note that after the linear operation (e.g., after the dot product), additional steps can be performed. For example, the output of the linear operation can be viewed as a soft prediction, which can be mapped to a hard prediction, e.g., by mapping it to 0 or 1 (whichever is closest) in the case of binary labels, e.g., by rounding towards the label value (e.g., 0 or 1); e.g., for multi-label predictions, by mapping or rounding to a 0-1 vector, etc. The soft values can be trained, e.g., after rounding, to consider reported predictions obtained from novel data inputs and / or specific instances of training data and noise. For example, a classifier can be trained to generate values close to 0 or 1 depending on the desired classification, while the prediction obtained during test time is always greater than 0 whenever it is greater than 1. 1 / 2 can be treated as 1, and whenever it is less than 1 / 2 can be considered as 0.

[0022] In an embodiment, a radius is calculated that indicates the robustness of the predicted label output, e.g., how much noise in the label can be resisted without changing the label output. For example, the radius can include a number of label changes where the predicted label for a novel input will not change.

[0023] Such a radius can depend on the training data, but also on the novel input itself. For some novel inputs, the radius can be large, e.g., for cases where the classification is clear. For some novel inputs, the radius can be smaller, e.g., for cases where the classification is more difficult. Interestingly, the radius can be automatically calculated without knowing which labels may be corrupted.

[0024] Having a radius is an important advantage, among other things, because it allows for the construction of deep classifiers that are provably robust against label flipping attacks, at least up to a limit indicated by the radius. For example, using its output, a classifier according to an embodiment can include a certification guaranteeing that its predictions would not be different if it had been trained on data with a certain number of flipped labels. Interestingly, in some embodiments, such certification bounds can be obtained without additional runtime cost compared to standard classification.

[0025] The guarantees provided are independent of corrupted data. Therefore, it is much more useful than conventional sanitation methods because it protects data from targeted (e.g., adversarial) attacks as well as non-adversarial corruption.

[0026] Interestingly, it may not even be necessary to actually perform repeated training. For example, if the structure of the classifier part of the base classifier is analytically sufficient, the probability that label noise at a certain noise level will lead to a specific classification or may cause the prediction to change can be calculated analytically, or alternatively bounded by a suitable inequality. Instead of actually repeatedly performing training on various instances of labels with label noise, the probability can be directly calculated or at least directly estimated by calculating a bound on the probability. For example, when the classifier is configured to calculate a linear combination of features generated by the encoder, it can be estimated by calculating a bound. Although the linear classifier part has a simple structure, experiments have shown that it does not need to affect the accuracy too much, because the encoder can be much more powerful than the classifier part.

[0027] Although various estimates can be calculated, for example, by approximating the equations in different ways, a particularly advantageous way to calculate the inequalities is by selecting a suitable large class of inequalities that can be indexed by variables. At inference time, bounds can be refined by selecting values of parameters for which the bounds are most stringent. For example, such optimizations can be performed using conventional solvers (e.g., Newton's approximation). Inference time can also be referred to as "test time" or "prediction time."

[0028] In general, there are many ways in which an encoder can be selected. For example, in an embodiment, the encoder can include a neural network, in particular a deep neural network. The neural network can include layers such as convolutional layers, ReLu layers, max pooling layers, etc. The output of the neural network can include multiple values representing features of the input. For example, the neural network can be a so-called feature extractor. For example, the encoder input can include an image, and the output can be a vector of multiple values including multiple features. There are many ways to obtain such an encoder.

[0029] The encoder can be trained during a phase that can be called pre-training, which occurs before the phase in which the system is applied to novel input data. The latter can be called test time or inference time. Unlike conventional systems, inference time can include multiple training rounds. However, it is also possible to train multiple classifiers between pre-training and inference in a phase that can be called multi-classifier training. Note that in a preferred embodiment, there is no multi-classifier training and no additional training is performed during inference. Instead, the system relies on estimating, for example, bounds on the likelihood of a particular label appearing in a particular situation.

[0030] Regardless of whether improvements are made during the classifier training phase or to eliminate the need for training at inference time, the encoder will be trained. However, pre-training can be separate from the use of the encoder. For example, the encoder can be obtained from a third party, for example, as an off-the-shelf feature extractor, such as an image encoder, etc.

[0031] Training the encoder can be performed in various ways. For example, the encoder can be obtained by training a neural network on the training input data portion of the training data. For example, the training can be performed as a so-called autoencoder.

[0032] Unsupervised training methods can be applied to pre-train the parameters of the network. This has the advantage of allowing the use of large amounts of unlabeled data. Training with unlabeled data avoids the possibility of label noise. After pre-training, the network's parameters are set to a potentially good local minimum, from which supervised learning can follow, for example, known as fine-tuning. For example, this fine-tuning can use labeled data, possibly a selected subset of the provided training data. For example, the network can first be trained as an autoencoder and then fine-tuned by combining it with a classifier (e.g., a linear classifier), for example, using supervised learning. To further reduce the impact of corrupted labels while still using supervised learning, the encoder can be trained on more types of labels than will be used at test time. For example, the encoder can even be trained on different types of labels than will be used at test time. For example, if the task is to distinguish between the labels of 1 and 7, the encoder can be trained on digits other than 1 and 7. This ensures that corruption in the 1 / 7 labels will not degrade the encoder, while still allowing it to learn features relevant to the task.

[0033] Training an autoencoder on labeled data can be done simultaneously. For example, the output of the encoder can be trained to allow recovery of the original input, and the combination of the encoder and classifier can be trained on a classification problem.

[0034] Sparsity conditions can be applied to autoencoders. For example, sparsity can be defined in terms of population sparsity and lifetime sparsity. On the one hand, population sparsity ensures a simple representation of the data by allowing only a small subset of outputs to be activated simultaneously. On the other hand, lifetime sparsity controls the activation frequency of each output throughout the dataset, thereby ensuring sparse but high activations of each output. For example, an autoencoder can be a so-called sparse autoencoder.

[0035] Supervised learning of the encoder can be performed on training data that is known to contain no or less corruption. For example, such training data can be selected from a larger pool of training data (e.g., by experts who can review the data). It is also possible to train the encoder and classifier on the same training data as used during inference. This has the disadvantage that corruption in the labels will affect the training of the encoder. On the other hand, the impact of such corruption can be mitigated by test-time countermeasures (e.g., actual or simulated repeated training and testing of the classifier portion of the base classifier).

[0036] For example, supervised learning (e.g., as only learning or as fine-tuning) can be performed by training a combination of an encoder and a temporary classifier, for example, a combination h(x)β can be trained, where x is the input data, h is the encoder, and β is a vector. When training is complete, the vector β can be discarded. Then, in the multi-classifier training phase, or at test time, multiple vectors β' can be trained for many combinations of labels and noise. During the subsequent training period, the encoder can remain fixed. However, in an embodiment, it is not even necessary to calculate multiple vectors β', because instead, the result of applying such training can be estimated, for example, by evaluating a suitable limit.

[0037] There are many options for the classifier component. As mentioned above, a particularly advantageous option is to limit the classifier to linear operations, possibly followed by rounding operations. This has the advantage that predictions are particularly tractable, and repeated prediction results can be obtained without actually performing repeated predictions. However, many other options are possible. They can also be suitable for analysis, or even in the absence of such a requirement, repeated training can be performed. The combination of an encoder component and a classifier component has the advantage that training can be limited to only the classifier component, which requires fewer resources than fully training the encoder and classifier components together. However, in an embodiment, the classifier component is selected to be smaller than the encoder component. For example, the encoder and classifier can be defined by parameters, and the number of parameters defining the encoder can be greater than the number of parameters defining the classifier component. For example, the encoder parameters can be twice or more, ten times or more, etc., as the parameters of the classifier component. For example, the encoder and classifier may comprise neural networks, and the encoder neural network may comprise more nodes and / or layers than the classifier neural network. For example, the classifier may comprise only a single layer. For example, the encoder may be at least twice, ten times, etc., larger. In an embodiment, the classifier comprises linear operations and additional nonlinear operations. The nonlinear operations may be, for example, sigmoid operations. In an embodiment, the classifier may include a single linear operation and a nonlinear operation, for example, a linear operation and a single sigmoid operation for each label output. The nonlinear operation may be a value function, for example, mapping x to max(0, x) or a differenced version (e.g., In(1+e x )wait).

[0038] Therefore, there are different ways to obtain robustness. In the first example, the training of the classifier part is carried out multiple times. Repeated training can be performed at inference time, or pre-calculated and stored and applied at inference time. In the second example, the training of the classifier is carried out multiple times, but the classifier part is arranged so that training and prediction can be combined. For example, in an embodiment, one-pass or online learning is used, wherein the classifier is trained after a single pass through the training data. In the second example, training and prediction of novel inputs are combined so that training is performed at inference time. However, it is noted that part of the training can be pre-calculated. In the third example, the training of the classifier part is not actually carried out multiple times, but instead uses inequalities and the like to calculate an estimate (e.g., a bound) of the likelihood of predicting a specific label in the presence of label noise. The latter depends on the novel input and is therefore carried out at inference time. In the third example, the resources required to obtain the prediction are actually the same as those required to obtain the prediction without additional robustness.

[0039] Referring back to the radius, this number can be used for various purposes. While a radius can be associated with a specific input, the radius for the entire system can be obtained by, for example, sampling multiple novel inputs and averaging the radii. Two types of radii can be referred to as specific radii (e.g., for specific novel input data) and global radii (e.g., the average of multiple specific radii).

[0040] This radius can be used as a metric for quality control—not just for the specific predictions obtained for novel inputs, but also for the quality control of the entire system (e.g., including the encoder).

[0041] For example, an estimate of the number of corrupted labels in the training data can be obtained, and before accepting a predicted label for a novel input, the estimated number of corrupted labels can be compared to a radius. This estimate can be obtained by having an expert or another system review the labels (e.g., a sample thereof). For example, multiple labels can be verified, and the number of incorrect labels can be inferred for the entire training set. If the estimated number of incorrect labels is below a certain radius, then the corrupted labels are unlikely to have changed the prediction, and therefore the prediction has high reliability.

[0042] In practice, a lower bar can be used. For example, if the estimated number of corrupted labels (e.g., incorrect labels) is lower than a multiple of a certain radius, a certain multiple can be defined and the prediction accepted. For example, a multiple of 2 or more, 4 or more, etc. can be used. In this case, the hard guarantee provided by the radius is lost, but it has the advantage of generating fewer false positives.

[0043] If the result is that novel inputs are rejected, or if inputs are rejected that are greater than a certain threshold (such as greater than 10, or greater than 1%, etc.). Steps can be taken to improve the prediction. For example, corrected labels can be obtained, for example, from an expert, and the new labels can be used for supervised learning of the encoder together with a temporary classifier. For example, corrupted labels can be discarded from the training data and the encoder retrained on the corrected training data. On the other hand, it is also possible to train with new or corrected labels during the classifier training phase or at inference time. In this case, the encoder will remain fixed, but the system will nevertheless be improved due to better training of the classifier part. This will even apply to embodiments in which repeated training is estimated, since this estimate depends on the training data, in particular on the features and labels.

[0044] The role of the expert may be reduced. For example, during system operation during a testing phase, where multiple inferences are performed. Novel inputs and corresponding robust predictions can be stored. For example, in the case of an autonomous vehicle, a classifier may be called many times, and many pairs of input data and labels may be collected. An estimate of the number of corrupted labels in the stored data can be obtained. For example, an expert can perform this estimate, but it can also be obtained from an agent. For example, another system may contribute to this estimate. For example, the number of overrides by a user of an autonomous vehicle can be considered a measure of prediction quality.

[0045] If the number of corrupted labels in the stored data is low, in particular, if it is estimated to be below the global radius, then the data can be safely added as additional training material. For example, the additional data can be used for supervised training of the encoder and / or for training during the classifier training phase (if any) or at test time.

[0046] Because the predicted labels have increased robustness, e.g., they are less likely to change in the presence of label noise, it is likely that the number of corrupted labels is lower. This is even more so because the source of this data is likely less likely to be corrupted by an adversary.

[0047] There are various applications for embodiments of the method. For example, a control signal can be determined based on a predicted label of a physical system and / or an autonomous device operating in the physical system. For example, a steering or braking or acceleration signal may depend on a classification of the vehicle environment. Alternatively, it can be the physical system itself that is controlled (e.g., computer controlled). For example, the physical system can be a motor or the like, and the control signal can control the motor or the like. For example, if the physical system is unsafe, it can shut down the system or put it into a stable state. For example, a manufacturing machine can enter a controlled shutdown state, and the autonomous vehicle can transfer control back to a human operator.

[0048] For example, conventional control systems may base their decisions on one or more physical quantities of a physical system. Various physical quantities, such as pressure, density, or rotational speed, can be measured directly and relatively easily, for example, using one or more sensors. However, for other physical quantities, such as fill mass flow or the relative air mass of a motor, direct measurement may be expensive, cumbersome, or sensors for directly measuring the quantity may not be available. To enable control of a machine based on such physical quantities, it is beneficial to classify the machine's state directly based on available sensor data.

[0049] One aspect relates to a classification system configured to classify sensor data with improved robustness against label noise. The system can be implemented in a single device or distributed across multiple devices. For example, the system can be used in, for example, a controller.

[0050] The classification system is electronic. For example, it can be an electronic device, such as a mobile electronic device. The classification system can be a mobile phone, a set-top box, a smart card, a computer, etc.

[0051] Embodiments can be applied to sensor signals received from many types of sensors, such as, for example, video, radar, LIDAR, ultrasonic, motion. For example, the sensor signal can include, for example, an image of the environment (e.g., traffic conditions, for example, a physical system, for example, a manufacturing machine, for example, a robot, for example, a robotic arm) from an image sensor. The classification generated by the embodiments based on the sensor signal and the training data can be used to calculate control signals for controlling a physical system, such as, for example, a computer-controlled machine, such as a robot, a vehicle, a household appliance, a power tool, a manufacturing machine, a personal assistant, or an access control system. For example, the calculation can include classifying the sensor data. For example, a traffic participant can be classified as a pedestrian, in response to which a control signal for the autonomous vehicle is calculated or modified, for example, the speed of the car can be reduced.

[0052] Embodiments of the classification method can be implemented as a computer-implemented method on a computer, or implemented in dedicated hardware, or a combination of the two. The executable code of the embodiments of the method can be stored on a computer program product. Examples of computer program products include memory devices, optical storage devices, integrated circuits, servers, online software, etc. Preferably, the computer program product includes: non-transitory program code stored on a computer-readable medium for carrying out the embodiments of the method when the program product is executed on a computer.

[0053] In an embodiment, the computer program comprises: computer program code, which is adapted to carry out all or part of the steps of an embodiment of the method when the computer program is run on a computer. Preferably, the computer program is embodied on a computer readable medium.

[0054] Another aspect of the presently disclosed subject matter is a method of making a computer program available for download. This aspect is used when the computer program is uploaded to, for example, Apple's App Store, Google's Play Store, or Microsoft's Windows Store, and when the computer program is available for download from such stores. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] Further details, aspects and embodiments will be described, by way of example only, with reference to the accompanying drawings. The elements in the drawings are illustrated for simplicity and clarity and are not necessarily drawn to scale. In the drawings, elements corresponding to elements already described may have the same reference numerals. In the drawings,

[0056] Figure 1 schematically illustrates an example of an embodiment of a classification system,

[0057] Figure 2 schematically illustrates an example of an embodiment of a classification system,

[0058] Figure 3 schematically illustrates an example of an embodiment of a classification system,

[0059] Figure 4 It shows the authentication accuracy against label flipping as q changes.

[0060] Figure 5 An example of an embodiment of a method of classifying sensor data with improved robustness against label noise is schematically illustrated.

[0061] Figure 6 schematically illustrates an example of an embodiment of a classification system,

[0062] Figure 7a schematically illustrates a computer-readable medium having a writable portion comprising a computer program according to an embodiment, Figure 7b Schematically shown is a representation of a processor system according to an embodiment.

[0063] Figure 1-4 List of reference signs in , 6, 7:

[0064] 100 classification system

[0065] 110 Classification Equipment

[0066] 130 processor system

[0067] 140 Memory

[0068] 150 Communication Interface

[0069] 200, 300 classification system

[0070] 210 Input interface for novel input data

[0071] 220 encoder

[0072] 225 Encoder Trainer

[0073] 230 Classifier

[0074] 235 Classifier Trainer

[0075] 237 Noise Adder

[0076] 240 Aggregator

[0077] 250 output interface

[0078] 330 Estimator

[0079] 335 Pre-Computer

[0080] 340 Radius Computer

[0081] 350 output interface

[0082] 600 Classification and Control Systems

[0083] 610 sensor system

[0084] 620 Classification System

[0085] 630 Controller

[0086] 640 Updater

[0087] 1000 Computer readable medium

[0088] 1010 Writable part

[0089] 1020 Computer Programs

[0090] 1110(s) integrated circuit(s)

[0091] 1120 processing units

[0092] 1122 Memory

[0093] 1124 ASIC

[0094] 1126 Communication Components

[0095] 1130 Interconnect

[0096] 1140 processor system. DETAILED DESCRIPTION

[0097] While the presently disclosed subject matter is susceptible to embodiment in many different forms, one or more specific embodiments are shown in the drawings and will be described in detail herein, it being understood that this disclosure is to be considered an exemplification of the principles of the presently disclosed subject matter and is not intended to be limited to the specific embodiments shown and described.

[0098] Hereinafter, for the sake of understanding, the elements of the embodiments are described in operation. However, it will be apparent that the respective elements are arranged to carry out the functions described as being carried out by them.

[0099] Furthermore, the presently disclosed subject matter is not limited to the embodiments, such as described herein or to features recited in mutually different dependent claims.

[0100] Figure 1 An example of an embodiment of the classification system 100 is schematically shown. For example, Figure 1 The classification system 100 can be used to classify sensor data according to one or more labels.

[0101] The classification system 100 may include a processor system 130, a memory 140, and a communication interface 150. The classification system 100 may be configured to communicate with a training data storage device 210. The storage device 210 may be a local storage device of the system 100, such as a local hard drive or memory. The storage device 210 may be a non-local storage device, such as a cloud storage device. In the latter case, the storage device 210 may be implemented as a storage interface to the non-local storage device. Classification systems like the system 100 may be distributed across different devices in various ways. For example, Figure 1 As shown in , the classification device 110 may include a processor system 130, a memory 140 and a communication interface 150. However, other solutions are possible.

[0102] System 110 can communicate internally or with other systems, external storage devices, input devices, output devices, and / or one or more sensors via a computer network. The computer network can be the Internet, an intranet, a local area network (LAN), a wireless local area network (WLAN), or the like. The computer network can be the Internet. The system includes a connection interface configured to facilitate communication within or outside the system as needed. For example, the connection interface can include a connector, such as a wired connector, such as an Ethernet connector, an optical connector, or a wireless connector, such as an antenna, such as a Wi-Fi, 4G, or 5G antenna.

[0103] For example, system 100 may include a communication interface 150. A computer network may include additional elements, such as routers, hubs, etc. In system 100, communication interface 150 may be used to receive input (such as training data or novel input data) and provide output, such as a robust prediction.

[0104] Implementations of system 100 may be implemented in a processor system (eg, one or more processor circuits, such as a microprocessor), examples of which are shown herein. Figure 2 and 3 The functional units shown may be functional units of a processor system. For example, Figure 2 and 3 Can be used as a blueprint of the possible functional organization of a processor system. In these figures, the processor circuit(s) are not shown separately from the units. For example, Figure 2 and 3 The functional units shown in FIG1 may be implemented in whole or in part in the form of computer instructions stored at system 100 (e.g., in electronic memory of system 100) and executable by a microprocessor of system 100. In hybrid embodiments, the functional units are implemented partially in hardware (e.g., as a coprocessor, such as a neural network coprocessor) and partially in software stored and executed on system 100. The parameters and / or training data for the network may be stored locally on system 100, or may be stored in cloud storage.

[0105] Figure 2 An example of an embodiment of a classification system 200 is schematically shown.

[0106] Figure 2 An encoder 220, h and a classifier 230 are shown in FIG. The encoder and classifier are combined to classify the input. For example, novel input data 210 received at the input interface 210 can be mapped by the encoder 220 into a potential representation, such as a feature representation. The classifier 230 can then map the potential representation into a classification prediction. For example, the prediction can be provided on the output interface 250. For example, the input can be received from one or more sensors (e.g., an image sensor, etc.). For example, the input can be received in digital form. For example, the output interface can provide a predicted classification to the controller, for example, to calculate a control signal. For example, the control signal can depend on the classification, such as to achieve a certain goal, or to operate within safety boundary conditions, etc. For example, the input and output interfaces can include APIs, connectors, wireless interfaces, antennas, etc.

[0107] The encoder may comprise a neural network, typically a deep neural network. For example, the encoder may be a feature extractor. It is possible to pre-train the extractor, possibly even independently of the classifier 230 and / or the system 200. However, the system 200 may include an encoder trainer 225 for training the encoder, or for fine-tuning the encoder. This training may be performed with or without a (temporary) classifier (such as the classifier 230). For example, the encoder may be trained in a supervised manner using training data comprising pairs of input data and labels. For example, the encoder may be trained in an unsupervised manner, for example as an autoencoder.

[0108] Training module 225 is optional. For example, the encoder can be pre-trained, which can be done offline. For example, the encoder can be obtained from a third party. On the other hand, including training module 225 has the advantage that fine-tuning, such as additional training, such as additional supervised training, can be performed even after the system has been used for inference.

[0109] Figure 2 The embodiment shown in includes a classifier 230. As discussed, and as will be shown further below, the classifier is also optional.

[0110] Classifier 230 uses the output of encoder 220, e.g., a potential representation of the sensor signal obtained from interface 210, and uses it to perform classification. In an embodiment, classifier 230 can be trained on training data 210, but with noise added thereto. For example, the system can include noise adder 237 and classifier trainer 235. For example, the classifier trainer can be trained multiple times, e.g., at least ten times, e.g., at least 1000 times, etc. The trained classifier can be stored and reused. A trained classifier can also be used once for a single prediction and then discarded.

[0111] For example, the noise adder can add noise to the label, for example, according to a predefined noise level. For example, the noise level can be represented by a parameter q. For example, the label can be changed with probability q. If the label is a binary label, the label can be switched with probability q. For example, if the label is a range label, a certain amount of noise (e.g., Gaussian noise) can be added to it, for example with a mean and variance defined by the noise level. For example, with probability q, such a label can flip between 0 and non-zero. If the label is non-zero, an additional amount of noise can be added to it.

[0112] The classifier trainer can be adapted to the type of classifier. For example, this could be regression learning. This can be used, for example, if the classifier comprises a neural network, perhaps just a single layer. If the classifier comprises linear operations, least squares learning is particularly advantageous because a one-pass learning method is possible.

[0113] The training data may be at least partially precomputed. For example, portions related to the input data may be prepared, such as to which the encoder 220 may be applied. Other computational steps may also be precomputed, for example, if least squares learning is used, the inverse may be precomputed, such as the Moore-Penrose inverse (e.g., the matrix X(X)) may be precomputed. T X) -1 ). For example, if regularization is used, the matrix X(X T X+λI) -1 et al. found that regularization performed better.

[0114] Multiple learned classifiers are applied to novel input data, which can be obtained from interface 210. The results can be aggregated by aggregator 240. For example, the aggregator can count which tags appear most frequently. The results of aggregator 240 can be forwarded to an output interface. The output interface can be a display interface, for example, to display the results on a display. The output interface can be a digital interface, for example, to send the results electronically, etc. The aggregator can also output how many classifiers produced the result, for example, as a percentage, even in the presence of noise. For example, the aggregator can calculate the variance of the predictions of multiple classifiers.

[0115] During the training of multiple classifiers, the encoder is typically not trained further. This has the advantage that the classifiers can be trained faster, potentially much faster than if the encoders were trained together. This works even better if some type of one-pass learning is available. Least squares learning of linear classifiers is a particularly good example of one-pass learning, but many other methods are also known.

[0116] In addition to calculating a robust prediction (e.g., the most common prediction when applying many classifiers that were trained on slightly different noisy data), other variables can also be calculated. For example, the number of labels at the noise level at which the predicted label becomes less reliable can be calculated or estimated. For example, the noise level at which the prediction level is given is used for less than 50% of the classifiers. Instead of 50%, other percentages can be used.

[0117] For example, multiple learned classifiers can also be applied to the training input data to discover training data pairs that are likely to be erroneous, e.g., where the robustly predicted output differs from the label in the training data.

[0118] Figure 3 Different approaches to system 200 are shown in FIG. Figure 3 An example of an embodiment of a classification system 300 is schematically shown. Similar to system 200, system 300 also includes an input interface 210 and an encoder 220. Encoder 220 can be trained based on training data 210 by a training module 225. However, the latter is optional. Similar to system 200, encoder 220 is partially compatible with a classifier, such as classifier 230. In addition, system 200 produces more robust predictions by estimating which label is most frequently generated when multiple classifiers are trained on training labels with additive noise.

[0119] However, unlike system 200, in system 300, multiple predictions are not actually performed. Thus, multiple trained classifiers, whether pre-computed or dynamically calculated, are not required. Instead, estimator 330 directly estimates how likely a particular label prediction is, assuming a certain level of label noise. For example, bounds, such as inequalities, can be calculated that limit these probabilities.

[0120] A particularly efficient way to compute good bounds is not to define all such bounds in advance, but to define a large number of such bounds, for example, indexed by some value t. This approach is used, for example, for the so-called Chernoff bound. At inference time, the best inequality can be selected, giving the tightest bound.

[0121] Assuming a certain level of label noise, it is easier to estimate the number of times a particular label will appear if the classifier is small. This is similar to system 200, where it is easier to repeatedly train the classifier 230 if the classifier 230 is smaller. For example, one may choose to include a linear operation in the classifier 230. Such a linear classifier is suitable for training in one pass, for example, using a least squares method, possibly with regularization. It can be shown that a particular prediction for a particular novel input reduces to a dot product with respect to the training labels. Thus, it can be taken as (α T y) to obtain predictions, where y represents the training label and α T depends on the training input and on the novel input. With this expression, we can calculate the T y) is less than 1 / 2, and thus if the vector y now contains noise, a 0 label will result. Likewise, the probability that it is greater than 1 can be estimated directly rather than simulated, and thus a 1 label will result. If this is a range label, then this (α T y ≥ γ) for any desired γ. Instead of estimating it directly, one can also use α T y represents a fast simulation.

[0122] As noted, systems 200 and 300 may be implemented on a computer or combination of computers (eg, on system 100).

[0123] In various embodiments of systems 100, 200, and 300, the communication interface can be selected from a variety of alternatives. For example, the interface can be a network interface to a local area network or a wide area network (e.g., the Internet), a storage interface to an internal or external data storage device, a keyboard, an application programming interface (API), etc.

[0124] Systems 100, 200, and 300 may have a user interface that may include well-known elements such as one or more buttons, a keyboard, a display, a touch screen, a pointing device such as a mouse, etc. The user interface may be arranged to accommodate user interaction for configuring the system, training a network on a training set, or applying the system to new sensor data, etc.

[0125] The storage device may be implemented as an electronic memory, such as flash memory, or a magnetic memory, such as a hard disk. The storage device may include multiple discrete memories, such as 140 and 210, that together constitute the storage device. The storage device may include temporary storage, such as RAM. The storage device may be a cloud storage device.

[0126] Systems 100, 200 and 300 may be implemented in a single device. Typically, systems 100, 200, 300 each include one or more microprocessors that execute appropriate software stored at the system; for example, the software may have been downloaded and / or stored in a corresponding memory, such as a volatile memory such as RAM or a non-volatile memory such as flash memory. Alternatively, the system may be implemented in whole or in part in programmable logic, for example as a field programmable gate array (FPGA). The system may be implemented in whole or in part as a so-called application-specific integrated circuit (ASIC), for example, an integrated circuit (IC) customized for its specific purpose. For example, the circuitry may be implemented in CMOS, for example, using a hardware description language such as Verilog, VHDL, etc. In particular, systems 100, 200, 300 may include circuitry for evaluating a neural network.

[0127] The processor circuit may be implemented in a distributed manner, for example, as multiple sub-processor circuits. The storage device may be distributed across multiple distributed sub-storage devices. Some or all of the memory may be electronic memory, magnetic memory, etc. For example, the storage device may have volatile and non-volatile portions. Part of the storage device may be read-only.

[0128] Figure 6 An example of an embodiment of a classification system 600 is schematically shown, illustrating various applications and improvements. Figure 6 The system 600 shown in FIG. 6 includes a sensor system 610, a classification system 620, and a controller 630. For example, the sensor system 610 may include a camera, LIDAR, etc., and generates a sensor signal for the classifier 620. For example, the sensor signal may include an image. The controller 630 uses the classification of the system 620. For example, the system 600 may be used in an autonomous machine (e.g., an autonomous vehicle).

[0129] In an embodiment, the controller 630 may receive an indication of the quality of the classification from the classifier 620. For example, the controller may receive an estimate or bound of the radius and only accept the prediction if the latter is less than the estimated number of corrupted labels, or only accept if the latter is less than a multiple of the estimated number of corrupted labels. For example, the controller 630 may receive the variance of the predicted classification, e.g., as predicted based on noisy labels. The controller may be configured to accept the classification only if the reported variance is below the bound.

[0130] In an embodiment, system 600 includes an updater 640. For example, updater 640 can be configured to record instances where a classification is rejected due to too low quality. Such examples can be added to the training data, but with corrected labels. Furthermore, after corrupted labels are discovered, the training data can be improved by correcting them. For example, the improved and / or expanded training can be used to train or retrain, for example, to fine-tune, an encoder used in system 620.

[0131] For example, the updater 640 may store not only examples of classifications that were not accepted, but also, or only cases where the classification was accepted. The number of errors (e.g., corrupted labels) in the recorded examples may be estimated, for example, automatically or by a human expert. The updater 640 may be configured to retrain the encoder using the recorded input sensor data together with the predicted labels if the estimated number of erroneous labels in the recorded data is lower than the radius of the system. In the latter case, the radius may be taken as a global radius.

[0132] In an embodiment, the sensor system 610 , the classifier 620 , and the controller 630 may be implemented in a single device (eg, a car), while the updater 640 is implemented on one or more computers in the cloud.

[0133] Several further optional refinements, details and embodiments are illustrated below.

[0134] Robustness against label flipping attacks can be achieved by randomized smoothing during training. For example, using a method that includes a randomized smoothing process, a general method for verifying robustness against label flipping attacks, or indeed data poisoning attacks in general, is achieved. This general method can be improved for some specific cases, such as label flipping attacks using least squares classification.

[0135] For simplicity, we first describe an embodiment of a binary-valued function. For example, this can be used for a binary classification problem. The method can be extended to a multi-class setting, as further expanded below.

[0136] A randomized smoothing operator can be constructed that transforms a binary-valued function and smoothing metrics (where ∫ x μ(x)dx=1) maps to the expected value of φ in the case of μ, e.g.

[0137]

[0138] We can use g(μ, φ) to represent a "hard thresholded" version of G, i.e., predicting a smoothed version of the most likely output given μ. In the binary case, this can be done by checking whether the expectation is greater than 1 / 2:

[0139] g(μ,φ)=1{G(μ,φ)≥1 / 2}. (2)

[0140] Intuitively, for two very similar smoothness metrics μ, It can be expected that for most φ, g(μ, φ) = g(ρ, φ). In addition, as G(μ, φ) approaches 0 or 1, the degree to which μ and ρ can differ and still preserve the property should increase, because this increases the "margin" of the function φ relative to the metric μ being 0 or 1, respectively. More formally, the general randomized smoothness guarantee can be defined as follows:

[0141] Definition 1: Let μ, are two smooth metrics relative to X. Then the randomized smoothing robustness guarantee is the distance metric d(μ, ρ) and the function The specification makes it possible for all g(μ,φ)=g(ρ,φ) whenever d(μ,ρ)≤f(G(μ,φ)). (3)

[0142] Example 1 Randomized smoothing ensures that a smoothness metric can be used Gaussian around the point x0 to be classified, Gaussian around the perturbed example and can take the form

[0143] f(p)=|Φ -1 (p)| (4)

[0144] where KL(·) represents the KL divergence, and Φ -1 Represents the inverse CDF of the Gaussian distribution.

[0145] Therefore, randomized smoothing of continuous data can be performed with Gaussian noise, but this is by no means a requirement. For example, in the following example, it is used as an alternative method for handling discrete variables.

[0146] Example 2 Randomized smoothing guarantees can be used with respect to the basic input x0∈{0,...,K} d The factored smoothing measure defined For α∈[0,1], in

[0147]

[0148] ρ is defined similarly for the perturbation input x1. Their guarantees can be taken as follows

[0149] f(p)=g α,K,d (max(p,1-p)). (5)

[0150] In words, a smooth distribution can be one where each dimension is independently unaffected by probability α, or uniformly randomly perturbed to different values with probability 1-α. Here, g is a function that can be precomputed combinatorially: for example, g α，K，d (p) can be defined as μ α,K and ρ α,K The maximum number of dimensions that can be different ensures that μ α,K The set with metric p in ρ α，K The case has at least It can be shown that this value is independent of x0 and x1 and depends only on α, K, and d.

[0151] Yet another example is a form of randomized smoothing that does not require strict assumptions about the distribution, but still allows similar guarantees to be provided.

[0152] Example 3 (General bound from) Given any two smooth distributions μ, ρ, we have a general randomized smoothing robustness certification:

[0153] d(μ,ρ)=KLρμ,

[0154] The last two randomized smoothing guarantees can be applied particularly advantageously to binary variables, e.g., labels in binary classification tasks.

[0155] Note that, in practice, for most classifiers, the function value cannot be calculated exactly. For example, one can resort to Monte Carlo approximation instead. For example, one can perform a type of randomized smoothing in which a number of random samples are drawn from μ and a model is constructed using these samples about More precisely, in the hard prediction and otherwise in the case of an upper bound, the bound should be about This ensures that in both cases we underestimate the true radius of the certified robustness of the classifier. After obtaining these estimates, for example, as in this way, they can be plugged into the corresponding randomized smoothed robustness guarantee to provide a high-probability certified robustness bound for the classifier.

[0156] Randomized smoothing can be applied at test time, e.g., where the function is the classifier we wish to smooth. This classifier can be a deep classifier, for example, a neural network. Note that it is not required that the function It is a deep classifier, and it is not required to be a traditional classifier at all. It can be applied to any mapping from some input space to one or more outputs (e.g., binary outputs).

[0157] However, in the embodiments, a very different form of randomized smoothing is used. As a trained classifier, we can think of is any learning algorithm that can be understood as inputting a training dataset and additional examples x without corresponding labels n+1 , aiming to predict it. In other words, The combination goal is to first The classifier is trained on , and then outputs the probability of a new example x in {0,1} n+1 Therefore, the test-time output can be considered a function of both the test-time input and the training data used to generate the classifier. This perspective allows us to reason about how changes in the training data affect the classifier at test time.

[0158] When randomized smoothing is applied to this setting, the labels y in the training set can be 1:nThe randomization is performed instead of randomization relative to the input. Similar to the previous application of randomized smoothing, if the predictions from this process have a large margin when training with these randomly perturbed labels, it will confer a certain degree of adversarial robustness to a certain number of adversarially corrupted labels. In embodiments, randomization relative to the training input can be combined with randomization relative to the novel input x n+1 The random combination.

[0159] To formalize this intuition, consider two different assignments Y1,Y2 ∈ {0, 1} among the n training labels n , which differ by exactly r labels. Let μ and ρ be the distributions obtained by independently flipping each label in Y1 and Y2 with probability q, and let μ i , ρ i is a single label y i distribution.

[0160] Clearly, as r increases, d(μ,ρ) should also increase. In fact, it can be shown mathematically that the closed form of the exact KL divergence between these two distributions is Inserting the robustness guarantee (6), we obtain if only

[0161]

[0162] in This implies that for any training set and test point, as long as (8) satisfies, the prediction of g will not change until r flips. This bound behaves precisely as a function of expectation as p and q: as the margin p increases and as the variance of the label noise q increases, g will remain constant for more label flips, effectively overwhelming the difference between Y1 and Y2.

[0163] This method has a simple closed form, but the bounds are not strict. Strict bounds can be derived via combinatorial methods. By precalculating the quantity from equation (5) for each r You can is compared to each of these and thereby certified robustness to the highest possible number of tag flips. This computation may be more expensive, but it provides significantly tighter robustness guarantees for Given a bound of , approximately twice the number of label flips is authenticated. Experiments use this tighter bound, but emphasize that meaningful results can be achieved even with a looser bound, which is, moreover, orders of magnitude cheaper to compute.

[0164] If you naively think: Considering that a complete training-plus-single-prediction process would require us to train multiple classifiers, by randomly sampling the label y multiple times, all to predict a single example, the algorithm presented in the previous section is clearly impractical. However, in embodiments, the architecture and training process can be constrained in a way that drastically reduces this cost, bringing it in line with the cost of classifying a single example. In fact, even if the same results are achieved, the training of multiple classifiers can be eliminated altogether.

[0165] For example, given a training set such as We assume that there is a feature map It will typically consist of a deep network pre-trained on a similar task, or possibly trained in an unsupervised manner on x. 1:n Training on labels can then be simplified to learning to extract the latent representation h(x i ) to label y i For example, you can use the classifier The latter training process is restricted to be performed via a linear least squares solution.

[0166] Assuming the existence of the encoder h is not such a big assumption. For example, using a pre-trained model in a meta-learning setting shows the transferability of pre-trained features.

[0167] Given the feature map h, is the training point feature, and y=y 1：n ∈{0, 1} n In this example, the training process involves finding a least squares fit to the training data, for example, by finding the parameters via the normal equations

[0168]

[0169] Can be achieved through a linear function Making predictions on novel inputs. We found that fitting classification tasks with the least squares loss works quite well in practice.

[0170] The prediction can be expressed as a function of the label, for example, in the so-called kernel representation. For example, in the case of least squares, this reduces the prediction to a linear function of y, which simplifies randomization with respect to the label. Specifically, let α = X(X T X) -1 h(x n+1 ) T , (10)

[0171] predict It can be equivalently expressed by α T y is given.

[0172] The latter can be viewed as a kernel representation of a linear classifier. Thus, we can compute α once and then randomly sample many different sets of labels in order to construct a standard randomized smooth bound. Alternatively, we can precompute the term representing the input, in this case X(X T X) -1 item and reuse it for multiple test points.

[0173] For better adjustment Regularization

[0174] It may be the case that the training points do not perform well for linear regression in the feature space. To address this, one can instead solve the least squares Regularized version. In this type of training, there are also pre-computed closed-form solutions to the problem. For example, one can solve

[0175] α=X(X T X+λI) -1 h(x n+1 ) T (11)

[0176] The rest of our algorithm remains unchanged. The regularization parameter can be determined empirically, but a good example is, in is an estimate of the variance, and κ(·) is the condition number equal to the ratio of the largest and smallest singular values. The (1+q) term is added to help account for the variance caused by label flipping. For example, it can be written as To estimate the variance.

[0177] What makes randomized smoothing methods even more convincing in this setting is that, due to the linear structure of the prediction, one can actually abandon sampling-based methods entirely and directly form bounds on the tail probabilities, e.g., using Chernoff bounds. Specifically, since whenever α T When y ≥ 1 / 2 and 0 otherwise, the basic prediction function will be a new example x n+1 predicts label 1, so we can easily form an upper bound on the probability that a hard prediction will lead to one prediction or the other via the Chernoff bound. That is, we can form an upper bound on the probability that the classifier outputs label 0 by

[0178]

[0179] Conversely, the probability that the classifier outputs label 1 has a similar upper bound

[0180]

[0181] This is the same bound as the previous case evaluated at -t, so to determine both the label that the smoothed classifier will predict and an upper bound on the probability of the opposite prediction, we can solve the unconstrained minimization problem with respect to t and then let the label of t dictate which label to predict and let the value of t determine the bound.

[0182] Note that it will not always be the case that one of the predictions is bounded by 1 / 2, but one of the bounds will often be empty; therefore, we can define the classifier's predictions in terms of non-empty bounds. The optimization problem with respect to t can be solved by noting that the objective is logarithmically convex in t. That is, the logarithm of the expression above can be minimized:

[0183]

[0184] This is a one-dimensional convex optimization problem that can be solved using many methods, for example, using Newton's method.

[0185] Algorithm 1 below is an embodiment of randomized smoothing for label flipping robustness. In this embodiment, no sampling is actually performed in the final algorithm. Instead, the expected bound on the probability is computed directly, in this case via the Chernoff bound, without resorting to Monte Carlo approximation. Therefore, this method is able to generate certifiable robust predictions against label noise using the same complexity as traditional prediction approximations. This assumes that the cost of the model is dominated by computing the probability of all training points x i The characteristic h(x i ) costs.

[0186] Algorithm 1: Randomized smoothing for label flip robustness

[0187] Input: Feature Map Noise parameter q; training set (potentially with adversarial labels); additional input x for prediction n+1

[0188] 1. Precompute the matrix M:

[0189] M=X(X T X+λI) -1 (14)

[0190] Where X≡h(x 1：n )as well as

[0191] a. Calculate the α vector

[0192] α=Mh(x n+1 ) T (15)

[0193] b. Calculate the optimal Chernoff parameter t via optimization, for example, using Newton's method

[0194]

[0195] And make

[0196] Output: Prediction and the certification radius, thus guaranteeing that the prediction will remain constant until r flips, where

[0197]

[0198] (or a larger radius using the exact method)

[0199] Various experiments were conducted. For example, experiments were conducted on the MNIST 1 / 7 test cases and the Dogfish binary classification challenge, the latter of which was adapted from ImageNet. For each dataset, we calculated the certified test set accuracy under r label flips. That is, for each possible number r of adversarial flips, we plotted the fraction of the test set that was both correctly classified and certified to be unchanged under at least r flips.

[0200] Because these datasets represent binary classification tasks, it is technically possible to achieve a certified accuracy of 50% for r = ∞ by holding g constant. This would seem to imply that any number of flips certified with less than 50% accuracy is meaningless. However, we note that accuracy is a function of the transferability of the pretrained feature embedder; the certified accuracy for a given r and q can be significantly improved with more carefully chosen pretrained networks. Additionally, we observe that our certification procedure still achieves highly significant accuracy with zero label flips, e.g., 97.6% for MNIST with q = 0.1.

[0201] Figure 4 A graph 400 is shown, plotting the certified accuracy of adversarial label flipping as q varies on the MNIST 1 / 7 test set. The noise level hyperparameter q controls the accuracy / robustness tradeoff. The solid line represents the certified accuracy, except for the undefended classifier, which represents the upper bound. The dashed line is the overall non-robust accuracy of the classifier.

[0202] exist Figure 4 Shown in FIG. 4 is a y-axis 420 representing the authentication accuracy. The x-axis 411 shows the number of label flips. The x-axis 410 shows the number of label flips as a fraction of the training set.

[0203] Line 421 shows the authentication accuracy for q = 0.1 and regularization parameter λ = 28241. There is a sharp drop in line 421 as we move from 0 to 1 label flip. To make this more visible, reference line 431 shows the same model with the same parameters, which is constant for the 0 label flip case. Notice that the authentication accuracy drops dramatically with just a single label flip, highlighting the importance of robust prediction.

[0204] Lines 422 and 432 correspond to lines 421 and 431, but for q=0.2 and λ=30809.

[0205] Lines 423 and 433 correspond to lines 421 and 431, but at q = 0.4 and λ = 35944.

[0206] Line 435 is a reference line where g is a constant and has a constant authentication accuracy of 0.5.

[0207] Line 430 is a reference line showing the performance where the model is trained conventionally without robustness improvements. Note that it has the best accuracy. Although not plotted, its accuracy drops rapidly as the number of label flips increases.

[0208] Thus, lines 431, 432, and 433 show the actual performance of embodiments of the method where q = 0.1, 0.2, and 0.4 are used with zero label flips. As expected, the (q = 0.1) line is above the (q = 0.2) line, which is above the (q = 0.4) line. Interestingly, even after one flip, the performance of q = 0.4 is better than that of q = 0.1, and continues in this manner.

[0209] The MNIST 1 / 7 dataset consists only of classes 1 and 7—a total of 13,007 training points and 2,163 test points. A convolutional neural network is trained on the other eight MNIST digits to learn a 50-dimensional feature embedding, and then computed as described above. The Chernoff bound on . In this example, the encoder had not previously seen examples of 1s and 7s, but had learned features from other digits. This approach eliminates the possibility that a corrupted 1 / 7 training point could corrupt the encoder.

[0210] Figure 4 The certification accuracy on the test set is shown in Figure 1 for varying probability q. As in previous work on randomized smoothing, the noise parameter q balances the trade-off between robustness and accuracy. As q increases, the margin required to authenticate a given number of flips is will decrease. On the other hand, this results in noisier training labels, which reduces the margin. The plot indicates the accuracy on the certified test set, as well as the non-robust accuracy of each classifier. Line 435 represents the performance of the constant classifier, assuming equal representation of the classes, and is used as a reference.

[0211] Additional experiments found that the unregularized solution achieved almost 100% non-robust accuracy, but achieved lower robustness.

[0212] A similar experiment was performed on the pike test case. In this binary classification test, ImageNet images labeled as either dog or fish were selected. In this case, the encoder was a ResNet-50 trained on the standard ImageNet training set, but all images labeled as any kind of dog or fish were removed.

[0213] Although the notation and algorithm are slightly more complex, all the embodiments discussed above can be extended to a multi-class setting. In this case, one can consider class labels y∈1,...,K and find some smooth predictions so that the classifier's prediction for a new point will not change if the labels in the training set are flipped by some number r.

[0214] For example, consider the classifier Outputs the index of one of the K categories. In this formulation, for a given category c∈[K], we can define

[0215]

[0216] Among them if Output category c, then φ c (x) = 1 {φ(x) = c} is the indicator function. In this case, the hard threshold g can be evaluated by returning the class with the highest probability. That is,

[0217] g(μ, φ) = argmax c G(μ, φ, c) (22).

[0218] In this case as well, the model can be partially linearized as in the binary case, which abandons the need to actually perform random sampling at all, but instead uses the Chernoff bound to directly form the bounds on the randomized classifier. Using the same notation, the equivalent least squares classifier for the multiclass setting can be found for some set of weights

[0219]

[0220] where mY∈{0,1} n×K is binary, where each row is equal to the one-hot encoding of the class label. Note that the resulting Now we have the matrix Refers to row i. At the prediction time, a new point x n+1 The predicted class of can be given by the prediction with the highest value, e.g.

[0221]

[0222] Alternatively, following the same logic as in the binary case, the same prediction can be written in terms of the α variable as

[0223] where Y i Indicates Y i The i-th column of .

[0224] In the randomized smoothing setting, the class of any label can be flipped with probability q, with the replacement label chosen uniformly at random from the remaining K-1 labels. Suppose the predicted class label is i, we can form a bound on the probability of all replaced classes 'i

[0225] P(α T Y i <α T Y i′ ) (26).

[0226] By the Chernoff bound, we have

[0227]

[0228] Depends on y j =i,y j =i′ or y j ≠i and y j ≠i′, random variable Y ji -Y ji′ Takes three different distributions. Specifically, the variable can take +1, 0, -1 terms with associated probabilities

[0229]

[0230] Combining these cases directly to the Chernoff bound gives

[0231]

[0232] Again, this problem is convex in t, and so can be solved efficiently using, for example, Newton's method. And again, since the opposite case can be computed via the same expression, we can similarly optimize this in an unconstrained manner. Specifically, we can do this for every pair of categories i and i', and return the i that gives the minimum lower bound on the worst-case choice of i'.

[0233] To compute the actual authentication radius, for the case of K classes, the following KL divergence bound is derived. Let μ, ρ, and μ i , ρ i As defined above, except that when the label flips with probability q, it is changed uniformly randomly to one of the other K-1 categories. In addition, let Y1 i becomes the i-th element of Y1, meaning it is the “original” category that may or may not flip when sampling from μ. First, note that each dimension of the distributions μ and ρ is independent, with

[0234]

[0235] Inserting the robustness guarantee (6), we obtain if only

[0236]

[0237] Setting K = 2 restores the divergence term (7) and the bound (8).

[0238] Figure 5 An example of an embodiment of a method 500 is schematically shown for classifying sensor data with improved robustness against label noise. The method 500 may be computer-implemented and include

[0239] - Provide (510) training data The training data includes multiple training input data (x i ) and the corresponding label (y i ), the training input data represents the physical properties of the physical system obtained from one or more sensors, and the basic classifier defined for the training data includes an encoder (h) and a classifier (β), the encoder is configured to convert the input data (x i ) is mapped to the latent representation (h(x i ), the classifier is configured to be applied to the latent representation, wherein the encoder is defined by pre-trained parameters, and the basic training function is configured to optimize the parameters defining the classifier based on the training data,

[0240] - providing (520) novel input data (x n+1 ),

[0241] - defining (530) the noise level (q) of the plurality of labels,

[0242] - Computing (540) predicted labels for novel inputs with improved robustness to label noise, including estimating the most likely label given repeated application of a base training function to training labels incorporating noise according to the noise level and subsequently applying a base classifier configured according to the base prediction function to the novel input.

[0243] Calculating the predicted labels can be performed in at least two ways. For example, one method may include repeatedly applying a basic training function to training labels incorporating noise according to the noise level, and then applying a basic classifier configured according to the basic prediction function to novel inputs.

[0244] On the other hand, the predicted labels can also be calculated by

[0245] - Estimating the most likely label in the case of repeated applications comprises evaluating (542) one or more inequalities to obtain bounds on the likelihood.

[0246] The encoder may comprise a neural network. Possibly, the classifier portion of the base classifier may also comprise a neural network. However, in typical embodiments, the encoder comprises a neural network, while the classifier portion comprises no, or at most a shallow, neural network, e.g., a single sigmoid or single layer. Machine learning methods may be used to train neural networks, e.g., the encoder neural network.

[0247] For example, accessing the training data and / or receiving the input data can be performed using a communication interface (e.g., an electronic interface, a network interface, a memory interface, etc.). For example, parameters, such as parameters of the network, can be stored or retrieved from an electronic storage device (e.g., a memory, a hard drive, etc.). For example, applying the neural network to the data in the training data and / or adjusting the stored parameters to train the network can be performed using an electronic computing device (e.g., a computer). The encoder can also output the mean and / or variance instead of directly outputting the mean and / or variance. In the case of the mean and variance, to obtain the output, a Gaussian of the definition can be sampled.

[0248] The encoder neural network can have multiple layers, which can include, for example, convolutional layers, etc. For example, the neural network can have at least 2, 5, 10, 15, 20, or 40 hidden layers, or more, etc. The number of neurons in the neural network can be, for example, at least 10, 100, 1,000, 10,000, 100,000, 1,000,000, or more, etc.

[0249] Many different ways of carrying out the method are possible, as will be apparent to one skilled in the art. For example, the order of the steps may be carried out in the order shown, but the order of the steps may vary, or some steps may be carried out in parallel. Furthermore, other method steps may be inserted between steps. Inserted steps may represent refinements of a method such as that described herein, or may be unrelated to the method. For example, some steps may be carried out at least partially in parallel. Furthermore, a given step may not be fully completed before starting the next step.

[0250] Embodiments of the method may be performed using software (e.g., together with one or more of portions 541 and 542) comprising instructions for causing a processor system to perform method 500. The software may comprise only those steps employed by a particular sub-entity of the system. The software may be stored in a suitable storage medium such as a hard disk, floppy disk, memory, optical disk, etc. The software may be sent as a signal along a line, or wirelessly, or using a data network (e.g., the Internet). The software may be available for downloading and / or remote use on a server. Embodiments of the method may be performed using a bitstream that is arranged to configure programmable logic (e.g., a field programmable gate array (FPGA)) to perform the method.

[0251] It will be appreciated that the presently disclosed subject matter also extends to computer programs, in particular computer programs on or in a carrier, suitable for putting the presently disclosed subject matter into practice. The program may take the form of source code, object code, a code intermediate source and object code, such as a partially compiled form, or any other form suitable for use in an implementation of an embodiment of the method. An embodiment relating to a computer program product comprises computer executable instructions corresponding to each processing step of at least one of the methods described. These instructions may be subdivided into subroutines and / or stored in one or more files that may be statically or dynamically linked. Another embodiment relating to a computer program product comprises computer executable instructions corresponding to each device, unit and / or part of at least one of the systems and / or products described.

[0252] Figure 7aA computer-readable medium 1000 is shown having a writable portion 1010 including a computer program 1020, which includes instructions for causing a processor system to perform a classification method according to an embodiment. The computer program 1020 can be embodied as a physical label on the computer-readable medium 1000, or by magnetization of the computer-readable medium 1000. However, any other suitable embodiments are also conceivable. In addition, it will be appreciated that although the computer-readable medium 1000 is shown here as an optical disk, the computer-readable medium 1000 can be any suitable computer-readable medium, such as a hard disk, solid-state memory, flash memory, etc., and can be non-recordable or recordable. The computer program 1020 includes instructions for causing a processor system to perform the classification method.

[0253] Figure 7b A schematic representation of a processor system 1140 is shown according to an embodiment of the classification system. The processor system includes one or more integrated circuits 1110. Figure 7b Schematically illustrates the architecture of one or more integrated circuits 1110. Circuit 1110 includes a processing unit 1120 (e.g., a CPU) for running a computer program component to perform a method according to an embodiment and / or implement its modules or units. Circuit 1110 includes a memory 1122 for storing programming code, data, etc. A portion of memory 1122 may be read-only. Circuit 1110 may include a communication element 1126, such as an antenna, a connector, or both. Circuit 1110 may include a dedicated integrated circuit 1124 for implementing some or all of the processing defined in the method. Processor 1120, memory 1122, dedicated IC 1124, and communication element 1126 may be connected to each other via an interconnect 1130 (e.g., a bus). Processor system 1110 may be arranged for contactless and / or contactless communication using an antenna and / or connector, respectively.

[0254] For example, in an embodiment, the processor system 1140 (e.g., a classification system) may include a processor circuit and a memory circuit, wherein the processor is arranged to execute software stored in the memory circuit. For example, the processor circuit may be an Intel Core i7 processor, an ARM Cortex-R8, or the like. In an embodiment, the processor circuit may be an ARM Cortex M0. The memory circuit may be a ROM circuit or a non-volatile memory, such as a flash memory. The memory circuit may be a volatile memory, such as an SRAM memory. In the latter case, the device may include a non-volatile software interface, such as a hard drive, a network interface, or the like, which is arranged to provide the software.

[0255] As used herein, the term "non-transitory" will be understood to exclude transient signals, but rather to include all forms of storage devices, including both volatile and non-volatile memory. Although device 1140 is shown as including one of each described component, various components may be replicated in various embodiments. For example, the processor may include multiple microprocessors that are configured to independently execute the methods described herein, or are configured to perform the steps or subroutines of the methods described herein such that the multiple processors collaborate to implement the functionality described herein. Additionally, in the case of implementing device 1140 in a cloud computing system, the various hardware components may belong to separate physical systems. For example, the processor may include a first processor in a first server and a second processor in a second server.

[0256] It should be noted that the above-mentioned embodiments illustrate rather than limit the presently disclosed subject matter, and that those skilled in the art will be able to design many alternative embodiments.

[0257] In the claims, any reference signs placed between brackets shall not be construed as limiting the claim. The use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in the claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. When preceding a list of elements, expressions such as "at least one of..." indicate that all elements or any subset of elements are selected from the list. For example, the expression "at least one of A, B, and C" should be understood to include only A, only B, only C, both A and B, both A and C, both B and C, or all of A, B, and C. The presently disclosed subject matter can be implemented by hardware comprising several different elements and by a suitably programmed computer. In a device claim that lists several parts, several of these parts may be embodied by the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

[0258] In the claims, references in parentheses refer to reference signs in the drawings of exemplary embodiments or formulas of the embodiments, thereby increasing the intelligibility of the claims. These references should not be construed as limiting the claims.

Claims

1. A computer-implemented method for classifying sensor data with improved robustness against label noise, the method comprising - providing training data, wherein the training data includes a plurality of training input data and corresponding labels, wherein the training input data represents a physical property of a physical system obtained from one or more sensors, - a base classifier defined for the training data, the base classifier comprising an encoder configured to map input data to a latent representation and a classifier configured to be applied to the latent representation, wherein, The encoder is defined by pre-trained parameters, and the basic training function is configured to optimize the parameters defining the classifier based on the training data, - provide novel input data, - define the noise level for multiple labels, - Compute predicted labels for novel inputs with improved robustness to label noise, the computation consisting of estimating the most likely label given repeated: The basic training function is applied to the training labels incorporating noise according to the noise level and then applying a base classifier to the novel input, the base classifier being configured according to the base training function; - calculating a radius indicating the number of label changes, where the predicted label for the novel input will not change if the number of label changes does not exceed the indicated number of label changes; - obtain an estimate of the number of corrupted labels in the training data, Before accepting predicted labels for novel inputs, the estimated number of corrupted labels is compared to the radius.

2. The method according to claim 1, wherein The classifier includes a linear operation configured to be applied to the latent representation.

3. The method according to claim 2, wherein: The basic training function is configured to apply least squares optimization to derive parameters defining a linear operation from the training data.

4. A method according to any one of the preceding claims, comprising computing a linear operation to obtain a combined result of applying a base training function and a base prediction function to novel inputs and training labels incorporating noise.

5. The method according to any one of claims 1 to 3, wherein include: Estimating the most likely label in the case of repeated applications involves evaluating one or more inequalities to obtain bounds on the likelihood.

6. The method according to claim 5, wherein: An inequality is defined for a plurality of values of a variable, the method comprising approximating the value of the variable, the inequality giving the tightest bound for the variable, the approximation comprising a Newton approximation.

7. The method according to any one of claims 1 to 3, wherein - the encoder comprises a neural network, and / or - the encoder is a feature extractor configured to generate a plurality of features, and / or - the encoder is trained as an autoencoder, and / or - The encoder is trained on the training data, and / or -The encoder is trained on the training data together with the temporary classifier. The method of claim 7 , wherein the training data is a selected subset of the provided training data.

9. The method according to any one of claims 1 to 3, wherein - the encoder and the classifier are defined by parameters, the number of parameters defining the encoder is greater than the number of parameters defining the classifier, and / or - the encoder and the classifier comprise neural networks, said encoder neural network comprising more nodes and / or layers than the classifier neural network, -Classifier includes linear operation and sigmoid operation.

10. The method according to claim 1, comprising: - Train the encoder depending on the radius and the estimated number of corrupted labels.

11. The method according to claim 10, comprising: - If the estimated number of damaged tags is higher than the radius, then - obtain corrected labels from the training data and / or discard corrupted labels, and - Train the encoder on the corrected labels and / or on training data without the discarded labels.

12. The method according to claim 1, wherein Computing predicted labels for one or more novel input data, the method includes -Store novel input data and predicted labels, - If the estimate of the number of corrupt labels among the stored predicted labels is below the radius, then - Train the encoder on the stored novel input data and predicted labels.

13. The method according to any one of claims 1 to 3, wherein: The training input data is associated with one or more binary labels in the training data.

14. The method according to any one of claims 1 to 3, comprising - Determining control signals based on predicted labels of the physical system and / or autonomous devices operating in the physical system.

15. A classification system configured to classify sensor data with improved robustness against label noise, the classification system comprising - a training data storage device configured to store training data, the training data comprising a plurality of training input data and corresponding labels, the training input data representing physical properties of a physical system obtained from one or more sensors, - a base classifier defined for the training data, the base classifier comprising an encoder configured to map input data to a latent representation and a classifier configured to be applied to the latent representation, wherein, The encoder is defined by pre-trained parameters, and the basic training function is configured to optimize the parameters defining the classifier based on the training data, - a novel input data interface configured to obtain novel input data, - a processor system configured to - define the noise level for multiple labels, Compute predicted labels for novel inputs with improved robustness to label noise, the computation consisting of estimating the most likely label given repeated: The basic training function is applied to the training labels incorporating noise according to the noise level and then applying a base classifier to the novel input, the base classifier being configured according to the base training function; - calculating a radius indicating the number of label changes, where the predicted label for the novel input will not change if the number of label changes does not exceed the indicated number of label changes; - obtain an estimate of the number of corrupted labels in the training data, Before accepting predicted labels for novel inputs, the estimated number of corrupted labels is compared to the radius.

16. A transitory or non-transitory computer-readable medium comprising data representing instructions which, when executed by a processor system, cause the processor system to carry out the method according to any one of claims 1 to 14.

Images