Ambiguating predictive uncertainty in real-world classification with adversarial gradient aggregation

Abstract

Attributing predictive uncertainty in real-world classification with adversarial gradient aggregation is provided. A method of obtaining a predicted uncertainty attribution of an object in an input image includes: receiving the input image; generating a prediction of an object in the input image; estimating an uncertainty associated with prediction of an object in the input image; calculating an uncertainty attribution representing a region of the input image resulting in the estimated uncertainty, including generating a plurality of adversarial gradients, each adversarial gradient corresponding to a modification of the input image configured to change the estimated uncertainty; and generating an output indicative of the calculated attribution of uncertainty.

Description

Technical Field

[0001] The present disclosure relates to object detection, and more particularly, to determining model uncertainty in object detection systems and methods. Background Art

[0002] Object detection techniques can use various detection models to detect or predict, classify, and label objects and regions in captured images. Detection models typically have associated uncertainties (e.g., prediction or prediction uncertainty) that indicate the confidence level of the model's predictions. Summary of the invention

[0003] A method for obtaining uncertainty attributions for predictions of objects in an input image, comprising: receiving an input image; generating predictions of objects in the input image; estimating uncertainty associated with the predictions of objects in the input image; calculating uncertainty attributions for regions of the input image that represent the estimated uncertainty, including generating a plurality of adversarial gradients, each adversarial gradient corresponding to a modification of the input image configured to change the estimated uncertainty; and generating an output indicative of the calculated uncertainty attributions.

[0004] In other features, generating a plurality of adversarial gradients includes performing at least one of: introducing noise into an input image and modifying selected pixels of the input image. Generating a plurality of adversarial gradients includes identifying at least one adversarial gradient that causes the estimated uncertainty to decrease below a threshold. Generating a plurality of adversarial gradients includes identifying an adversarial gradient that causes the estimated uncertainty to decrease below the threshold with a minimal amount of modification to the input image. Computing an uncertainty attribution includes identifying an uncertainty attribution for a selected object class.

[0005] In other features, the method further comprises: to identify uncertainty attribution, and corresponds to the uncertainty attribution of category c along the integration path, H(x) corresponds to the uncertainty, and f c (x) corresponds to a classification score for category c and input x. The method also includes modifying the input image based on the calculated uncertainty attribution and generating a second prediction of the object in the modified input image, and modifying the input image includes masking a portion of the image corresponding to the calculated uncertainty attribution.

[0006] A computing device configured to obtain uncertainty attributions for predictions of objects in an input image includes a processing device configured to execute instructions stored in a memory to perform the following operations: receive an input image; generate predictions of objects in the input image; estimate uncertainty associated with the predictions of objects in the input image; calculate uncertainty attributions for regions of the input image that represent the estimated uncertainty, including generating multiple adversarial gradients, each adversarial gradient corresponding to a modification of the input image configured to change the estimated uncertainty; and generate an output indicative of the calculated uncertainty attributions.

[0007] In other features, to generate the plurality of adversarial gradients, the processing device is configured to execute instructions to perform at least one of: introducing noise into an input image and modifying selected pixels of the input image. To generate the plurality of adversarial gradients, the processing device is configured to execute instructions to identify at least one adversarial gradient that causes the estimated uncertainty to decrease below a threshold. To generate the plurality of adversarial gradients, the processing device is configured to execute instructions to identify an adversarial gradient that causes the estimated uncertainty to decrease below a threshold with a minimal amount of modification to the input image. To compute an uncertainty attribution, the processing device is configured to execute instructions to identify an uncertainty attribution for a selected object class.

[0008] In other features, the processing device is configured to execute instructions to to identify uncertainty attribution, and corresponds to the uncertainty attribution of category c along the integration path, H(x) corresponds to the uncertainty, and f c (x) corresponds to a classification score for category c and input x. The processing device is configured to execute instructions to modify the input image based on the calculated uncertainty attribution and generate a second prediction of the object in the modified input image, and to modify the input image, the processing device is configured to execute instructions to mask a portion of the image corresponding to the calculated uncertainty attribution.

[0009] A computer-controlled machine comprises: at least one sensor configured to generate an input image; a control system configured to receive the input image; generate predictions of objects in the input image; estimate uncertainty associated with the predictions of objects in the input image; calculate uncertainty attributions for regions of the input image that represent the estimated uncertainty, including generating a plurality of adversarial gradients, each adversarial gradient corresponding to a modification of the input image configured to change the estimated uncertainty; and generating an output signal indicative of the calculated uncertainty attributions; and an actuator configured to control operation of the computer-controlled machine in response to the output signal.

[0010] In other features, to generate the plurality of adversarial gradients, the control system is configured to execute instructions to perform at least one of: introducing noise into the input image and modifying selected pixels of the input image. To generate the plurality of adversarial gradients, the control system is configured to execute instructions to identify at least one adversarial gradient that causes the estimated uncertainty to decrease below a threshold. To generate the plurality of adversarial gradients, the processor is configured to execute instructions to identify an adversarial gradient that causes the estimated uncertainty to decrease below the threshold with a minimum amount of modification to the input image. The control system is configured to execute instructions to modify the input image based on the calculated uncertainty attribution and generate a second prediction of an object in the modified input image. The computer-controlled machine includes an autonomous robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A system for training a neural network according to the principles of the present disclosure is generally shown.

[0012] Figure 2 A computer-implemented method for training and utilizing a neural network in accordance with the principles of the present disclosure is generally presented.

[0013] Figure 3A An audio data tagging system according to the principles of the present disclosure is generally shown.

[0014] Figure 3B A portion of a data capture system in accordance with the principles of the present disclosure is generally shown.

[0015] Figure 3C An alternative audio data labeling system in accordance with the principles of the present disclosure is generally shown.

[0016] Figure 4A An example uncertainty attribution process for the systems and methods of the present disclosure is generally shown.

[0017] Figure 4B Example uncertainty attributions obtained in accordance with the principles of the present disclosure are generally shown.

[0018] Figure 4C An example quantitative evaluation of the systems and methods of the present disclosure is generally presented.

[0019] Figure 4D The steps of an example uncertainty attribution method according to the principles of the present disclosure are shown.

[0020] Figure 5 Depicted is a schematic diagram of the interaction between a computer-controlled machine and a control system in accordance with the principles of the present disclosure.

[0021] Figure 6 Depicted is a Figure 5Schematic diagram of a control system configured to control a vehicle, which can be a partially autonomous vehicle, a fully autonomous vehicle, a partially autonomous robot, or a fully autonomous robot.

[0022] Figure 7 Depicted Figure 5 Schematic diagram of a control system configured to control a manufacturing machine, such as a punch cutter, a cutting machine, or a gun drill, of a manufacturing system (e.g., part of a production line).

[0023] Figure 8 Depicted Figure 5 Schematic diagram of a control system configured to control a power tool, such as a drill or driver, having an at least partially autonomous mode.

[0024] Fig. 9 Depicting a device configured to control an automated personal assistant Figure 5 Schematic diagram of the control system.

[0025] Fig.10 Depicted Figure 5 Schematic diagram of a control system configured to control a monitoring system, such as a controlled access system or a surveillance system.

[0026] Fig.11 Depicted Figure 5 Schematic diagram of a control system configured to control an imaging system, such as an MRI device, an x-ray imaging device, or an ultrasound device. DETAILED DESCRIPTION

[0027] Embodiments of the present disclosure are described herein. However, it should be understood that the disclosed embodiments are merely examples, and other embodiments may take various and alternative forms. The drawings are not necessarily to scale; some drawings may be enlarged or minimized to show the details of specific components. Therefore, the specific structural and functional details disclosed herein should not be interpreted as limiting, but merely as a representative basis for teaching those skilled in the art to adopt the embodiments in different ways. As will be understood by those of ordinary skill in the art, the various features shown and described with reference to any of the figures may be combined with the features shown in one or more other figures to produce embodiments that are not explicitly shown or described. The combination of the features shown provides representative embodiments for typical applications. However, for specific applications or implementations, various combinations and modifications of features consistent with the teachings of the present disclosure may be desirable.

[0028] Unless the context clearly indicates otherwise, "a", "an", and "the" as used herein refer to both singular and plural references. For example, a "processor" programmed to perform various functions refers to one processor programmed to perform each and all functions, or more than one processor programmed together to perform each of the various functions.

[0029] Capturing perceptual uncertainty in machine learning models is integral to the critical application of object detection. Model uncertainty often manifests as aspects of the system or data generation process that are not accurately understood, such as sensor noise or a lack of diversity in the representation data used during training. The ability to quantify this uncertainty and attribute it to its sources can help users thoroughly understand the causes of uncertainty, thereby facilitating interpretability in critical applications such as autonomous driving. In sensitive contexts, domain experts may use uncertainty explanations to focus their attention on specific features that the model identifies as anomalous.

[0030] Therefore, object detection systems and methods can implement uncertainty estimation techniques for the purpose of active learning or non-distributed detection. In some examples, a mechanism for attributing prediction uncertainty to input features can be implemented. For example, gradient-based interpretation methods (such as integral gradient (IG) interpretation methods) are applicable to attributing uncertainty to input features. However, IG interpretation explicitly requires a reference that the model can predict with certainty. Methods using black, white, or blurred images as references can result in low performance in attributing uncertainty. Therefore, other methods train auxiliary deep generative models to create possible changes to the input so that the model becomes more certain in its predictions. Uncertainty can be characterized by comparing the difference between the original input and the generated reference. In another example, a reference generated by a deep generative model for IG interpretation is used to attribute the uncertainty of the model. However, these methods rely heavily on the performance of the generative model, which can be troublesome when the underlying task is complex.

[0031] Uncertainty attribution systems and methods according to the present disclosure are configured to more accurately explain uncertainty in real-world settings. These systems and methods avoid the need for realistic counterfactual examples and are therefore applicable to complex data sets. For example, by using gradients to find adversarial examples for the model, the need for generative models can be minimized. The purpose of searching for adversarial examples is not to mislead the model, but to increase the certainty of the model. Therefore, adversarial examples can serve as a reference for IG explanations that explain model uncertainty. Because adversarial examples are well defined and easy to generate, the systems and methods of the present disclosure are configured to handle high-resolution, complex data sets in real-world settings.

[0032] Figure 1An example system 100 for training a neural network (e.g., of an ML model) is shown. System 100 may be configured to (and / or include circuitry configured to) implement the systems and methods of the present disclosure described in more detail below. System 100 may include an input interface for accessing training data 102 for the neural network. For example, Figure 1 As shown, the input interface can be constituted by a data storage interface 104, which can access the training data 102 from a data storage unit 106. For example, the data storage interface 104 can be a memory interface or a persistent storage interface, such as a hard disk or SSD interface, but can also be a personal, local or wide area network interface, such as a Bluetooth, Zigbee or Wi-Fi interface or an Ethernet or fiber optic interface. The data storage unit 106 can be an internal data storage unit of the system 100, such as a hard disk drive or SSD, but can also be an external data storage unit, for example, a network accessible data storage unit.

[0033] In some embodiments, data store 106 may also include a data representation 108 of an untrained version of the neural network, which may be accessed by system 100 from data store 106. However, it will be appreciated that training data 102 and data representation 108 of the untrained neural network may also be accessed from different data stores, e.g., via different subsystems of data store interface 104. Each subsystem may be of a type as described above for data store interface 104.

[0034] In some embodiments, the data representation 108 of the untrained neural network may be generated internally by the system 100 based on the design parameters of the neural network, and thus may not be explicitly stored on the data storage 106. The system 100 may also include a processor subsystem 110 that may be configured to provide an iterative function during operation of the system 100 as a replacement for a stack of neural network layers to be trained. Here, the individual layers of the replaced layer stack may have mutually shared weights and may receive as input the output of the previous layer, or, for the first layer of the layer stack, receive as input an initial activation and a portion of the input of the layer stack.

[0035] The processor subsystem 110 may be further configured to iteratively train the neural network using the training data 102. Here, the training iterations of the processor subsystem 110 may include a forward propagation portion and a backward propagation portion. The processor subsystem 110 may be configured to perform the forward propagation portion by: determining an equilibrium point of the iterative function at which the iterative function converges to a fixed point, in addition to defining other operations of the forward propagation portion that may be performed, wherein determining the equilibrium point includes using a numerical root finding algorithm to find a root solution of the iterative function minus its input; and providing the equilibrium point in place of an output of a layer stack in the neural network.

[0036] The system 100 may also include an output interface for outputting a data representation 112 of the trained neural network, which may also be referred to as trained model data 112. Figure 1 As shown, the output interface can be constituted by the data storage interface 104, which in these embodiments is an input / output ("IO") interface, through which the trained model data 112 can be stored in the data storage 106. For example, the data representation 108 defining the "untrained" neural network can be replaced during or after training, at least in part, with the data representation 112 of the trained neural network, because the parameters of the neural network, such as the weights, hyperparameters and other types of parameters of the neural network, can be adapted to reflect the training on the training data 102. Figure 1 This is also illustrated in the example diagram by reference to the same data records 108, 112 on the data store 106. In some embodiments, the data representation 112 may be stored separately from the data representation 108 defining the "untrained" neural network. In some embodiments, the output interface may be separate from the data storage interface 104, but may generally be of the type described above for the data storage interface 104.

[0037] Figure 2 A data annotation / augmentation system 200 configured to (and / or including circuitry configured to) implement a system for annotating and / or augmenting data is depicted. The data annotation system 200 may include at least one computing system 202 configured to implement all or part of the systems and methods of the present disclosure explained in more detail below. The computing system 202 may include at least one processor 204 operably connected to a memory unit 208. The processor 204 may include one or more integrated circuits that implement the functionality of a central processing unit (CPU) 206. The CPU 206 may be a commercially available processing unit that implements an instruction set such as one of the x86, ARM, Power, or MIPS instruction set families. The various components of the system 200 may be implemented with the same or different circuits.

[0038] During operation, CPU 206 may execute stored program instructions retrieved from memory unit 208. The stored program instructions may include software that controls the operation of CPU 206 to perform the operations described herein. In some embodiments, processor 204 may be a system on a chip (SoC) that integrates the functionality of CPU 206, memory unit 208, network interfaces, and input / output interfaces into a single integrated device. Computing system 202 may implement an operating system for managing various aspects of operation.

[0039] Memory unit 208 may include volatile memory and nonvolatile memory for storing instructions and data. Nonvolatile memory may include solid-state memory, such as NAND flash memory, magnetic and optical storage media, or any other suitable data storage device that retains data when computing system 202 is disabled or loses power. Volatile memory may include static and dynamic random access memory (RAM) that stores program instructions and data. For example, memory unit 208 may store machine learning model 210 (e.g., in Figure 2 210), or an algorithm, a training data set 212 for the machine learning model 210, an original source data set 216, etc.

[0040] The computing system 202 may include a network interface device 222 configured to provide communications with external systems and devices. For example, the network interface device 222 may include a wired and / or wireless Ethernet interface defined by the Institute of Electrical and Electronics Engineers (IEEE) 802.11 series of standards. The network interface device 222 may include a cellular communication interface for communicating with a cellular network (e.g., 3G, 4G, 5G). The network interface device 222 may be further configured to provide a communication interface to an external network 224 or a cloud.

[0041] The external network 224 may be referred to as the World Wide Web or the Internet. The external network 224 may establish a standard communication protocol between computing devices. The external network 224 may allow information and data to be easily exchanged between computing devices and the network. One or more servers 230 may communicate with the external network 224.

[0042] Computing system 202 may include input / output (I / O) interface 220, which may be configured to provide digital and / or analog input and output. I / O interface 220 may include an additional serial interface (eg, a universal serial bus (USB) interface) for communicating with external devices.

[0043] The computing system 202 may include a human-machine interface (HMI) device 218, which may include any device that enables the system 200 to receive control inputs. Examples of input devices may include human-machine interface inputs, such as keyboards, mice, touch screens, voice input devices, and other similar devices. The computing system 202 may include a display device 232. The computing system 202 may include hardware and software for outputting graphics and text information to the display device 232. The display device 232 may include an electronic display screen, a projector, a printer, or other suitable devices for displaying information to a user or operator. The computing system 202 may be further configured to allow interaction with a remote HMI and a remote display device through a network interface device 222.

[0044] System 200 can be implemented using one or more computing systems. Although this example depicts a single computing system 202 that implements all described features, it is intended that various features and functions be separated and implemented through multiple computing units that communicate with each other. The specific system architecture selected can depend on various factors.

[0045] The system 200 can implement a machine learning model 210 configured to analyze a raw source data set 216. For example, the CPU 206 and / or other circuits can implement the machine learning model 210. The raw source data set 216 can include raw or unprocessed sensor data, which can represent an input data set for a machine learning system. The raw source data set 216 can include video, video clips, images, audio, text-based information, and raw or partially processed sensor data (e.g., a radar map of an object). In some embodiments, the machine learning model 210 can be a deep learning or neural network algorithm designed to perform a predetermined function. For example, a neural network algorithm can be configured to identify an event or object in a video clip based on audio data.

[0046] The computer system 200 may store a training data set 212 for the machine learning model 210. The training data set 212 may represent a set of previously constructed data used to train the machine learning model 210. The machine learning model 210 may use the training data set 212 to learn weighting factors associated with a neural network algorithm. The training data set 212 may include a set of source data having corresponding achievements or results that the machine learning model 210 attempts to replicate through the learning process.

[0047] The machine learning model 210 can operate in a learning mode using the training data set 212 as input. The machine learning model 210 can be executed in multiple iterations using data from the training data set 212. For each iteration, the machine learning model 210 can update the internal weighting factors based on the achieved results. For example, the machine learning model 210 can compare the output results (e.g., annotations) with the results included in the training data set 212. Because the training data set 212 includes expected results, the machine learning model 210 can determine when the performance is acceptable. After the machine learning model 210 reaches a predetermined performance level (e.g., 100% consistency with the results associated with the training data set 212), the machine learning model 210 can be executed using data that is not in the training data set 212. The trained machine learning model 210 can be applied to a new data set to generate annotated data.

[0048] The machine learning model 210 may be configured to identify specific features in the raw source data 216. The raw source data 216 may include multiple instances or input data sets (e.g., video streams or segments including audio data) for which annotation results are desired. By way of example only, the machine learning model 210 may be configured to identify objects or events in a video segment based on audio data and annotate these events. The machine learning model 210 may be programmed to process the raw source data 216 to identify the presence of specific features. The machine learning model 210 may be configured to identify features in the raw source data 216 as predetermined features. The raw source data 216 may be obtained from a variety of sources. For example, the raw source data 216 may be actual input data collected by a machine learning system. The raw source data 216 may be machine generated for testing the system. By way of example, the raw source data 216 may include raw video and / or audio data from a camera, audio data from a microphone, and the like.

[0049] In an example, the machine learning model 210 can process the raw source data 216 and output video and / or audio data including one or more indications of the identified events. The machine learning model 210 can generate a confidence level or factor for each generated output. For example, a confidence value exceeding a predetermined high confidence threshold can indicate that the machine learning model 210 is confident that the identified event (or feature) corresponds to a particular event. A confidence value less than a low confidence threshold can indicate that the machine learning model 210 has some uncertainty that a particular feature exists.

[0050] like Figure 3A and 3B , system 300 may include an image (e.g., video) capture device 302, an audio capture array 304, and a computing system 202. The system may receive video stream data associated with a data capture environment from the image capture device 302. The system 202 may be configured to perform video object detection to identify one or more objects in a corresponding image of the video stream data. The system 202 may receive audio stream data corresponding to at least a portion of the video stream data from the audio capture array 304. The audio capture array 304 may include one or more microphones 306 or other suitable audio capture devices. The systems and methods described herein may be configured to use output from at least a first machine learning model (e.g., such as machine learning model 210 or other suitable machine learning model configured to provide output including one or more object or event detection predictions) to mark at least some objects of the video stream data and / or audio stream data.

[0051] The system 202 may calculate (e.g., using at least one probability-based function or other suitable technique or function) at least one offset value for at least a portion of the audio stream data corresponding to at least one tagged object of the video stream data based on at least one data capture characteristic. The system 202 may synchronize at least a portion of the video stream data with a portion of the audio stream data corresponding to at least one tagged object of the video stream data using at least the at least one offset value. The at least one data capture characteristic may include one or more characteristics of at least one image capture device, one or more characteristics of at least one audio capture array, one or more characteristics corresponding to a position of at least one image capture device relative to at least one audio capture array, one or more characteristics corresponding to movement of an object in the video stream data, one or more other suitable data capture characteristics, or a combination thereof.

[0052] System 202 may use one or more tags and at least one offset value of the labeled objects of the video stream data to mark at least a portion of the audio stream data corresponding to at least one labeled object of the video stream data. Each corresponding tag may include an event type, an event start indicator, and an event end indicator. System 202 may use at least some of the labeled portions of the audio stream data to generate training data. System 202 may use the training data to train a second machine learning model. System 202 may use the second machine learning model to detect one or more sounds associated with the audio data provided as input to the second machine learning model. The second machine learning model may include any suitable machine learning model and may be configured to perform any suitable functions, such as those described herein with reference to Figures 4-11.

[0053] In some embodiments, Figure 3CAs generally shown in , the computing system 202 can be configured to mark audio data based on sensor data received from one or more sensors, such as those described herein or any other suitable sensor or combination of sensors. The system 202 can receive audio stream data associated with the data capture environment from the audio capture array 354 or any suitable audio capture device (such as one or more microphones 306 or other suitable audio capture devices). It should be understood that the audio capture array 354 may include features similar to the audio capture array 304 features, and may include any suitable number of audio capture devices. The system 202 can receive sensor data associated with the data capture environment from at least one sensor (e.g., such as sensor 352) asynchronous with respect to the audio capture array 354. The sensor 354 may include at least one of an induction coil, a radar sensor, a LiDAR sensor, a sonar sensor, an image capture device, any other suitable sensor, or a combination thereof. The audio capture array 354 can be positioned away from the sensor 354, positioned close to the sensor 354, or positioned to be in any suitable relationship with the sensor 354.

[0054] System 202 may use output from at least a first machine learning model (e.g., machine learning model 210 or other suitable machine learning model) to identify at least some events in sensor data. Machine learning model 210 may be configured to provide output including one or more event detection predictions based on sensor data. System 202 may synchronize at least a portion of sensor data associated with a portion of audio stream data corresponding to at least one event of sensor data. System 202 may use one or more tags extracted for corresponding events of sensor data values ​​to mark at least a portion of audio stream data corresponding to at least one event of sensor data. Each corresponding tag may include an event type, an event start indicator, and an event end indicator. System 202 may use at least some of the marked portions of audio stream data to generate training data. System 202 may use training data to train a second machine learning model. System 202 may use a second machine learning model to detect one or more sounds associated with audio data provided as input to a second machine learning model. The second machine learning model may include any suitable machine learning model and may be configured to perform any suitable functions, such as those described herein with reference to FIGS. 4-11.

[0055] Any of the systems described above and / or in more detail below may implement the uncertainty attribution systems and methods of the present disclosure, including measuring model uncertainty for a classification task and obtaining uncertainty attribution using an integrated gradient interpretation technique.

[0056] Figure 4AAn example uncertainty attribution process 400 of the systems and methods of the present disclosure is shown. Process 400 includes uncertainty estimation, attribution, and quantification. As shown at 402, given an input image and a trained classifier, the prediction uncertainty and the aleatoric and epistemic components of the prediction uncertainty can be directly estimated by the predicted entropy. Adversarial gradient integration is shown at 404. Starting from a randomly initialized point around the input image ("random initialization"), a reference is generated that makes the model's predictions certain by adding adversarial perturbations. The adversarial gradient is then integrated throughout the generation path to attribute uncertainty to the pixels of the input image. In order to quantitatively evaluate this attribution, the most important pixels are gradually masked, as shown at 406. The resulting image is input into the trained classifier to determine whether the uncertainty is proportionally reduced.

[0057] For example, in 402, according to An uncertainty estimate is performed on the output of the classifier 408. Weights is learned to fit some available training dataset Therefore, the function f(x) transforms the eigenvector Mapping to a single standard The elements in represent the membership probabilities across categories in set c. In this example, as shown at 410, the output of classifier 408 has multiple peaks corresponding to high membership probabilities of different categories (e.g., "butterfly" and "flower"). The uncertainty arises from the possibility of belonging to different categories.

[0058] Entropy is considered a measure of prediction uncertainty H(x) according to the following equation:

[0059]

[0060] where f c (x) represents the predicted probability of membership in class c. In a Bayesian setting, the posterior distribution can be taken into account via the weights in the model The entropy in Equation 1 can be further decomposed into stochastic (Equation 2) and cognitive (Equation 3) components:

[0061] and

[0062]

[0063] The stochastic and epistemic components represent different types of uncertainty, where the stochastic component is caused by noise from the data that resembles inherent noise (e.g., two examples with the same feature may have different outcomes), and the epistemic component is caused by insufficient data or inappropriate modeling choices.

[0064] IG interpretation can be performed to generate uncertainty attributions. In the example, given a path function γ(α) with α∈[0,1], the path-integrated gradient of the input x along the i-th dimension is defined as follows:

[0065]

[0066] in is the gradient of F along the i-th dimension of the input x. Here, γ(α) is parameterized as the reference image x 0 and the straight line path between the observed image x (e.g., according to γ(α)=x 0 +α×(xx 0 ), α∈[0,1]), and Equation 4 can be simplified to:

[0067]

[0068] F(x)=f c (x) can represent the classification score for a particular class, such that these attributions capture the elements in the image that are associated with that class. To perform uncertainty attribution, H(x) is assigned to F(x), and thus all classes are considered in an attempt to identify pixels that confuse the classifier model. Additionally, aleatoric uncertainty and epistemic uncertainty can be assigned or To explain independently.

[0069] While using the modified integrated gradient helps in the step of attributing uncertainty, there is still a gap between the prediction and the explanation of uncertainty. To attribute the predicted class of the model, the original reference input x 0 is set to a black image (i.e., all pixel intensities are set to zero because no object corresponds to a black image). However, in order to attribute uncertainty, a reference input to the model is required that is deterministic. In some examples, generative models are used to generate counterfactuals as references for IG explanations, which are superior to adaptations of ordinary IG explanations and variants. However, training generative models is difficult and only works for a limited type of datasets. It is not feasible to train generative models to produce valid references from real-world datasets characterized by complex scenes and high resolution.

[0070] To integrate the adversarial gradient, portions of the input image are modified (e.g., iteratively), such as by introducing noise, changing pixels, etc. For example, when noise is added to the input image, relevant objects may become more easily detectable / identifiable, while other objects may become less easily detectable / identifiable. As an example, a corresponding portion of the input image corresponding to the peak shown at 410 can be blurred by adding noise to the input image.

[0071] Therefore, at 404, an adversarial gradient aggregation technique is implemented according to uncertainty attribution of the present disclosure. Instead of relying on a generative model to produce a realistic reference, adversarial examples are directly generated as references. Given an input and a trained model, adversarial situations are well defined and easy to find. Here, a targeted adversarial example is clearly defined as a perturbation example that is closest to the original input, such that it changes the model's prediction from uncertain to certain. Since the shortest path in the input space does not necessarily correspond to the shortest path in the learned feature space, the integral from the adversarial example to the straight line input may not be optimal. Therefore, the steepest ascent path γ(α) is selected as the integral path, and the uncertainty attribution attr i Defined by:

[0072]

[0073] where the adversarial example x0 = γ(0) and the corresponding input x = γ(1). For the path defined by the steepest ascent, the definition is as follows:

[0074]

[0075] The negative sign in Equation 7 is a result of the direction being opposite to the case of finding adversarial examples, which leads to:

[0076]

[0077] It is integrated along the path until H(x) is less than a threshold. In some examples, the gradient ascent may encounter a local maximum that prevents further ascent. Therefore, in some examples, a signed gradient may be used instead of the original gradient to facilitate crossing the decision boundary.

[0078] Since there are various plausible explanations for uncertainty, rather than relying on just a single reference, the systems and methods according to the present disclosure generate multiple references with diverse starting points. In particular, diversity is achieved by randomly initializing the search starting from a ∈ sphere around the input image x. Finally, explanations are obtained by aggregating individual attributions from multiple starting points. This aggregation can also mitigate instabilities associated with gradients.

[0079] As described above, uncertainty is explained by combining scores from all classes, which confounds the model. However, in real-world applications, it may be desirable to determine why the model is uncertain for one or more specific classes, such as the ground truth class. The techniques described in this article provide reference to multiple source functions and can be modified to account for specific classes, as follows:

[0080]

[0081] It integrates along the path until the model is certain on a particular class c. Unlike the above method which randomly generates uncertain references, the modification of Equation 9 describes the uncertainty by selecting references from a single class.

[0082] Continue to refer Figure 4A ,exist Figure 4B and Figure 4C The evaluation of uncertainty attribution for the present disclosure is described in . Figure 4A As shown at 406 in , the most important pixels are gradually masked and the resulting image is input into a trained classifier 408 to determine whether the uncertainty is reduced proportionally.

[0083] Several qualitative examples of uncertainty attribution obtained through the above techniques are given in Figure 4B , which shows an example of a heat map interpretation of uncertainty using the systems and methods of the present disclosure. In input image 420, the prediction for the class "Italian Greyhound" may be uncertain because other objects in image 420 have valid class names in the dataset (e.g., "jeans" and "cornflower"). Regions corresponding to other objects are highlighted in heat map images 422 and 424. In other words, heat map image 422 indicates objects or regions that may cause prediction uncertainty in the output of the model / classifier.

[0084] Similarly, for input image 426, the ground truth class of the target object is "turtle". However, the relatively small size of the target object and the unexpected background features of the turtle (e.g., hands, trees and leaves, etc.) lead to prediction uncertainty. Therefore, the model predicts the "acorn" class. Heat map images 428 and 430 highlight the objects / areas of input image 426 that lead to prediction uncertainty and incorrect predictions.

[0085] Figure 4C Quantitative evaluation of the systems and methods described herein in multiple examples is shown. Given input images 432-1, 432-2, and 432-3 with uncertainty attributions and corresponding heat map images 434-1, 434-2, and 434-3, pixels are masked (e.g., in an order of importance defined by the attribution values) to show how the masked pixels affect the prediction uncertainty. For example, pixels in the area corresponding to the uncertainty attribution (i.e., the pixels that contribute the most to the uncertainty) are masked. As shown in 436-1, 436-2, and 436-3, masking a portion of the pixels significantly reduces the uncertainty of the model. For example, as Figure 4A As shown at 406 in , masking the pixels corresponding to the uncertainty attribution shown in the heat map image results in one of the peaks being removed (and thus increasing the certainty of the output of classifier 408).

[0086] Figure 4DSteps of an example uncertainty attribution method 440 in accordance with principles of the present disclosure are shown. For example, one or more processors or processing devices are configured to execute instructions to implement method 440, such as one or more processors of the system described herein. At 442, method 440 (e.g., an image classifier) ​​receives an input image. At 444, method 400 (e.g., a classifier) ​​generates an output indicating a prediction of an object in the input image (e.g., using a model, such as an ML model). At 446, method 400 estimates the uncertainty of the prediction (i.e., generates an uncertainty estimate). For example, the uncertainty estimate indicates a corresponding probability that the input image contains objects belonging to various categories.

[0087] At 448, method 400 performs uncertainty attribution on the image by aggregating adversarial gradients. For example, performing uncertainty attribution can include generating multiple adversarial examples by introducing noise and / or other modifications to the input image and identifying image regions (e.g., selected pixels) corresponding to uncertainty.

[0088] At 450, method 400 masks selected pixels in the image based on the uncertainty attribution. For example, pixels determined to contribute most to the uncertainty (e.g., a predetermined number of pixels with the highest attribution value) are masked in the input image. At 452, method 400 predicts an object in the masked input image and generates an output indicating the predicted object.

[0089] Figure 5-11 Depicted are example systems and devices that may implement uncertainty attribution systems and methods consistent with the present disclosure. Figure 5 A schematic diagram of the interaction between a computer-controlled machine 500 and a control system 502 is depicted. The computer-controlled machine 500 includes an actuator 504 and a sensor 506. The actuator 504 may include one or more actuators, and the sensor 506 may include one or more sensors. The sensor 506 is configured to sense a condition of the computer-controlled machine 500. The sensor 506 may be configured to encode the sensed condition into a sensor signal 508 and transmit the sensor signal 508 to the control system 502. Non-limiting examples of the sensor 506 include video, radar, LiDAR, ultrasonic, and motion sensors. In some embodiments, the sensor 506 is an optical sensor configured to sense an optical image of the environment near the computer-controlled machine 500.

[0090] The control system 502 is configured to receive sensor signals 508 from the computer controlled machine 500. As described below, the control system 502 may be further configured to calculate actuator control commands 510 based on the sensor signals and transmit the actuator control commands 510 to the actuators 504 of the computer controlled machine 500.

[0091] like Figure 5 As shown, the control system 502 includes a receiving unit 512. The receiving unit 512 can be configured to receive the sensor signal 508 from the sensor 506 and transform the sensor signal 508 into an input signal x. In an alternative embodiment, the sensor signal 508 is directly received as the input signal x without the receiving unit 512. Each input signal x can be a part of each sensor signal 508. The receiving unit 512 can be configured to process each sensor signal 508 to generate each input signal x. The input signal x can include data corresponding to the image recorded by the sensor 506.

[0092] The control system 502 includes a classifier 514. The classifier 514 can be configured to classify the input signal x into one or more labels using a machine learning (ML) algorithm, such as a neural network. For example, the classifier 514 corresponds to the above-mentioned classifier 408. The classifier 514 is configured to be parameterized by parameters, such as those described above (e.g., parameter θ). The parameter θ can be stored in and provided by the non-volatile storage unit 516. The classifier 514 is configured to determine the output signal y from the input signal x. Each output signal y includes information that assigns one or more labels to each input signal x. The classifier 514 can transmit the output signal y to a conversion unit 518. The conversion unit 518 is configured to convert the output signal y into an actuator control command 510. The control system 502 is configured to transmit the actuator control command 510 to the actuator 504, and the actuator 504 is configured to actuate the computer-controlled machine 500 in response to the actuator control command 510. In some embodiments, actuator 504 is configured to actuate computer-controlled machine 500 directly based on output signal y.

[0093] When the actuator 504 receives the actuator control command 510, the actuator 504 is configured to perform an action corresponding to the associated actuator control command 510. The actuator 504 may include control logic configured to transform the actuator control command 510 into a second actuator control command for controlling the actuator 504. In one or more embodiments, the actuator control command 510 may be used to control a display instead of or in addition to the actuator.

[0094] In some embodiments, instead of or in addition to the computer-controlled machine 500 including the sensor 506, the control system 502 includes the sensor 506. Instead of or in addition to the computer-controlled machine 500 including the actuator 504, the control system 502 may also include the actuator 504.

[0095] like Figure 5As shown, the control system 502 also includes a processor 520 and a memory 522. The processor 520 may include one or more processors. The memory 522 may include one or more memory devices. The classifier 514 (e.g., ML algorithm) of one or more embodiments may be implemented by the control system 502, which includes a non-volatile storage unit 516, a processor 520, and a memory 522.

[0096] The non-volatile storage 516 may include one or more persistent data storage devices, such as a hard drive, an optical drive, a tape drive, a non-volatile solid-state device, cloud storage, or any other device capable of persistently storing information. The processor 520 may include one or more devices selected from a high-performance computing (HPC) system, including a high-performance core, a microprocessor, a microcontroller, a digital signal processor, a microcomputer, a central processing unit, a field programmable gate array, a programmable logic device, a state machine, a logic circuit, an analog circuit, a digital circuit, or any other device that manipulates a signal (analog or digital) based on computer executable instructions residing in the memory 522. The memory 522 may include a single memory device or multiple memory devices, including but not limited to a random access memory (RAM), a volatile memory, a non-volatile memory, a static random access memory (SRAM), a dynamic random access memory (DRAM), a flash memory, a cache memory, or any other device capable of storing information.

[0097] The processor 520 may be configured to read into the memory 522 and execute computer executable instructions residing in the non-volatile storage 516 and embodying one or more anomaly detection methods of one or more embodiments. The non-volatile storage 516 may include one or more operating systems and applications. The non-volatile storage 516 may store compiled and / or interpreted computer programs created using various programming languages ​​and / or technologies, including but not limited to Java, C, C++, C#, Objective C, Fortran, Pascal, Java Script, Python, Perl, and PL / SQL, alone or in combination.

[0098] When executed by processor 520, the computer executable instructions of non-volatile storage 516 may cause control system 502 to implement one or more of the anomaly detection methods disclosed herein. Non-volatile storage 516 may also include data supporting the functions, features, and processes of one or more embodiments described herein.

[0099] The program code embodying the algorithms and / or methods described herein can be distributed individually or collectively as a program product in a variety of different forms. The program code can be distributed using a computer-readable storage medium having computer-readable program instructions thereon for causing a processor to implement aspects of one or more embodiments. Computer-readable storage media that are inherently non-transitory may include volatile and non-volatile and removable and non-removable tangible media implemented in any method or technology for storing information, such as computer-readable instructions, data structures, program modules, or other data. Computer-readable storage media may also include RAM, ROM, erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other solid-state memory technology, portable compact disk read-only memory (CD-ROM) or other optical storage, cassettes, tapes, disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and can be read by a computer. Computer-readable program instructions may be downloaded from a computer-readable storage medium to a computer, another type of programmable data processing device, or another device over a network, or to an external computer or external storage device.

[0100] Computer-readable program instructions stored in a computer-readable medium can be used to direct a computer, other types of programmable data processing devices, or other devices to operate in a particular manner, so that the instructions stored in the computer-readable medium produce an article of manufacture, including instructions that implement the functions, actions, and / or operations specified in the flowchart or chart. In certain alternative embodiments, the functions, actions, and / or operations specified in the flowchart and chart can be reordered, processed serially, and / or processed concurrently according to one or more embodiments. In addition, any one of the flowchart and / or chart may include more or fewer nodes or blocks than those shown in accordance with one or more embodiments.

[0101] A process, method or algorithm may be embodied in whole or in part using appropriate hardware components (e.g., application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), state machines, controllers, or other hardware components or devices) or a combination of hardware, software, and firmware components.

[0102] Figure 6A schematic diagram of a control system 502 configured to control a vehicle 600, which may be an at least partially autonomous vehicle or an at least partially autonomous robot, is depicted. The vehicle 600 includes an actuator 504 and a sensor 506. The sensor 506 may include one or more video sensors, cameras, radar sensors, ultrasonic sensors, LiDAR sensors, and / or location sensors (e.g., GPS). One or more of the one or more specific sensors may be integrated into the vehicle 600. Alternatively or in addition to the one or more specific sensors identified above, the sensor 506 may include a software module that is configured to determine the state of the actuator 504 when executed. A non-limiting example of a software module includes a weather information software module configured to determine current or future weather conditions near the vehicle 600 or other location.

[0103] The classifier 514 of the control system 502 of the vehicle 600 can be configured to detect an object near the vehicle 600 based on the input signal x. In such an embodiment, the output signal y may include information characterizing that the object is near the vehicle 600. The actuator control command 510 may be determined based on the information. The actuator control command 510 may be used to avoid a collision with the detected object.

[0104] In some embodiments, the vehicle 600 is an at least partially autonomous vehicle, and the actuator 504 may be embodied in the brakes, propulsion system, engine, drive train, or steering of the vehicle 600. Actuator control commands 510 may be determined to control the actuator 504 so that the vehicle 600 avoids colliding with a detected object. The detected objects may also be classified according to what the classifier 514 thinks they are most likely to be, such as a pedestrian or a tree. Based on the classification, the actuator control commands 510 may be determined. In scenarios where adversarial attacks may occur, the above system may be further trained to better detect objects or identify changes in lighting conditions or angles of sensors or cameras on the vehicle 600.

[0105] In some embodiments where the vehicle 600 is an at least partially autonomous robot, the vehicle 600 may be a mobile robot configured to perform one or more functions, such as flying, swimming, diving, and walking. The mobile robot may be an at least partially autonomous lawn mower or an at least partially autonomous cleaning robot. In these embodiments, the actuator control commands 510 may be determined so that a propulsion unit, a steering unit, and / or a braking unit of the mobile robot may be controlled so that the mobile robot may avoid colliding with the identified object.

[0106] In some embodiments, vehicle 600 is an at least partially autonomous robot in the form of a gardening robot. In such embodiments, vehicle 600 may use an optical sensor as sensor 506 to determine the state of plants in the environment near vehicle 600. Actuator 504 may be a nozzle configured to spray a chemical. Based on the identified species and / or the identified state of the plant, actuator control commands 510 may be determined to cause actuator 504 to spray the plant with an appropriate amount of an appropriate chemical.

[0107] The vehicle 600 may be an at least partially autonomous robot in the form of a household appliance. Non-limiting examples of household appliances include a washing machine, a stove, an oven, a microwave, or a dishwasher. In such a vehicle 600, the sensor 506 may be an optical sensor configured to detect the state of an object to be processed by the household appliance. For example, where the household appliance is a washing machine, the sensor 506 may detect the state of the laundry within the washing machine. The actuator control command 510 may be determined based on the detected state of the laundry.

[0108] Figure 7 A schematic diagram of a control system 502 of a system 700 (e.g., a manufacturing machine) configured to control a manufacturing system 702 (e.g., a portion of a production line), such as a punch cutter, a cutter, or a gun drill, is depicted. The control system 502 can be configured to control an actuator 504 configured to control the system 700 (e.g., a manufacturing machine).

[0109] The sensor 506 of the system 700 (e.g., a manufacturing machine) may be an optical sensor configured to capture one or more attributes of the manufactured product 704. The classifier 514 may be configured to determine a state of the manufactured product 704 based on the captured one or more attributes. The actuator 504 may be configured to control the system 700 (e.g., a manufacturing machine) for subsequent manufacturing steps of the manufactured product 704 based on the determined state of the manufactured product 704. The actuator 504 may be configured to control the function of the system 700 (e.g., a manufacturing machine) on the subsequent manufactured product 706 of the system 700 (e.g., a manufacturing machine) based on the determined state of the manufactured product 704.

[0110] Figure 8 A schematic diagram of a control system 502 configured to control a power tool 800 (eg, a drill or driver) having at least a partially autonomous mode is depicted. The control system 502 may be configured to control an actuator 504 configured to control the power tool 800.

[0111] The sensor 506 of the power tool 800 can be an optical sensor configured to capture one or more properties of the working surface 802 and / or the fastener 804 being driven into the working surface 802. The classifier 514 can be configured to determine the state of the working surface 802 and / or the fastener 804 relative to the working surface 802 based on the captured one or more properties. The state can be that the fastener 804 is flush with the working surface 802. Alternatively, the state can be the hardness of the working surface 802. The actuator 504 can be configured to control the power tool 800 so that the driving function of the power tool 800 is adjusted based on the determined state of the fastener 804 relative to the working surface 802 or one or more captured properties of the working surface 802. For example, if the state of the fastener 804 is flush with the working surface 802, the actuator 504 may no longer continue the driving function. As another non-limiting example, the actuator 504 can apply additional or less torque based on the hardness of the working surface 802.

[0112] Fig. 9 Depicted is a schematic diagram of a control system 502 configured to control an automated personal assistant 900. The control system 502 may be configured to control an actuator 504 configured to control the automated personal assistant 900. The automated personal assistant 900 may be configured to control a household appliance, such as a washing machine, stove, oven, microwave, or dishwasher.

[0113] Sensor 506 may be an optical sensor and / or an audio sensor. The optical sensor may be configured to receive a video image of user 902's gesture 904. The audio sensor may be configured to receive a voice command of user 902.

[0114] The control system 502 of the automated personal assistant 900 may be configured to determine an actuator control command 510 configured to control the system 502. The control system 502 may be configured to determine the actuator control command 510 based on the sensor signal 508 of the sensor 506. The automated personal assistant 900 is configured to transmit the sensor signal 508 to the control system 502. The classifier 514 of the control system 502 may be configured to execute a gesture recognition algorithm to identify a gesture 904 made by the user 902, determine the actuator control command 510, and transmit the actuator control command 510 to the actuator 504. The classifier 514 may be configured to retrieve information from the non-volatile storage in response to the gesture 904 and output the retrieved information in a form suitable for receipt by the user 902.

[0115] Fig.10A schematic diagram of a control system 502 configured to control a monitoring system 1000 is depicted. The monitoring system 1000 can be configured to physically control access through a door 1002. A sensor 506 can be configured to detect scenes relevant to a decision whether to grant access. The sensor 506 can be an optical sensor configured to generate and transmit image and / or video data. The control system 502 can use such data to detect a person's face.

[0116] The classifier 514 of the control system 502 of the monitoring system 1000 can be configured to interpret the image and / or video data by matching the identities of known persons stored in the non-volatile storage 516 to determine the identity of the person. The classifier 514 can be configured to generate an actuator control command 510 in response to the interpretation of the image and / or video data. The control system 502 is configured to transmit the actuator control command 510 to the actuator 504. In this embodiment, the actuator 504 can be configured to lock or unlock the door 1002 in response to the actuator control command 510. In some embodiments, non-physical logical access control can also be performed.

[0117] The monitoring system 1000 may also be a surveillance system. In such an embodiment, the sensor 506 may be an optical sensor configured to detect a scene under surveillance, and the control system 502 is configured to control the display 1004. The classifier 514 is configured to determine a classification of the scene, for example, whether the scene detected by the sensor 506 is suspicious. The control system 502 is configured to transmit the actuator control command 510 to the display 1004 in response to the classification. The display 1004 may be configured to adjust the displayed content in response to the actuator control command 510. For example, the display 1004 may highlight an object that the classifier 514 considers suspicious. Using embodiments of the disclosed system, the surveillance system can predict that an object will appear at a certain time in the future.

[0118] Fig.11 A schematic diagram of a control system 502 configured to control an imaging system 1100, such as an MRI device, an x-ray imaging device, or an ultrasound device, is depicted. For example, the sensor 506 can be an imaging sensor. The classifier 514 can be configured to determine a classification of all or part of a sensed image. The classifier 514 can be configured to determine or select an actuator control command 510 in response to the classification obtained by the trained neural network. For example, the classifier 514 can interpret an area of ​​the sensed image as a potential abnormality. In this case, the actuator control command 510 can be determined or selected to cause the display 1102 to display the imaging and highlight the potential abnormal area.

[0119] Although exemplary embodiments are described above, these embodiments are not intended to describe all possible forms covered by the claims. The words used in this specification are descriptive words, not restrictive words, and it should be understood that various changes can be made without departing from the spirit and scope of the present disclosure. As mentioned above, the features of various embodiments can be combined to form further embodiments of the present invention, which may not be explicitly described or illustrated. Although various embodiments may have been described as providing advantages or being superior to other embodiments or prior art implementations in terms of one or more required characteristics, it is recognized by those of ordinary skill in the art that one or more features or characteristics can be weighed to achieve the desired overall system properties, depending on the specific application and implementation. These properties may include, but are not limited to, cost, strength, durability, life cycle cost, marketability, appearance, packaging, size, applicability, weight, manufacturability, ease of assembly, etc. Therefore, insofar as any embodiment is described as being less desirable than other embodiments or prior art implementations in terms of one or more characteristics, these embodiments are not outside the scope of the present disclosure and may be desirable for specific applications.

Claims

1. A method for obtaining an uncertainty attribution for a prediction of an object in an input image, the method comprising: receiving an input image; Generate predictions of objects in the input image; estimating the uncertainty associated with the prediction of objects in the input image; computing an uncertainty attribution representing a region of the input image that causes the estimated uncertainty, wherein computing the uncertainty attribution comprises generating a plurality of adversarial gradients, each adversarial gradient corresponding to a modification of the input image configured to change the estimated uncertainty; as well as Generates output indicating the computed uncertainty attributions.

2. The method of claim 1 , wherein generating multiple adversarial gradients comprises performing at least one of: (i) introducing noise into an input image and (ii) modifying selected pixels of the input image.

3. The method of claim 1 , wherein generating a plurality of adversarial gradients comprises identifying at least one adversarial gradient that causes the estimated uncertainty to drop below a threshold.

4. The method of claim 1 , wherein generating a plurality of adversarial gradients comprises identifying an adversarial gradient that causes the estimated uncertainty to drop below a threshold with a minimal modification to the input image. The method of claim 1 , wherein calculating the uncertainty attribution comprises identifying the uncertainty attribution for the selected object class.

6. The method according to claim 5 further comprises: to identify uncertainty attribution, where corresponds to the uncertainty attribution of category c along the integration path, H(x) corresponds to the uncertainty, and f c (x) is the classification score corresponding to category c and input x.

7. The method of claim 1 further comprising modifying an input image based on the calculated uncertainty attribution and generating a second prediction of the object in the modified input image, wherein modifying the input image comprises masking a portion of the image corresponding to the calculated uncertainty attribution.

8. A computing device configured to obtain a predicted uncertainty attribution for an object in an input image, the computing device comprising a processing device configured to execute instructions stored in a memory to: receiving an input image; Generate predictions of objects in the input image; estimating the uncertainty associated with the prediction of objects in the input image; computing an uncertainty attribution representing a region of the input image that causes the estimated uncertainty, wherein computing the uncertainty attribution comprises generating a plurality of adversarial gradients, each adversarial gradient corresponding to a modification of the input image configured to change the estimated uncertainty; as well as Generates output indicating the computed uncertainty attributions.

9. The computing device of claim 8, wherein to generate the plurality of adversarial gradients, the processing device is configured to execute instructions to perform at least one of: (i) introducing noise into an input image and (ii) modifying selected pixels of the input image.

10. The computing device of claim 8, wherein to generate the plurality of adversarial gradients, the processing device is configured to execute instructions to identify at least one adversarial gradient that causes the estimated uncertainty to drop below a threshold.

11. The computing device of claim 8, wherein to generate the plurality of adversarial gradients, the processing device is configured to execute instructions to identify an adversarial gradient that causes the estimated uncertainty to drop below a threshold with a minimal modification to the input image.

12. The computing device of claim 8, wherein to compute the uncertainty attribution, the processing device is configured to execute instructions to identify the uncertainty attribution for the selected object class.

13. The method of claim 12, wherein the processing device is configured to execute instructions to to identify uncertainty attribution, where corresponds to the uncertainty attribution of category c along the integration path, H(x) corresponds to the uncertainty, and f c (x) is the classification score corresponding to category c and input x.

14. A computing device according to claim 8, wherein the processing device is configured to execute instructions to modify the input image based on the calculated uncertainty attribution and generate a second prediction of the object in the modified input image, and wherein to modify the input image, the processing device is configured to execute instructions to mask the image portion corresponding to the calculated uncertainty attribution.

15. A computer-controlled machine comprising: at least one sensor configured to generate an input image; The control system is configured as: receiving an input image; Generate predictions of objects in the input image; estimating the uncertainty associated with the prediction of objects in the input image; computing an uncertainty attribution representing a region of the input image that causes the estimated uncertainty, wherein computing the uncertainty attribution comprises generating a plurality of adversarial gradients, each adversarial gradient corresponding to a modification of the input image configured to change the estimated uncertainty; as well as generating an output signal indicative of the calculated uncertainty attribution; as well as An actuator is configured to control the operation of the computer-controlled machine in response to the output signal.

16. A computer-controlled machine according to claim 15, wherein to generate the plurality of adversarial gradients, the control system is configured to execute instructions to perform at least one of: (i) introducing noise into the input image and (ii) modifying selected pixels of the input image.

17. A computer-controlled machine according to claim 15, wherein to generate the plurality of adversarial gradients, the control system is configured to execute instructions to identify at least one adversarial gradient that causes the estimated uncertainty to drop below a threshold.

18. A computer-controlled machine according to claim 15, wherein to generate the plurality of adversarial gradients, the processor is configured to execute instructions to identify an adversarial gradient that causes the estimated uncertainty to drop below a threshold with a minimal modification to the input image.

19. The computer-controlled machine of claim 15, wherein the control system is configured to execute instructions to modify the input image based on the calculated uncertainty attribution and generate a second prediction of the object in the modified input image.

20. The computer controlled machine of claim 15, wherein the computer controlled machine comprises an autonomous robot.

Images