Neural Network Activation Profiling for Prediction Confidence
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep Learning Neural Networks (DLNNs) struggle with inaccurate self-estimates of performance due to learned confidence values that are susceptible to the same errors as the network prediction, particularly on inputs where the model performs poorly, and existing methods for improving confidence either rely on input space distribution or embedding space, which are not always accurate or require additional computational steps.
Innovation Solution
A technique that treats the entire activation state of the network as a multivariate sample, recording activation data during training to create a measurement log that determines if a novel input produces an activation pattern out of distribution from the training data, using this log to assess confidence during inference without additional training steps.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If the network uses its own output to estimate confidence, then the confidence value is generated efficiently, but the confidence becomes inaccurate for inputs where the model performs poorly
Solution Approach 1:
The patent introduces an intermediary measurement log that records activation data from a reference model during training. This log serves as a mediator between the input data and the confidence estimation, allowing the current model to compare its activations against the recorded reference activations to determine confidence without relying on potentially erroneous self-estimates.
Solution Approach 2:
The patent performs preliminary action by recording activation data from the reference model during the training phase and storing it in a measurement log. This pre-recorded data is then available during inference to enable accurate confidence estimation without requiring additional training steps or computational resources at inference time.
2Measurement precision
If additional computational steps are added during training to improve confidence, then confidence accuracy improves, but training time and computational cost increase
Solution Approach 1:
The patent performs the computationally intensive task of recording and storing activation data during training as a preliminary action. By completing this data collection and processing work during training and storing it in a measurement log, the patent enables accurate confidence estimation during inference without requiring additional computational steps at that stage.
Solution Approach 2:
The patent creates a copy of the reference model's activation patterns by recording them in the measurement log during training. This copied information is then available during inference to enable confidence estimation without requiring the full reference model to run again, thus avoiding additional training time while maintaining accuracy.
3Ease of operation
If the network treats all inputs equally, then processing is simplified, but the network cannot distinguish between in-distribution and out-of-distribution inputs
Solution Approach 1:
The patent applies local quality by computing confidence values individually for each neuron based on its activation patterns. Each neuron's activation is compared against its recorded reference activation in the measurement log to determine local confidence, allowing the system to distinguish between reliable and unreliable predictions at the neuron level and aggregate these to determine overall prediction reliability.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
The invention relates to a technique for improving confidence estimates associated with neural networks. The technique involves computing neuron activation statistics during training, evaluating neuron activations during inferencing and determining how the activations compare with the previously computed statistics (e.g. whether prediction activations are within the bounds of the training activation statistics). The comparison may be used to compute a confidence value for the neural network.