Likelihood Ratio OOD Detection for Background-Shifted Inputs
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing deep generative model-based approaches for out-of-distribution (OOD) detection fail to distinguish between semantic content relevant for classification and background statistics, often assigning higher likelihood to OOD inputs, leading to erroneous classifications.
Innovation Solution
A likelihood ratio method using a semantic model trained on in-distribution data and a background model trained on perturbed in-distribution data to correct for background statistics, enhancing OOD detection by calculating a likelihood ratio that isolates semantic components.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If a deep generative model is trained on training data to detect OOD inputs, then the model can identify anomalous data, but the model assigns higher likelihood to OOD inputs than in-distribution inputs, leading to erroneous classifications
Solution Approach 1:
The patent segments the likelihood evaluation into two separate components: a semantic model that evaluates class-specific features and a background model that evaluates overall data distribution. By dividing the single generative model into these two specialized components, the system can separately assess semantic relevance and distributional likelihood, preventing OOD inputs from receiving erroneously high likelihood scores while maintaining accurate in-distribution classification.
2Measurement precision
If a neural network classifier is deployed to classify bacterial genomes, then high accuracy is achieved on training classes, but the classifier wrongly classifies OOD inputs from unseen bacterial classes with high confidence
Solution Approach 1:
The patent introduces an intermediary OOD detection layer between the neural network classifier and the final classification output. This intermediary component evaluates whether an input belongs to any known in-distribution class before allowing classification. By adding this intermediate check, the system maintains high classification accuracy for known classes while preventing confident misclassification of unseen bacterial classes as known classes.
3Productivity
If the model processes all inputs through full classification, then comprehensive analysis is performed, but computing resources are wasted on OOD inputs that cannot be correctly classified
Solution Approach 1:
The patent performs preliminary OOD detection before executing the full classification pipeline. By first evaluating whether an input belongs to the training data distribution using the background model, the system can early-screen and exclude OOD inputs from resource-intensive full classification processing. This preliminary action significantly reduces computing resource consumption on OOD inputs while maintaining comprehensive analysis for valid in-distribution inputs.
Data Source
Figure 1A
Figure 1B
Figure 1C
AI summary
The present disclosure is directed to systems and method to perform improved detection of out-of-distribution (OOD) inputs. In particular, current deep generative model-based approaches for OOD detection are significantly negatively affected by and struggle to distinguish population level background statistics from semantic content relevant to the in-distribution examples. In fact, such approaches have even been experimentally observed to assign higher likelihood to OOD inputs, which is opposite to the desired behavior. To resolve this problem, the present disclosure proposes a likelihood ratio method for deep generative models which effectively corrects for these confounding background statistics.