Ensemble Model OOD Detection via Adversarial Mutual Information
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing machine learning systems face challenges in accurately detecting out-of-distribution (OOD) data with high computational costs and inability to maintain a false positive detection rate of less than or equal to 1%, particularly failing to reliably identify samples containing Gaussian random noise.
Innovation Solution
The method involves creating an ensemble machine learning model by training two machine learning models together, perturbing the training dataset with mutual information to generate an adversarial dataset, and using this dataset to train the ensemble model, which then detects OOD data with a low false positive rate through improved uncertainty estimates.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If prior methods use uncertainty estimates to improve OOD detection rate, then detection accuracy is improved, but computational costs increase significantly
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing uncertainty estimates during the training phase, rather than computing them during inference. The model learns to associate feature representations with uncertainty levels, enabling fast OOD detection at runtime without the high computational cost of re-computing uncertainty estimates.
Solution Approach 2:
The patent creates a copy of the training process by training multiple models with different initializations or architectures that all process the same training data. These models share computational resources and can be ensembled, reducing overall computational cost while maintaining detection accuracy through diverse perspectives.
2Measurement precision
If prior methods increase uncertainty estimation quality, then OOD detection rate improves, but false positive rate cannot be maintained at ≤1%
Solution Approach 1:
The patent applies local quality by assigning different uncertainty thresholds or detection criteria to different regions or types of data inputs. The model learns to distinguish between different kinds of OOD samples and applies appropriate detection sensitivity to each, reducing false positives for benign variations while maintaining high detection rates for actual OOD data.
Solution Approach 2:
The patent implements feedback mechanisms where the model continuously refines its uncertainty estimates based on performance metrics and manual validation. This iterative process allows the system to learn from false positives and adjust its detection thresholds, progressively improving reliability while maintaining high detection accuracy.
3Loss of time
If prior methods use standard training techniques, then training time is reduced, but ability to detect Gaussian noise OOD samples fails
Solution Approach 1:
The patent applies preliminary action by incorporating Gaussian noise augmentation during the training phase, where the model is pre-exposed to noisy variations before actual inference. This preliminary exposure teaches the model to distinguish between noise and meaningful patterns, enabling detection of Gaussian noise OOD samples without significantly extending training time due to efficient batch processing.
Solution Approach 2:
The patent applies parameter changes by dynamically adjusting training parameters such as learning rates, batch sizes, and noise injection levels to optimize both training speed and detection capability. The model adapts its internal parameters during training to balance computational efficiency with enhanced ability to detect subtle Gaussian noise variations.
Data Source
AI summary
Provided are systems, methods, and computer program products including at least one processor programmed or configured to perturb at least one training dataset based on mutual information extracted from an ensemble machine learning model to provide at least one adversarial training dataset, execute at least two machine learning models of an ensemble machine learning model, train at least two machine learning models with the at least one training dataset by feeding an input or output of one of the at least two machine learning models to the other of the at least two machine learning models, train the ensemble machine learning model with the at least one adversarial training dataset, receive a runtime input from a client device, and provide the runtime input to the trained ensemble machine learning model to generate a signal output indicating that the runtime input includes an out-of-distribution sample.