Out-of-Distribution Input Detection With Skipped-Layer Gates
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for detecting out-of-distribution (OOD) inputs in neural networks are inefficient and inaccurate, particularly in dynamic neural networks (DyNNs), as they rely on unreliable exit calculations based on input complexity and limited exits, leading to significant computing resource waste and incorrect predictions.
Innovation Solution
A skipping mechanism is employed in neural networks that estimates probabilities using deep deterministic uncertainty (DDU) or energy scores at multiple gates, allowing for efficient detection by skipping layers and determining OOD inputs based on a predetermined threshold, discarding inputs if a certain number of gates predict them as OOD.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If conventional OOD detection methods use full inference to detect OOD inputs, then detection accuracy is improved, but computing resource consumption increases significantly
Solution Approach 1:
The neural network is segmented into multiple gates (first gate, second gate, third gate) that can independently evaluate OOD probability. Instead of requiring full inference through all layers, each gate processes the input separately and contributes to the overall OOD detection decision, enabling partial inference that reduces computational resource consumption while maintaining detection accuracy.
Solution Approach 2:
The system performs partial inference by evaluating only the necessary gates based on the input characteristics. The OOD detection mechanism stops processing after sufficient gates have evaluated the input, avoiding unnecessary computation through all network layers while maintaining adequate detection accuracy.
2Productivity
If early-exit DyNN uses multiple exits for OOD detection, then detection efficiency is improved, but the reliability of complexity-based exit selection deteriorates
Solution Approach 1:
The patent replaces the unreliable mechanical/algorithmic complexity calculation (bit-length of compressed input) with a probabilistic approach using deep deterministic uncertainty (DDU) values. Each gate outputs a DDU value that represents the uncertainty of the input, providing a more reliable and trustworthy basis for exit selection that adapts to the specific architecture and data characteristics of the neural network.
Solution Approach 2:
The system changes the parameter used for exit selection from input complexity (bit-length) to deep deterministic uncertainty values generated by each gate. This parameter transformation makes the exit selection process more reliable by using a metric that directly reflects the network's confidence in its prediction rather than a proxy measure like compressed input length.
3Device complexity
If early-exit DyNN limits the number of exits, then architecture simplicity is maintained, but OOD detection capability deteriorates
Solution Approach 1:
The neural network is divided into multiple gates (first gate, second gate, third gate) that can independently perform OOD detection. This segmentation allows the system to maintain a relatively simple architecture at each individual gate while achieving comprehensive OOD detection capability through the collective action of multiple gates, avoiding the need for complex multi-exit DyNN structures.
Solution Approach 2:
Each gate in the neural network is designed to serve multiple functions: it performs its primary function in the neural network computation while simultaneously providing OOD detection capability through DDU value calculation. This multi-functionality allows the system to achieve robust OOD detection without adding separate dedicated detection components or complex exit mechanisms.
Data Source
AI summary
A method and a system for using a skipping mechanism to automatically detect out-of-distribution (OOD) inputs to neural networks in an efficient and accurate manner are provided. The method includes: receiving a proposed input to a neural network at a first gate of the neural network; estimating, based on an output generated by the first gate, a first probability that the proposed input is classifiable as being OOD; forwarding the first proposed input to at least one additional gate of the neural network, including skipping at least one layer of the neural network; estimating, based on a respective output generated by each respective additional gate, a corresponding probability that the proposed input is classifiable as being OOD; and determining, based on the estimated probabilities, whether the proposed input is classifiable as being OOD by determining whether at least a minimum number of the estimated probabilities exceed a predetermined threshold.