Anomaly Detection via Reconstruction Error Thresholding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing neural network models face challenges in anomaly detection, particularly with imbalanced data distributions and out-of-distribution data, leading to incorrect decisions and reduced performance in applications like autonomous driving and medical diagnostics, where the softmax function fails to provide confidence in identifying in-distribution or out-of-distribution data.
Innovation Solution
The method employs an artificial neural network (ANN) to extract features, generate a latent representation, and compute a reconstruction error, determining anomalies by comparing the error to a predefined threshold, allowing for the detection of out-of-distribution data and reducing the memory footprint by using tied weights in autoencoders.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If softmax function is used for classification, then classification output is generated, but confidence in identifying in-distribution or out-of-distribution data is not provided
Solution Approach 1:
The patent introduces a reconstruction model as an intermediary component between the input data and the classification decision. This reconstruction model generates a reconstructed version of the input data, and the difference between the original and reconstructed data serves as an anomaly score. This intermediary mechanism provides the missing confidence information about data distribution without replacing the softmax classification function.
Solution Approach 2:
The patent segments the anomaly detection task into two independent components: (1) the original classification pathway using softmax for category prediction, and (2) a separate reconstruction pathway using an autoencoder-like structure for anomaly scoring. This segmentation allows both functions to operate independently and provide complementary information without interfering with each other.
2Reliability
If neural network model size is increased to improve anomaly detection, then detection capability is enhanced, but memory footprint increases
Solution Approach 1:
The patent uses a reconstruction model that learns to copy the essential features of the input data in a compressed latent representation. Instead of storing or processing the entire high-dimensional input data, the model creates a compact copy in the latent space that captures the most important patterns, thereby reducing memory requirements while maintaining detection capability.
Solution Approach 2:
The patent extracts only the essential features needed for anomaly detection by passing input data through an encoder that transforms high-dimensional input into a lower-dimensional latent representation. This extraction process removes redundant information and retains only the critical patterns, reducing memory footprint while preserving detection accuracy.
3Reliability
If feature extraction is performed to generate latent representation, then anomaly detection is enabled, but computational complexity increases
Solution Approach 1:
The patent transforms the anomaly detection problem from the original high-dimensional input space into a lower-dimensional latent space. By changing the dimensionality through the encoder, the model simplifies the computational task of detecting anomalies while maintaining the essential information needed for accurate detection.
Solution Approach 2:
The patent performs feature extraction and dimensionality reduction as a preliminary action before the actual anomaly detection and classification steps. The encoder pre-processes the input data by extracting essential features and organizing them in a latent representation, which simplifies subsequent processing and reduces computational complexity in later stages.
Data Source
AI summary
A method of anomaly detection and energy-efficient inference determination includes receiving an input. A set of features of the input are extracted using an artificial neural network (ANN) to generate a latent representation of the input. A reconstruction of the input is generated using the ANN, based on the latent representation. A reconstruction error is computed based on the generated reconstruction and the input. The reconstruction error is compared to a predefined threshold to determine whether the in-distribution data or out-of-distribution data. An anomaly is detected in response to an out-of-distribution determination. A decision model is provided with the latent representation in response to the input being determined to be in-distribution data. In turn, the decision model computes an inference based on the latent representation.