DNN Anomaly Detection Using Feature Subspace Distance
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Deep neural networks (DNNs) often fail to accurately detect anomalies in input data, leading to unexpected outputs that can cause errors in controlling vehicles, robots, security systems, and manufacturing operations, as they are trained on specific datasets and may misidentify objects that differ significantly from those in the training data.
Innovation Solution
The method involves using a K-means single value decomposition algorithm on an overcomplete dictionary of feature vectors to determine a sparse representation, allowing for the identification of anomalies by measuring the distance of feature vectors to a subspace, and using a greedy orthogonal matching pursuit algorithm to project feature vectors and determine if an object corresponds to an anomaly based on a threshold of three standard deviations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If deep neural networks are trained on specific datasets to perform computing tasks, then they can accurately detect objects in familiar environments, but they fail to accurately detect anomalies that differ significantly from training data
Solution Approach 1:
The system pre-computes a subspace from feature vectors of training data before actual object detection. This preliminary subspace serves as a reference framework that enables the DNN to identify anomalies by measuring deviation from normal patterns, without requiring retraining for each new anomaly type.
Solution Approach 2:
A subspace constructed from training feature vectors acts as an intermediary between the trained DNN and anomaly detection. This subspace mediates the comparison between new feature vectors and normal patterns, enabling anomaly identification through distance measurement rather than direct classification.
2Reliability
If deep neural networks rely on training data to make predictions, then they provide consistent results for familiar objects, but they produce unexpected outputs when encountering anomalous objects
Solution Approach 1:
The patent replaces the traditional classification-based DNN output mechanism with a distance-based anomaly detection mechanism. Instead of forcing anomalous objects into pre-defined classes, the system measures the distance of feature vectors from the training subspace, substituting categorical classification with continuous distance measurement.
3Adaptability or versatility
If the system uses distance measurement to identify anomalies, then it can detect objects that differ from training data, but it requires additional computational steps beyond standard DNN inference
Solution Approach 1:
The subspace is pre-computed and stored during the training phase, containing all the necessary information about normal patterns. During inference, only simple distance measurements are required, avoiding the need for complex real-time computations while maintaining anomaly detection capability.
Data Source
AI summary
An image including a first object can be input to a deep neural network trained to detect objects. The deep neural network can output a first feature vector corresponding to the first object. A first distance can be measured from the first feature vector to a feature vector subspace determined using a k-means single value decomposition algorithm on an overcomplete dictionary of feature vectors. The first object can be determined to correspond to an anomaly based on the first distance.