LSTM Autoencoder Thresholding for Out-of-Distribution Anomaly Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing generative adversarial networks (GANs) face challenges in effectively detecting anomalies, particularly in discrete and non-continuous data types like text, due to limitations in matching distributions and mode collapse, and lack of effective methods for recognizing out-of-distribution data.
Innovation Solution
A semi-supervised neural network framework incorporating a generator and discriminator in a zero-sum game setting, combined with an LSTM autoencoder, to detect anomalies by minimizing Jenson-Shannon Divergence and using reconstruction error thresholds to identify out-of-distribution samples.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If GAN is optimized to match data distribution of real samples, then the fake distribution gets closer to real distribution, but this makes anomaly detection more difficult as the discriminator cannot distinguish anomalies effectively
Solution Approach 1:
The patent segments the anomaly detection process into two distinct phases: (1) training phase where the generator learns to match the real data distribution, and (2) detection phase where the discriminator identifies anomalies. This segmentation allows the system to maintain both distribution matching capability and anomaly detection precision by operating at different stages.
Solution Approach 2:
The patent applies preliminary action by training the generator to match the real data distribution before the actual anomaly detection occurs. This preliminary training of the generator on normal data establishes a baseline distribution that the discriminator can later compare against to identify deviations, thereby enabling effective anomaly detection.
2Manufacturing precision
If the generator produces realistic data samples, then the quality of generated data improves, but the ability to detect out-of-distribution anomalies decreases
Solution Approach 1:
The patent introduces the discriminator as an intermediary component that mediates between the generator's realistic data production and the anomaly detection requirement. The discriminator acts as a separate evaluation module that can identify out-of-distribution samples even when the generator produces high-quality realistic data, thus resolving the contradiction between data quality and detectability.
3Measurement precision
If the discriminator becomes more skilled at flagging synthetic images, then classification accuracy improves, but the system becomes more complex with independent backpropagation procedures
Solution Approach 1:
The patent implements feedback mechanisms where the discriminator's classification results are fed back to the generator through the loss function, creating a continuous improvement loop. This feedback-based training allows both networks to be updated dynamically with coordinated optimization, managing the complexity through established GAN training protocols while maintaining high classification accuracy.
Data Source
AI summary
A data anomaly detection method and apparatus in which a deep neural network is trained on baseline data. Sequences of statistics of each layer of the deep neural network are saved, processed and used to train an LSTM autoencoder across a variety of reconstruction error thresholds, and a preferred threshold is selected for an optimized autoencoder. In an Inference mode, a data sample is presented to the autoencoder; the reconstruction error is calculated and compared to the threshold. If it is above the threshold, then the data sample is an out-of-distribution sample, and the sample is tagged as anomalous.