Anomaly Detection Latent Space on a Closed Hypersphere
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In anomaly detection using variational auto-encoders, the latent space being a Euclidean space leads to potential overflow issues due to infinitely positive and negative values, making it difficult to process anomalies effectively.
Innovation Solution
The latent space is transformed into a closed manifold, specifically a hypersphere, where normal and anomalous distributions are learned, and anomaly scores are calculated based on distances between reconstruction results and these distributions, using a von Mises-Fisher distribution and clipping techniques to manage output ranges.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If a Euclidean space is used as the latent space in a variational auto-encoder, then the model can represent normal and anomalous distributions with infinite range, but this leads to overflow issues in computer processing due to infinitely positive and negative values
Solution Approach 1:
The patent transforms the Euclidean latent space into a spherical manifold by constraining the encoded representations to lie on a unit sphere. This is achieved by normalizing the encoded vectors to have unit length, thereby converting the infinite-range Euclidean space into a finite spherical space while preserving the ability to represent both normal and anomalous distributions.
Solution Approach 2:
The patent changes the parameterization of the latent space from unbounded Euclidean coordinates to bounded spherical coordinates. By representing points on a sphere rather than in infinite Euclidean space, the model maintains representational flexibility while ensuring all values remain within computationally safe bounds.
2Reliability
If the latent space is transformed into a closed manifold (hypersphere), then overflow is prevented and detection performance improves, but the mathematical complexity of working with manifold spaces increases
Solution Approach 1:
The patent adopts a spherical manifold as the latent space, which provides finite bounds to prevent overflow while maintaining geometric properties useful for anomaly detection. The spherical structure naturally constrains all encoded representations to a bounded space without requiring complex clipping operations.
Solution Approach 2:
The patent replaces the need for complex overflow protection mechanisms with a geometric constraint inherent to spherical manifolds. Instead of using clipping or saturation functions to prevent overflow, the spherical structure itself ensures all values remain bounded, simplifying the computational approach.
3Measurement precision
If normal data and anomalous data are both used for learning, then the model can better distinguish anomalies, but the significant data amount bias between normal and anomalous data makes learning difficult
Solution Approach 1:
The patent applies different learning objectives to different data types within the same model framework. Normal data is used to learn the typical spherical distribution patterns, while anomalous data is used to learn deviations from these patterns. This local differentiation allows the model to handle the data imbalance by treating each data type according to its specific characteristics.
Solution Approach 2:
The spherical manifold structure provides a natural framework for handling imbalanced data by constraining all representations to a bounded space. This geometric constraint helps the model focus on directional relationships rather than magnitude differences, making it more robust to the significant imbalance between normal and anomalous data quantities.
Data Source
AI summary
A possible region of encoding results of anomalous samples is limited. An encoder storage unit 14 stores an encoder for projecting an input feature value into a latent space in which the latent space is a closed manifold, a normal distribution obtained by learning normal data and an anomalous distribution obtained by learning anomalous data are held on the manifold, and a decoder for reconstructing the output of the encoder. An encoding unit 15 obtains a reconstruction result output by the decoder when a feature value of target data is input to the encoder. An anomaly score calculation unit 16 calculates an anomaly score of the target data based on distances between the reconstruction result and the normal distribution and distances between the reconstruction result and the anomalous distribution.