Hybrid Anomaly Classification Using Distance Filtering and Density Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly classification methods perform poorly in data sets with varying densities and distances, as they struggle to accurately distinguish between anomalous and normal data points, especially when normal data has areas of varying densities or outliers with uniform relative density.
Innovation Solution
A hybrid method combining absolute distance and relative density criteria, where data sets are filtered based on absolute distance to clean the data and then classified using relative density, enhancing anomaly detection accuracy.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If relative density-based anomaly classification is applied directly to the original data set, then the method can handle varying densities, but distant outliers with uniform relative density incorrectly affect the classification accuracy
Solution Approach 1:
The data set is segmented into two parts: a training data set used to establish the density model, and a test data point whose anomaly status is to be determined. This segmentation prevents distant outliers in the training data from unduly influencing the classification of test points, as the model parameters are fixed after training.
Solution Approach 2:
Distant outliers are extracted and removed from the training data set through filtering based on distance metrics. By identifying and removing points that are excessively distant from the cluster center, the harmful influence of these outliers on density-based classification is eliminated while preserving the essential structure of normal data.
2Measurement precision
If distance-based filtering is applied to remove distant points, then classification accuracy improves, but the complexity of the method increases due to multiple processing steps
Solution Approach 1:
The method merges distance-based filtering and density-based classification into a unified hybrid approach. The distance filter and density classifier work together in sequence, where the filtering step prepares the data and the classification step leverages this preprocessing. This combination achieves high accuracy while maintaining reasonable complexity through modular design.
Solution Approach 2:
Distance-based filtering is performed as a preliminary action before applying density-based classification. By pre-processing the data to remove obvious distant outliers, the subsequent classification step operates on cleaner data, improving accuracy without requiring the classification algorithm itself to be overly complex.
Data Source
AI summary
A computer implemented method, a computerized system and a computer program product for anomaly classification. The computer implemented method comprises obtaining a data set, wherein the data set comprises a plurality of data points. The method further comprises filtering the data set based on an absolute distance criterion and performing anomaly classification on a test data point of the data set, wherein the anomaly classification is based on a relative density criterion. The method further comprises outputting an outcome of the anomaly classification.