Hierarchical Trees for Multidimensional Outlier Detection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for identifying outliers in real-time or near real-time data streams, particularly in multidimensional data, fail to accommodate diverse user criteria for dimensions and sliding window sizes, leading to inefficient outlier detection and varying degrees of outlierliness measurement.
Innovation Solution
The implementation of hierarchical trees to organize data points by dimensions and scales, allowing for dynamic scale-based outlier detection that adjusts to varying data rates and provides a measure of outlierliness, with extended dyadic trees for continuous attributes and categorical attributes represented through hierarchical relationships.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If hierarchical trees are implemented to organize multidimensional data points, then outlier detection accuracy and adaptability to diverse user criteria are improved, but device complexity and computational overhead increase
Solution Approach 1:
The patent segments multidimensional data space into hierarchical tree structures, where each dimension is organized into separate trees with nodes representing data partitions. This segmentation allows efficient organization of complex multidimensional data while maintaining adaptability to different user criteria through selective tree traversal based on specified dimensions and window sizes.
Solution Approach 2:
The patent introduces a hierarchical dimension alongside traditional data dimensions, creating a multi-level structure where data points are organized by scale and dimension. This additional hierarchical dimension enables efficient querying across different granularities and accommodates diverse user criteria without requiring complete reprocessing of the entire dataset.
2Adaptability or versatility
If scale-based outlier detection is implemented to provide outlierliness measurement, then adaptability to varying data rates and user criteria is improved, but processing time and computational resources increase
Solution Approach 1:
The patent implements dynamic scale-based outlier detection where the hierarchical trees are maintained and updated continuously as new data points arrive. The scale parameter dynamically adjusts based on the local density of data points, allowing the system to adapt to varying data rates and distributions without requiring manual reconfiguration or complete reprocessing.
Solution Approach 2:
The patent performs preliminary organization of data into hierarchical tree structures before outlier detection is requested. This preliminary action pre-computes the spatial relationships and density information needed for scale-based detection, so that when queries are made with specific dimensions and window sizes, the system can quickly traverse the pre-organized structure rather than computing from scratch.
3Loss of information
If hierarchical trees organize data by multiple dimensions and scales, then the ability to identify outliers with measure of outlierliness is improved, but memory requirements and data structure complexity increase
Solution Approach 1:
The patent implements nested hierarchical trees where smaller-scale partitions are nested within larger-scale partitions. This nesting allows the system to store density and spatial information at multiple scales without duplicating all data at every level. The nested structure efficiently captures outlierliness measurements by maintaining hierarchical relationships, where each level provides context for detecting anomalies at that scale.
Data Source
AI summary
A technique that includes receiving a data stream that is indicative of a plurality of multi-dimensional points in a processor-based machine and for each dimension, organizing data indicative of values of the points in the dimension in an associated hierarchical tree. The technique includes using the processor-based machine to determine whether a given point of the plurality of points is an outlier based on a combination of the trees.