Neural Network for Automatic Outlier Detection System Selection
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for detecting outlying data in time series data lack efficiency and accuracy, often resulting in false positives and false negatives due to the need for manual selection of outlier detection systems, which are not tailored to specific types of data sets.
Innovation Solution
A neural network is trained to select an appropriate outlier detection system based on input time series data, using a statistical classifier and human review to create a training data set that allows for the automatic selection of the most suitable system for each data set, enabling faster and more accurate detection of outlying data.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If manual selection of outlier detection systems is used, then flexibility in choosing appropriate systems for different data types is achieved, but efficiency and productivity deteriorate due to time-consuming manual processes
Solution Approach 1:
The system automatically selects the appropriate outlier detection system by analyzing characteristics of the input time series data itself, without requiring manual intervention. The data 'services itself' by providing the information needed to determine the best detection approach through its inherent patterns and properties.
Solution Approach 2:
The system changes the parameter of system selection from manual choice to automated choice based on data characteristics. By analyzing data parameters such as seasonality, trend, and noise patterns, the system dynamically determines which outlier detection algorithm is most appropriate for each specific dataset.
2Device complexity
If a single outlier detection system is used for all data types, then device complexity is reduced, but measurement precision deteriorates due to false positives and false negatives
Solution Approach 1:
The system creates a universal outlier detection framework that can handle multiple data types and scenarios. Instead of requiring separate specialized systems for each data type, a single multi-functional system analyzes data characteristics and adapts its detection approach accordingly, achieving both simplicity and precision.
Solution Approach 2:
The system segments the outlier detection process into two parts: (1) analysis of data characteristics to determine appropriate detection parameters, and (2) application of the selected detection algorithm. This segmentation allows a single system to achieve the precision of multiple specialized systems by dynamically adjusting its behavior based on data type.
3Measurement precision
If manual selection process is used, then accuracy can be maintained through expert judgment, but loss of time increases due to manual review requirements
Solution Approach 1:
The system replaces the mechanical process of manual expert selection with an automated computational process. Instead of human experts manually reviewing and selecting detection systems, the system uses algorithmic analysis of data characteristics to automatically determine the appropriate outlier detection approach, eliminating time loss while maintaining accuracy.
Solution Approach 2:
The system performs preliminary analysis of data characteristics before applying outlier detection. By pre-analyzing data patterns, seasonality, and other relevant features, the system prepares the optimal detection parameters in advance, enabling fast and accurate detection without time-consuming manual selection during the actual detection process.
Data Source
AI summary
Systems, device and techniques are disclosed for outlier discovery system selection. A set of time series data including time series data objects may be received. A sample of time series data objects may be extracted from the time series data. The sample of time series data objects may be decomposed into sub-components. Statistical classification may be used to select an outlier discovery system based on the sub-components. A neural network may be used to select an outlier discovery system based on the sub-components. A level of error of the neural network may be determined based on a comparison of the outlier discovery system selection made using statistical classification and the outlier discovery system selection made by the neural network. Weight of the neural network may be updated based on the level of error of the neural network.