Abnormal Data Detection via Temporal Segmentation and Correlation Analysis
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Detecting abnormal data points in databases is challenging due to the difficulty in distinguishing between peak and normal periods, as conventional methods based on average values may fail to accurately identify anomalies during peak periods.
Innovation Solution
Classifying data points into potential normal and abnormal groups based on time periods, analyzing correlations among abnormal data points, and identifying outliers using correlation coefficients and normal data ranges determined by average values and standard deviations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of manufacture
If conventional methods based on average values are used to detect abnormal data points, then the detection process is simple, but the accuracy deteriorates during peak periods
Solution Approach 1:
The patent segments the time series data into multiple groups based on time periods (e.g., peak periods, normal periods). This segmentation allows the system to apply different detection criteria to different time segments, resolving the contradiction by maintaining simplicity within each segment while improving overall accuracy through context-aware analysis.
Solution Approach 2:
The patent dynamically adjusts the detection method based on the time period. During peak periods, the system uses correlation analysis with other data groups to identify abnormalities, while during normal periods, it uses standard deviation-based methods. This dynamic adaptation maintains accuracy across varying conditions without requiring a completely complex unified system.
2Measurement precision
If data points are classified into multiple groups based on time periods, then the accuracy of abnormal data detection is improved, but the device complexity increases
Solution Approach 1:
The system divides data into temporal segments (groups) based on time periods. This segmentation enables accurate detection by considering the temporal context of each data point, improving accuracy without requiring complex analysis of every individual point in isolation.
Solution Approach 2:
The patent introduces correlation analysis as an intermediary mechanism between data groups. By measuring correlations between different data groups, the system can identify abnormalities indirectly, reducing the need for direct complex analysis of each data point and simplifying the overall detection logic.
3Reliability
If correlation analysis is performed between groups of potential abnormal data points, then the reliability of abnormal data identification is improved, but the processing time increases
Solution Approach 1:
The patent segments data into time-based groups and performs correlation analysis selectively between these segments rather than on all data points uniformly. This segmentation allows the system to focus computational resources on the most relevant comparisons, improving reliability while controlling processing time through targeted analysis.
Solution Approach 2:
The system performs correlation analysis only between groups of potential abnormal data points rather than exhaustively analyzing all possible combinations. This partial action approach maintains sufficient reliability for detection while significantly reducing the processing time required compared to complete exhaustive analysis.
Data Source
AI summary
A method, system, and computer program product for abnormal data detection. According to the method, a plurality of data points collected at different time points are classified into a plurality of groups. A plurality of groups of potential abnormal data points are determined from the plurality of groups. Correlations between a first group of the plurality of groups of potential abnormal data points with other groups of potential abnormal data points are determined. In response to the first group of the plurality of groups of potential abnormal data points being uncorrelated to a majority of the other groups of potential abnormal data points based on the correlations, data points in the first group are identified as abnormal data points.