Categorical Missing-Value Filling for Irregular Anomaly Detection Data
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing anomaly detection systems in cyber-physical systems face challenges due to the prevalence of irregularly sampled categorical time-series data with numerous missing values, leading to sub-par performance and noise introduction from current interpolation methods.
Innovation Solution
A Sparse and Irregular time series Processing Tool (SIPT) aligns irregular time-series data using a generated timestamp sequence, fills missing values with categorical data, and filters data to improve anomaly detection, utilizing neural networks for accurate pattern recognition and corrective actions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If current interpolation methods are used to fill missing values in irregular time-series data, then the data can be completed, but noise is introduced and detection accuracy deteriorates
Solution Approach 1:
The patent extracts and removes problematic missing values from the time-series data rather than interpolating them. By identifying and extracting the irregularly sampled data points that cause noise, the system prevents noise introduction while maintaining data integrity for anomaly detection.
Solution Approach 2:
The patent performs preliminary alignment of irregular time-series data to a regular timestamp sequence before anomaly detection. This preliminary action of organizing data into a consistent temporal structure prevents subsequent processing issues and improves detection accuracy without introducing noise.
2Loss of information
If irregularly sampled categorical time-series data with missing values is used directly, then data loss is minimized, but anomaly detection performance deteriorates
Solution Approach 1:
The patent introduces a regular timestamp sequence as an intermediary structure between the irregularly sampled data and the anomaly detection algorithm. This intermediary framework allows the system to maintain data completeness while providing a consistent temporal structure that improves detection performance.
Solution Approach 2:
The patent changes the temporal sampling parameter from irregular to regular by aligning data points to a standardized timestamp sequence. This parameter transformation maintains the original data information while improving the structural consistency required for reliable anomaly detection.
3Measurement precision
If data alignment and filling processes are implemented, then data quality improves, but computational complexity increases
Solution Approach 1:
The patent segments the data processing into distinct stages: alignment to timestamp sequence, identification of missing values, and categorical filling. This segmentation allows each step to be optimized independently, improving data quality while managing computational complexity through modular processing.
Data Source
AI summary
Systems and methods for generating categorical data for missing values in anomaly detection systems. In an embodiment, irregular time-series data can be aligned into regular time-series data by utilizing a generated timestamp sequence to obtain aligned time-series data. Missing values from the aligned time-series data can be filled with generated categorical time-series data. Anomaly detection can be performed for the cyber-physical system to obtain system anomalies. A corrective action can be performed to resolve issues with the cyber-physical system caused by the system anomalies.