Real-Time Transaction Anomaly Detection via Category Sampling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current transaction monitoring systems face inefficiencies in real-time anomaly detection due to a two-pass approach that is slow and I/O intensive, and they fail to consider temporal transaction distribution and overall operating conditions, leading to false positive alerts from the 'cold cache' phenomenon.
Innovation Solution
The system employs a real-time, false-positive resistant approach that samples and filters transaction trace data to identify important categories based on transaction frequency and concurrent load, creating baseline reference data that considers dependencies between performance and execution conditions, and dynamically adapts to transaction load to avoid false alerts.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If a two-pass approach is used to identify categories and create reference data, then comprehensive category identification is achieved, but the process becomes slow and I/O intensive
Solution Approach 1:
The patent applies preliminary action by pre-computing and storing category identifiers and statistical properties during the first pass, so that during the second pass only reference data creation is needed without re-reading transaction data. This separates category identification from reference data creation, improving overall productivity while maintaining accuracy.
Solution Approach 2:
The patent segments the monitoring process into distinct phases: first pass for category identification and statistical computation, second pass for reference data creation using stored category information. This segmentation avoids redundant I/O operations and improves efficiency.
2Reliability
If all transaction categories are monitored, then complete coverage is achieved, but the system becomes impossible to manage due to vast number of categories
Solution Approach 1:
The patent extracts only the most important transaction categories based on transaction frequency thresholds, rather than monitoring all possible categories. This reduces complexity while maintaining reliability for the most significant transaction types.
Solution Approach 2:
The patent applies different monitoring strategies to different categories: detailed statistical monitoring for high-frequency important categories, and simplified or no monitoring for low-frequency categories. This local differentiation reduces overall system complexity while maintaining coverage where it matters most.
3Device complexity
If transaction categories are identified based only on transaction count, then simple categorization is achieved, but temporal distribution and load-dependent behavior are ignored causing false positives
Solution Approach 1:
The patent changes the parameters used for category identification from simple transaction counts to include temporal distribution patterns and concurrent load conditions. This multi-parameter approach reduces false positives by accounting for the 'cold cache' phenomenon and load-dependent performance variations.
Solution Approach 2:
The patent incorporates feedback from historical performance data and concurrent load measurements to adjust anomaly detection thresholds. This feedback mechanism allows the system to learn normal performance variations under different load conditions and avoid false alarms.
Data Source
AI summary
A system and method for the distributed analysis of high frequency transaction trace data to constantly categorize incoming transaction data, identify relevant transaction categories, create per-category statistical reference and current data and perform statistical tests to identify transaction categories showing overall statistically relevant performance anomalies. The relevant transaction category detection considers both the relative transaction frequency of categories compared to the overall transaction frequency and the temporal stability of a transaction category over an observation duration. The statistical data generated for the anomaly tests contains next to data describing the overall performance of transactions of a category also data describing the transaction execution context, like the number of concurrently executed transactions or transaction load during an observation period. Anomaly tests consider current and reference execution context data in addition to statistic performance data to determine if detected statistical performance anomalies should be reported.