Transaction Anomaly Detection via Statistical Baseline Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing monitoring systems for high-transaction-volume applications, such as e-commerce platforms, face challenges in providing real-time, granular transaction monitoring and alerting that is both accurate and free from false positives, especially in high-load scenarios, where conventional threshold-based alerting systems generate numerous false alerts and fail to identify the root cause of performance issues.
Innovation Solution
A monitoring and alerting system that analyzes individual transactions to detect changes in overall application performance by extracting classification parameters, calculating statistical parameters from historical and current data, and using quantile estimations to identify deviations, thereby reducing false positives and pinpointing performance anomalies.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional threshold-based alerting systems are used to monitor individual transactions in high-volume applications, then real-time monitoring capability is achieved, but the number of false-positive alerts increases significantly
Solution Approach 1:
The patent segments transactions into distinct classes based on multiple parameters (URL, HTTP method, headers, cookies, form parameters, session ID) rather than monitoring individual transactions in isolation. This segmentation allows the system to establish baseline performance metrics for each transaction class, reducing false positives caused by natural performance variations in high-volume scenarios.
Solution Approach 2:
The patent extracts and analyzes only the most relevant classification parameters from transaction data rather than processing all transaction details. By selectively focusing on key parameters that define transaction classes (URL, HTTP method, critical headers), the system achieves effective monitoring while filtering out noise that would generate false alerts.
2Difficulty of detecting and measuring
If detailed transaction tracing at granular level is implemented to identify root causes, then performance problem identification capability is improved, but system overhead increases making it unsuitable for high-volume applications
Solution Approach 1:
The patent extracts only the essential classification parameters and performance metrics from detailed transaction traces, separating the information needed for anomaly detection from the complete transaction execution details. This extraction approach enables root cause identification through classification-based analysis without requiring full granular tracing of every transaction, thereby reducing system overhead.
Solution Approach 2:
The system performs preliminary classification of transactions into distinct classes before detailed performance analysis. By pre-grouping transactions based on their classification parameters (URL, HTTP method, headers, etc.), the system prepares data in advance for efficient baseline comparison and anomaly detection, avoiding the need for complex real-time analysis of every individual transaction detail.
3Measurement precision
If baseline-oriented alerting systems use historical performance measurements to establish expected values, then alerting accuracy is improved, but the systems still generate false positives based on single measurements
Solution Approach 1:
The patent merges multiple classification parameters (URL, HTTP method, headers, cookies, form parameters, session ID) to define comprehensive transaction classes. By combining these parameters, the system creates more specific and meaningful groups for baseline comparison, ensuring that alerts are generated based on deviations from class-specific baselines rather than generic thresholds, thereby reducing false positives from natural performance variations.
Solution Approach 2:
The system continuously updates baseline performance metrics for each transaction class based on historical measurements and uses this feedback to improve anomaly detection. By maintaining and updating class-specific baselines over time, the system adapts to normal performance variations and becomes more accurate in distinguishing actual anomalies from expected fluctuations, reducing false positives.
Data Source
AI summary
A combined transaction execution monitoring, transaction classification and transaction execution performance anomaly detection system is disclosed. The system receives and analyzes transaction tracing data which may be provided by monitoring agents deployed to transaction executing entities like processes. In a first classification stage, parameters are extracted from received transaction tracing data, and the transaction tracing data is tagged with the extracted classification data. A subsequent measure extraction stage analyzes the classified transaction tracing data and creates corresponding measurements which are tagged with the transaction classifier. A following statistical analysis process maintains statistical data describing the long term statistical behavior of classified measures as a baseline, and also calculates corresponding statistical data describing the current statistical behavior of the classified measures. The statistical analysis process detects and notifies significant deviations between the statistical distribution of baseline and current measure data. A subsequent anomaly alerting and visualization stage processes those notifications.