API Anomaly Detection Using Distribution Modeling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current approaches to detecting anomalies in API calls in payment processing networks often produce a high number of false positives, leading to unnecessary alerts for system administrators, as they fail to differentiate between actual and anticipated changes in transaction volumes.
Innovation Solution
A system that monitors API calls, builds a distribution model based on historical data, and uses probability distributions such as Pareto and Gumbel to identify anomalies, reducing false positives by recommending adjustments to the detection model over time.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Measurement precision
If current anomaly detection approaches are used to monitor API calls, then anomalies can be detected, but a large number of false positives are generated leading to unnecessary alerts
Solution Approach 1:
The patent segments the anomaly detection process into multiple components: (1) collecting API call metrics and establishing baseline distributions, (2) receiving anomaly detections from multiple sources, (3) evaluating each detection against the baseline to determine if it represents a true anomaly or expected variation, and (4) generating alerts only for confirmed anomalies. This segmentation allows the system to filter out false positives while maintaining detection capability.
Solution Approach 2:
The patent performs preliminary action by establishing baseline distributions of API call metrics before anomaly detection begins. These baselines capture expected variations in traffic patterns, enabling the system to pre-evaluate incoming anomaly detections against known normal behavior patterns, thereby reducing false positives before alerts are generated.
2Ease of operation
If threshold-based anomaly detection is applied uniformly, then simplicity is maintained, but context-specific variations (e.g., holiday sales) cause false anomalies to be detected
Solution Approach 1:
The patent applies dynamics by making the anomaly detection thresholds adaptive rather than static. The system continuously monitors API call metrics and dynamically adjusts baseline distributions based on observed patterns, allowing thresholds to evolve with changing traffic conditions. This enables the system to accommodate context-specific variations like holiday sales while maintaining detection sensitivity for true anomalies.
Solution Approach 2:
The patent changes parameters by transitioning from fixed threshold values to dynamic baseline distributions that are continuously updated based on historical data. The system modifies detection parameters adaptively, adjusting what constitutes an anomaly based on learned patterns of normal variation, thereby improving precision without sacrificing operational simplicity.
3Measurement precision
If detailed analysis of each API call variation is performed, then detection accuracy improves, but system complexity and processing overhead increase
Solution Approach 1:
The patent extracts only the essential metrics needed for anomaly detection from the complex stream of API calls. Instead of analyzing every detail of each API call, the system identifies and monitors key performance indicators that are sufficient for detecting anomalies, thereby reducing processing complexity while maintaining detection accuracy.
Solution Approach 2:
The system performs self-service by automatically establishing baselines, evaluating detections, and adjusting thresholds without requiring manual configuration or intervention. This automation reduces the operational complexity of managing a detailed analysis system while maintaining high detection accuracy through continuous adaptive learning.
Data Source
AI summary
A central server receives API calls requesting services. The central server identifies whether the API calls are associated with a merchant. A distribution is constructed based on the API calls. The central server further executes a pre-defined rule to identify a set of the API calls belonging to a maximum percentile in the distribution and a set of the API calls belonging to a minimum percentile in the distribution before estimating a set of the anomalous data points with one or more goodness of fit functions against the maximum percentile and the minimum percentile. A GUI receives a critical value from a user. In response to receiving the critical value, the central server generates probabilities of the set of the anomalous data points before displaying a set of the anomalous data points in response to the probabilities being less than the critical value.