Anomalous Event Detection via Automated Baseline Thresholds
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing systems for monitoring large volumes of event data struggle with real-time anomaly detection, often requiring manual analysis and relying on periodic reports, which can lead to delayed recognition of anomalous activity.
Innovation Solution
A system comprising multiple microservice processors that process data from various sources to generate metrics, which are then used by a monitoring processor to compute threshold baselines and display real-time indications of anomalies in a dashboard, accompanied by automated natural language summaries and notifications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If manual monitoring of periodic reports is used, then employees can identify anomalous behavior, but the approach requires significant time and may fail to recognize anomalies timely
Solution Approach 1:
The patent replaces the manual mechanical monitoring process with an automated electronic system that uses processors to continuously analyze event data, compute baselines, and generate alerts. This substitution eliminates the time delay inherent in manual periodic review while maintaining or improving detection accuracy through consistent automated analysis.
Solution Approach 2:
The system performs self-monitoring by automatically computing baselines from historical data, comparing current events against these baselines, and generating alerts without human intervention. The automated anomaly detection service eliminates the need for employees to manually review reports, thereby reducing time loss while maintaining reliability.
2Ease of operation
If visualization software such as Tableau is used, then data can be visualized, but the system runs into computing resource issues with high volumes of transaction data
Solution Approach 1:
The patent extracts only the essential metrics and anomaly indicators from the large volume of transaction data rather than visualizing all raw data. By computing aggregated metrics and comparing them against baselines, the system provides visualization of only the most relevant information, significantly reducing computing resource requirements while maintaining ease of operation.
Solution Approach 2:
The system segments the large volume of transaction data into manageable time-based event groups and processes them in discrete batches. This segmentation allows the visualization system to handle data in smaller, more efficient units rather than attempting to process and visualize all data simultaneously, reducing memory and processing power requirements.
3Loss of information
If periodic reports are generated and saved in a central location, then event activity can be reviewed, but the approach requires manual monitoring and periodic updates rather than real-time detection
Solution Approach 1:
The patent implements continuous automated monitoring that processes event data in real-time as it occurs, rather than relying on periodic batch processing. The system continuously computes baselines, compares current events against these baselines, and generates alerts immediately when anomalies are detected, eliminating the time delay inherent in periodic report generation and manual review.
Solution Approach 2:
The system pre-computes baseline metrics from historical data and stores them for rapid comparison with current events. This preliminary action allows the system to immediately detect anomalies by comparing new events against pre-established baselines without requiring time-consuming analysis at the moment of detection, thereby improving productivity while maintaining accurate data retention.
Data Source
AI summary
Systems and methods for processing high volumes of event data to produce a plurality of metrics and determining threshold baselines at which to alert to anomalous event activity. Raw event data is processed into staging tables comprises of the plurality of metrics. Baselines are determined based on one or more past instances of the time segment in a given cycle. Monitoring interfaces are also provided to aid rapid identification of anomalous activity.