Streaming Text Data Alert Evaluation Service Architecture
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional systems face challenges in managing and processing large volumes of streaming textual data for real-time alerting, particularly in dynamic environments, due to limited scalability and inefficiencies in evaluating time-sensitive data.
Innovation Solution
The solution involves duplicating streaming textual data to operate independently on ingestion and alert evaluation services, allowing for separate processing that enhances load balancing and scalability, enabling effective detection of alerts through single instance and windowed alert conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional real-time alerting systems process large volumes of streaming textual data, then alert detection capability is maintained, but system scalability deteriorates and processing efficiency decreases
Solution Approach 1:
The system divides the alerting functionality into separate alerting agents that can independently process different portions of streaming data. Each agent is responsible for specific alert rules and can operate autonomously, allowing the system to scale by adding more agents without affecting existing ones. This segmentation resolves the contradiction by enabling both high alert detection capability through distributed processing and system scalability through independent agent deployment.
Solution Approach 2:
The patent introduces a temporal dimension to alert evaluation by implementing time-based alert conditions and windowed aggregations. This allows the system to evaluate alerts not just on individual events but across time windows, enabling more sophisticated alert detection while maintaining scalability through efficient time-series data processing. The temporal dimension adds analytical depth without requiring proportional increases in system complexity.
2Loss of time
If conventional systems evaluate large quantities of streaming textual data in real-time, then near real-time alerting is achieved, but system complexity increases and performance deteriorates
Solution Approach 1:
The system performs preliminary actions by pre-compiling alert rules into optimized evaluation expressions and pre-configuring alerting agents with their specific rule sets. This preparation work is done beforehand, allowing the runtime system to simply evaluate pre-processed data against pre-compiled rules, significantly reducing real-time processing complexity while maintaining fast alert detection speeds.
Solution Approach 2:
The patent creates copies of alerting agents and rule evaluation logic that can be distributed across multiple processing units. Each copy independently evaluates data streams, eliminating the need for a single complex centralized processing system. This copying approach reduces the complexity of any single component while maintaining overall system capability through parallel evaluation.
Data Source
AI summary
In a computer-implemented method for performing alert notification on streaming textual data, streaming textual data is received. A plurality of copies of the streaming textual data is generated, wherein a first copy of the streaming textual data is transmitted to an ingestion service for persistent storage at a data plane and a second copy of the streaming textual data is transmitted to an alert evaluation service for performing alert evaluation on the streaming textual data, such that the ingestion service and the alert evaluation service are performed independently. The streaming textual data is evaluated by the alert evaluation service to detect an alert. Responsive to detecting an alert at the alert evaluation service, a notification of the alert is generated.