Event-Driven Metrics Service for Real-Time Monitoring
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional metric services in enterprise software applications are unable to provide real-time, explainable metrics due to their on-demand, high-level data processing, which consumes significant computational resources and results in slow performance.
Innovation Solution
An event-driven metrics service is implemented, where rules are defined to identify events triggering metric changes, allowing for real-time computation and persistence of changes as time-series data, reducing the need for large data aggregation and improving performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional on-demand data processing is used to provide metrics, then data aggregation is comprehensive, but computational resource consumption is high and performance is slow
Solution Approach 1:
The system pre-subscribes to event streams and establishes metric computation rules in advance, so that when events occur, the metrics are immediately computed and updated without requiring on-demand data aggregation. This preliminary setup of event subscriptions and computation rules eliminates the need for resource-intensive post-hoc data processing.
Solution Approach 2:
The system segments the monolithic on-demand data processing into distributed event stream subscriptions, where each subscription independently processes specific event types. This segmentation allows parallel processing of multiple event streams and metrics, improving overall computation speed while distributing computational load efficiently.
2Loss of time
If on-demand data processing is used, then data aggregation is thorough, but real-time monitoring capability is lacking
Solution Approach 1:
The system maintains continuous subscription to event streams, ensuring uninterrupted real-time monitoring. Event-driven metric updates occur continuously as events are generated, eliminating the batch processing delays inherent in on-demand approaches. This continuous action enables real-time visibility into system metrics without sacrificing processing efficiency.
3Loss of information
If high-level data processing is implemented, then metric generation is simplified, but explainability and traceability of metrics are reduced
Solution Approach 1:
The system provides feedback by publishing metric update events that include references to the source events and computation rules. This feedback mechanism enables traceability, as consumers can subscribe to metric events and trace back through the event chain to understand how each metric value was derived, maintaining explainability without oversimplifying the processing logic.
Data Source
AI summary
A method may include defining a rule identifying an event that triggers a change in a value of a metric. The defining of the rule includes generating a subscription to receive the event from an event stream. Occurrences of the event identified by the rule may be detected based on receiving the event from the event stream. In response to each occurrence of the event, the value of the metric and/or the change in the value of the metric may be evaluated. A notification may be sent to a software application consuming the metric based on the value of the metric and/or the change in the value of the metric satisfying a threshold. Related systems and computer program products are also provided.