Complex Event Processing State Recovery via Query-Specific Time Windows
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing complex event processing (CEP) systems face inefficiencies in state recovery due to unnecessary storage and processing overhead, as checkpoints are generated periodically without consideration for the properties of the event query, leading to excessive memory usage and processor utilization during recovery.
Innovation Solution
A window manager is implemented to determine a window size based on event query properties, selectively retrieving and replaying relevant events from a distributed log to restore the operational state of the CEP engine, reducing memory usage and processor load by focusing on the most up-to-date historical events.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If checkpoints are generated periodically without considering event query properties, then the operational state values can be recovered after failure, but excessive memory usage and processor utilization occur during recovery
Solution Approach 1:
The patent segments the event history into time windows based on event query properties (e.g., time range, event types). Instead of recovering all historical events, only events within the relevant time window are retrieved and replayed. This segmentation principle divides the large set of all events into manageable, relevant subsets, reducing memory usage during recovery while maintaining reliability.
Solution Approach 2:
The patent applies local quality by making the recovery process query-specific. Each event query has its own time window and event type filters, so the recovery process retrieves only the locally relevant events needed for that specific query. This avoids the waste of retrieving and processing all historical events uniformly, reducing overall memory usage while preserving recovery capability for each query context.
2Reliability
If all historical events are replayed during recovery, then complete operational state values are restored, but processor utilization increases excessively
Solution Approach 1:
The patent segments the event replay process by time windows and event query properties. Instead of replaying all historical events sequentially (which would exhaust processor resources), the system divides events into relevant time-based segments and replays only those segments. This reduces processor utilization during recovery while maintaining state recovery completeness for the specific query context.
Solution Approach 2:
The patent applies partial action by retrieving and replaying only the necessary portion of historical events (those within the time window and matching event types) rather than all events. This partial replay is sufficient to restore operational state values for the specific event query, avoiding the excessive processor utilization that would result from replaying the complete event history.
3Quantity of substance
If event query properties are considered for checkpoint generation, then memory usage during recovery decreases, but system complexity increases
Solution Approach 1:
The patent implements a window manager that serves multiple functions: it determines time windows based on event query properties, filters events accordingly, and controls the recovery process. This universal component handles various event query types (different time ranges, event types, aggregation functions) using a unified approach, reducing the need for separate recovery mechanisms for each query type and thereby limiting the increase in system complexity.
Data Source
AI summary
Methods by a processing system are disclosed that control recovery of operational state values of a complex event processing (CEP) engine that processes values of events. A window size is determined based on a property of an event query. Events' values are retrieved from a distributed log which are restricted to occurring within a timeframe defined based on the window size. The distributed log stores events' values that have been processed by the CEP engine. The retrieved events' values are replayed to the CEP engine for processing to recover the operational state values of the CEP engine. Related processing systems are disclosed that control recovery of operational state values of a CEP engine that processes values of events.