Append-Only Event Tables for Efficient Data Evaluation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current methods for processing sequences of events at different points in time are inefficient, as complex SQL queries consume significant resources and are burdensome to develop and maintain, lacking flexibility in monitoring and querying event-generated data.
Innovation Solution
The approach involves processing and storing events as an append-only sequence, allowing for efficient querying and evaluation of events at any time by using a method that selects and processes events based on event definitions and filters, enabling parallel processing of sub-sequences and handling logical updates without modifying existing records.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If complex SQL queries are used to process time-correlated events stored in different tables, then the ability to evaluate time-correlated events is achieved, but computing resource consumption increases significantly
Solution Approach 1:
The patent combines multiple event tables from different databases into a single materialized view that pre-joins and stores time-correlated event data. This merging eliminates the need for complex SQL queries to join multiple tables at query time, significantly reducing computing resource consumption while maintaining the ability to evaluate time-correlated events.
Solution Approach 2:
The patent performs preliminary processing by pre-computing and materializing event correlations in advance. The materialized view stores pre-joined event data with temporal relationships established beforehand, so that query execution only requires simple lookups rather than complex real-time joins, reducing runtime computing resources.
2Adaptability or versatility
If complex SQL queries are used to process time-correlated events, then event evaluation capability is provided, but query complexity increases making it beyond user ability
Solution Approach 1:
By merging multiple event tables into a single materialized view that encapsulates complex temporal relationships, the patent simplifies user queries from complex multi-table joins to simple single-table queries. Users can evaluate time-correlated events without needing to understand or write complex join logic.
Solution Approach 2:
The materialized view acts as an intermediary layer between the underlying complex event tables and user queries. It pre-processes and structures time-correlated event data in a user-friendly format, mediating between the complexity of data storage and the simplicity of user interaction.
3Adaptability or versatility
If custom code is included to store and query event-generated data, then flexibility in monitoring events is achieved, but development time and testing burden increase
Solution Approach 1:
The patent implements self-service functionality where the system automatically maintains materialized views through triggers and stored procedures that respond to underlying data changes. This automation eliminates the need for developers to write custom code for data synchronization and query processing, reducing development time and testing burden while maintaining monitoring flexibility.
Solution Approach 2:
The materialized view serves as an intermediary that automatically handles the complexity of event data integration. Instead of requiring custom code to manage data from multiple sources, the materialized view automatically maintains itself through database triggers, eliminating manual development and testing efforts.
4Ease of operation
If custom code is included to query event-generated data, then event monitoring capability is provided, but system flexibility decreases when adding new events or queries
Solution Approach 1:
The materialized view is designed with a universal structure that can accommodate multiple event types and time-correlation patterns through standardized columns and relationships. This universal design allows the same materialized view to support various event monitoring scenarios without requiring custom code modifications, maintaining flexibility when adding new events or queries.
Data Source
AI summary
Techniques are provided for evaluating base events, compound events, and logical update events in an event processing system. In one embodiment, the base events, compound events, and logical update events are definitions in a particular syntax language that specify conditions for evaluating input event data. In this embodiment, base event definitions are evaluated over the input event data; logical update event definitions are evaluated by determining whether the effect of any earlier event is changed by the arrival of a new event; and compound event definitions are evaluated over records representing events that have been selected by evaluating the base event definitions over the input event data.