Multi-pattern evaluation plan for complex event processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current complex event processing systems face challenges in efficiently detecting arbitrarily complex patterns in massive data streams due to high computation overhead and limited scalability, despite optimization techniques like pattern rewriting and sharing common subexpressions.
Innovation Solution
A system and method that calculate an optimal multi-pattern evaluation plan by generating an initial plan, applying a search method to modify it, assigning scores based on a cost function, and selecting the highest scored version, incorporating reordering and sharing of common sub-patterns to maximize efficiency.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If pattern rewriting and sharing common subexpressions techniques are used to optimize CEP systems, then pattern detection performance is improved, but computation overhead increases and scalability is limited
Solution Approach 1:
The patent segments the complex pattern matching process into multiple evaluation stages (e.g., filtering, grouping, aggregation). By dividing the workload into discrete stages that can be independently optimized and executed, the system reduces overall computation overhead while maintaining detection performance. Each stage processes only the necessary portion of data, avoiding redundant computations across the entire pattern set.
Solution Approach 2:
The system performs preliminary actions by pre-computing and caching intermediate results that are commonly used across multiple patterns. Evaluation plans are generated in advance that identify and prepare shared subexpressions before actual pattern matching begins. This preliminary preparation reduces the computation overhead during real-time pattern detection by avoiding redundant calculations.
2Adaptability or versatility
If multiple CEP patterns are processed in parallel under tight real-time constraints, then detection capability is enhanced, but scalability is limited by high computation overhead
Solution Approach 1:
The patent creates a universal evaluation plan framework that can handle multiple different CEP patterns through a common set of evaluation stages and operators. The system uses pattern matching rules that are applicable across diverse pattern types, allowing the same infrastructure to scale to handle increasingly complex and numerous patterns without proportionally increasing computation overhead. This multi-functional approach enables scalability by reusing evaluation logic across different pattern instances.
3Productivity
If existing optimization methods are applied to CEP systems, then processing efficiency is improved, but the quality of produced plans is compromised by ignoring significant parts of the solution space
Solution Approach 1:
The system employs dynamic plan generation that adapts evaluation strategies based on the specific characteristics of each pattern and the current data state. Rather than using static optimization rules, the system dynamically selects and adjusts evaluation operators, stage configurations, and data flow paths to match the specific requirements of each CEP pattern. This dynamic approach explores a broader solution space and produces higher quality plans that are optimized for their specific context rather than relying on general-purpose optimizations that may miss pattern-specific opportunities.
Data Source
AI summary
A system comprising: at least one hardware processor; and a non-transitory computer-readable storage medium having stored thereon program instructions, the program instructions executable by the at least one hardware processor to: receive a data stream representing events; receive a plurality of complex event patterns (CEPs) comprising (a) a set of conditions reflecting relations among said events, and (b) a set of attributes associated with each of said events; and calculate an optimal multi-pattern evaluation plan corresponding to said CEPs by: (i) generating an initial evaluation plan, (ii) applying a search method to calculate modified versions of said initial evaluation plan, (iii) assigning a score to each of said modified versions based on a cost function, and (iv) selecting one of said modified versions having a highest said score as said optimal multi-pattern evaluation plan.