Parallel CEP Query Execution via Operator Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional database models are not well-suited for processing continuous event streams due to their single-threaded nature, leading to scalability issues and unnecessary data loss when errors occur, as they cannot efficiently utilize concurrent processing power and often require manual intervention to recover from faults.
Innovation Solution
A CEP query processor that splits queries into separate operators, determines concurrency constraints, and spawns multiple threads to execute parts of the query concurrently, while introducing fault handling mechanisms to allow customized responses to errors, enabling continuous query execution without halting the entire process.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If CEP queries are executed in a single-threaded manner, then state maintenance is simplified and query correctness is ensured, but processing throughput is limited and concurrent processing power is wasted
Solution Approach 1:
The patent segments a CEP query into multiple independent operators that can be executed in parallel. Each operator processes events independently, allowing concurrent execution across multiple threads while maintaining the overall query functionality through coordinated operator execution.
Solution Approach 2:
The patent transitions from single-threaded sequential execution to multi-threaded parallel execution by adding the dimension of concurrent processing. This enables multiple operators to execute simultaneously across different threads,充分利用 available processing resources.
2Reliability
If CEP queries automatically restart after errors, then data loss is minimized and continuous processing is maintained, but incorrect events may cause repeated failures and waste processing resources
Solution Approach 1:
The patent implements a feedback mechanism where the system monitors query execution and automatically detects errors. Based on this feedback, the system intelligently determines whether to restart the query, preventing infinite loops from incorrect events while recovering from transient failures.
Solution Approach 2:
The patent enables the CEP query system to automatically handle its own errors and restarts without external intervention. The system self-diagnoses execution failures and autonomously decides on recovery actions, eliminating the need for manual administrator intervention.
3Loss of energy
If manual intervention is required to recover from query errors, then processing resources are not wasted on repeated failures, but data loss occurs and processing time is lost
Solution Approach 1:
The patent implements preliminary error handling mechanisms that detect and manage execution errors before they cause significant data loss. The system is pre-configured with error detection and automatic restart capabilities, enabling rapid response to failures without waiting for manual intervention.
Solution Approach 2:
The system performs self-diagnosis and self-recovery from execution errors without requiring external administrator intervention. This autonomous error handling eliminates both the data loss and time delay associated with manual recovery processes.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A continuous event processing (CEP) query processor determines how and when a CEP query can be processed in a concurrent manner, such that multiple threads of execution can concurrently perform at least some of the CEP query's operations concurrently with each other. According to one technique, a user can instruct a CEP query processor to attempt to execute a CEP query in a concurrent manner. The CEP query processor responsively determines whether the CEP query's execution, or parts thereof, can be made concurrent based on certain constraints that can depend on inheritance and operation type. Based on this determination, the CEP query processor can execute at least certain parts of a CEP query in parallel relative to the same event within an event stream.