Distributed Complex Event Processing Engine
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data processing tools are inadequate for efficiently analyzing vast amounts of event data from multiple sources in real-time, known as Big Data, which hinders the ability to identify complex events quickly and accurately.
Innovation Solution
A distributed complex event processing system that maps event data to instances of a complex event processing engine, allowing these instances to analyze event streams independently and identify complex events based on predefined query specifications, thereby processing data in parallel and reducing the load on each engine.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If conventional data processing tools are used to analyze vast amounts of event data, then processing capability is limited, but processing throughput and speed deteriorate
Solution Approach 1:
The patent divides the large-scale event data processing task into multiple smaller sub-tasks by partitioning the event cloud into distributed event streams. Each processing node handles a specific partition independently, allowing parallel processing across multiple nodes. This segmentation enables the system to process vast amounts of data without overwhelming individual processing engines, thereby improving throughput while managing computational resource constraints.
2Measurement precision
If event data from multiple sources are analyzed to identify complex events, then detection accuracy improves, but processing time increases
Solution Approach 1:
The patent performs preliminary filtering and partitioning of event data before complex event processing. Event data are pre-processed to identify relevant streams and partitions, and complex event query specifications are pre-compiled. This preliminary action reduces the volume of data requiring complex analysis, allowing the system to maintain high detection accuracy across multiple data sources while minimizing processing delay through efficient parallel execution.
3Productivity
If distributed event processing is implemented to handle Big Data, then processing capacity increases, but system complexity increases
Solution Approach 1:
The patent implements a universal complex event processing architecture where multiple processing nodes execute identical complex event query specifications across different event stream partitions. Each node performs the same processing functions locally, eliminating the need for complex centralized coordination logic. This multi-functional approach allows the system to scale processing capacity by simply adding more nodes running the same software, thereby increasing capacity while keeping system architecture complexity manageable through standardization.
Data Source
AI summary
A complex event query specification may be constructed that a complex event processing engine uses to analyze an event stream for an occurrence of a complex event. Event data stored in a distributed file system are mapped to respective instances of the complex event processing engine in respective distributed event streams. The distributed event streams are analyzed by the independently executing complex event processing engines in accordance with the complex event query specification. The occurrence of the complex event in any of the distributed event streams is indicated at the output of the complex event processing engines.