Complex Event Processing Cloud Scalability via IPC Segmentation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Complex Event Processing (CEP) engines face bottlenecks in processing capacity and network bandwidth, limiting their scalability and application usage, as they struggle to handle large volumes of event streams from multiple sources, with current solutions only scaling with CPU power and not network infrastructure.
Innovation Solution
A CEP Cloud is created by interconnecting multiple CEP engines using an Inter-Process-Communication (IPC) protocol, allowing for the formation of logical CEP Services that can interact and expand processing capacity without increasing network bandwidth usage, by differentiating between Native and Derived Event Streams for efficient communication and processing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If multiple CEP engines are interconnected to form a CEP Cloud, then processing capacity is vastly extended and scaled, but network bandwidth consumption increases
Solution Approach 1:
The patent segments the event stream processing by differentiating between Native Event Streams (ingested by individual CEP engines) and Derived Event Streams (shared across the CEP Cloud). This segmentation allows each engine to process only its native events locally while sharing only the essential derived events across the network, thereby scaling processing capacity without proportionally increasing network bandwidth consumption.
Solution Approach 2:
The patent introduces Derived Event Streams as an intermediary mechanism between multiple CEP engines. Instead of engines directly communicating and consuming each other's full event streams, they interact through the intermediary derived events that capture only the essential coordination information, reducing network overhead while enabling collaborative processing.
2Productivity
If a hub and spoke model is used with a central manager, then load is shared across multiple CEP engines, but the central manager remains a single point of consumption and correlation
Solution Approach 1:
The patent merges the roles of event consumption and event correlation across all CEP engines in the cloud. Rather than having a single central manager perform correlation, each engine correlates events locally based on its native event stream and relevant derived events. This distributes the correlation workload and eliminates the single point of failure and bottleneck.
Solution Approach 2:
The patent inverts the traditional hub-and-spoke architecture by making each CEP engine an independent node that performs both consumption and correlation functions. Instead of events flowing centrally to a manager for correlation, correlation happens distributedly at each engine, with engines collaborating through shared derived events rather than hierarchical control.
3Productivity
If CEP engines run on larger capacity servers, then processing power increases, but cost and infrastructure complexity increase
Solution Approach 1:
The patent makes each CEP engine multi-functional by enabling it to both consume native events and correlate events independently. This universality allows standard servers to function as complete processing nodes without requiring specialized high-capacity infrastructure, as each engine performs the full CEP workflow locally using distributed collaboration.
Solution Approach 2:
The patent creates copies of the full CEP functionality across multiple standard servers rather than concentrating processing power in a single large server. Each engine is a complete copy capable of independent event consumption and correlation, distributed across the cloud. This approach uses multiple inexpensive standard servers instead of one expensive high-capacity server.
Data Source
AI summary
A distributed system of CEP Services hosted by a CEP Cloud where the CEP Cloud is composed of a network of CEP Engines connected by an IPC Protocol is provided. CEP Services interact with each other via a series of Event Streams transported by an IPC protocol between one or multiple CEP Services. CEP Services have no theoretical limit to the number of CEP Engines that compose said services.