Logical Operator Pinning for Message Bus Flow Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed processing systems face challenges with scalability, network congestion, and computational resource bottlenecks due to increased event production and complex queries, leading to inefficient message flow and redundant computations in complex monitoring systems.
Innovation Solution
A system and method that pin logical operators to specific physical processing nodes on a message bus, using a model-driven 'data discrimination' approach to filter out unnecessary events, reducing message flow and balancing processing loads by assigning each logical operator to an initial physical processing node, and reassigning them based on message flow rates to minimize bottlenecks.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If the number of events and queries over events is increased to provide comprehensive monitoring, then monitoring capability and analytical processing are improved, but network congestion and computational resource bottlenecks worsen
Solution Approach 1:
The patent segments the event processing workload by distributing logical operators across multiple physical processing nodes in a clustered architecture. Each node handles a subset of event processing tasks, dividing the overall monitoring system into independent, manageable segments that can operate in parallel, thereby improving monitoring capability while preventing network congestion through distributed processing
Solution Approach 2:
The patent introduces a new dimension of event processing by implementing push-down subscriptions that allow query conditions to be evaluated at the event source level rather than requiring all events to traverse the entire network to central processing nodes. This dimensional shift in processing location reduces network traffic while maintaining comprehensive monitoring capability
2Productivity
If events are stored and retrieved for complex analytical processing, then analytical capability is improved, but event-management resources and database performance worsen
Solution Approach 1:
The patent applies preliminary action by pre-filtering and pre-processing events at the source using push-down subscriptions before events are stored or transmitted. Query conditions and filtering logic are evaluated in advance at the event generation point, reducing the volume of events that need to be stored and retrieved for analytical processing, thereby improving analytical capability while conserving event-management resources
Solution Approach 2:
The patent introduces intermediary processing layers (logical operators and push-down subscription mechanisms) between event sources and the event management database. These intermediaries perform preliminary filtering and transformation, reducing the burden on the database while enabling complex analytical processing through the intermediary layer
3Reliability
If redundant filtering and computation are performed at the application level, then processing thoroughness is improved, but computational efficiency and message flow performance worsen
Solution Approach 1:
The patent implements self-service by enabling event sources and intermediate processing nodes to autonomously perform filtering and computation using push-down subscriptions. Each node independently evaluates its own subscription conditions and filters events locally without requiring centralized coordination or redundant processing, maintaining processing thoroughness while dramatically improving computational efficiency through decentralized autonomous operation
Solution Approach 2:
The patent changes the operational parameters of event processing by transforming the traditional pull-based model into a push-based model with push-down subscriptions. This parameter change enables filtering conditions to be evaluated at different stages of the event lifecycle, allowing thorough processing to occur at the most efficient point in the data flow rather than requiring redundant filtering at multiple levels
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system, method, and computer readable medium for reducing message flow on a message bus are disclosed. The method includes determining if at least one logical operator in a plurality of logical operators requires processing on a given physical processing node in a group of physical nodes. In response to determining that the logical operator requires processing on the given physical processing node, the logical operator is pinned to the given physical processing node. Each logical operator in the plurality of logical operators is assigned to an initial physical processing node in the group of physical processing nodes on a message bus.