Distributed Management Nodes for Message-Oriented Data Feed Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current management systems for message-oriented distributed systems lack scalability and efficient resource utilization, particularly in handling the transformation and alteration of input application data, which is essential for providing derived application data in a timely and accurate manner.
Innovation Solution
A method and system where management nodes receive derived application data and application data processing history information, comparing it with a processing specification to manage and optimize the operation of processing nodes, allowing for efficient scaling with the number of applications and focusing management activities on specific processing paths.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If traditional management systems are used for message-oriented distributed systems, then system operation can be monitored, but scalability and resource utilization efficiency deteriorate
Solution Approach 1:
The management system is segmented into multiple management nodes distributed across the system. Each management node independently manages a subset of processing nodes and applications, allowing the system to scale by adding more management nodes without increasing central化管理 complexity. This divides the monolithic management function into manageable, scalable units.
Solution Approach 2:
The patent introduces a hierarchical dimension to management by organizing management nodes in levels (e.g., regional management nodes, global management nodes). This dimensional organization allows the system to scale vertically and horizontally, managing complexity through structured layers rather than flat expansion.
2Loss of information
If comprehensive monitoring of all processing nodes is implemented, then system visibility is improved, but resource utilization efficiency deteriorates
Solution Approach 1:
Each management node is assigned specific monitoring responsibilities for particular processing nodes or application subsets. This local quality approach ensures comprehensive visibility within each management node's domain while avoiding the overhead of every node monitoring every other node, thus maintaining resource efficiency.
Solution Approach 2:
Management nodes perform monitoring actions selectively based on system state and priority. Rather than continuously monitoring all parameters of all nodes equally, the system applies partial monitoring focused on critical metrics and high-priority components, reducing overall resource consumption while maintaining adequate system visibility.
3Reliability
If management activities cover all processing paths, then complete control is achieved, but processing speed deteriorates
Solution Approach 1:
Processing paths are segmented and assigned to specific management nodes based on their domain responsibility. This segmentation allows parallel management of different path segments, maintaining complete control across the entire system while avoiding sequential bottlenecks that would slow down processing.
Solution Approach 2:
Management nodes act as intermediaries between processing nodes and central coordination. They handle local management tasks and only escalate exceptional cases to higher-level management, reducing the overhead imposed on critical processing paths while maintaining comprehensive control through the intermediary layer.
Data Source
AI summary
A system for managing compositions of software components or applications is disclosed. In particular, systems in which the software components or applications communicate with one another using message-oriented middleware are considered. The system is fed by one or more data feeds, and produces a plurality of processed feeds for respective applications. The data items in the input data feeds are processed by the software components or applications which form a data feed processing graph, with each application having a processing path (or egress-rooted tree) which results in the processed feed for that application. Managing such systems is complex. This complexity is tackled by having each component processing a data feed item add an annotation to the data feed item, so that the cumulative annotation which is seen in the processed feed for an application provides a history of the processing carried out on the input data feed(s). At an egress of the processing system, this history can be compared (186) with an application-specific specification. Arranging the management of a data feed processing system in this way simplifies the management of such systems, and hence improves the efficiency of resource utilisation of such systems.