Hierarchical Node Grouping for Data Processing Scalability

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current membership management systems in data processing systems face scalability issues due to centralized single points of failure and high communication overhead in decentralized systems, as well as slow information dissemination in gossip protocols, which hinder efficient node grouping and communication.

Innovation Solution

A hierarchical membership management system that dynamically adjusts group membership based on real-time cost maps, allowing nodes to transfer between groups when costs exceed thresholds, using leader nodes to query and initiate transfers, thereby optimizing communication and computational costs.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If centralized membership management is used, then communication efficiency is improved, but scalability deteriorates due to single point of failure

Engineering Contradiction:

Improvecommunication efficiencyVSAvoidscalability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The patent divides the centralized membership management system into multiple decentralized leader nodes, each managing a subset of member nodes. This segmentation eliminates the single point of failure while maintaining efficient communication within each group, resolving the contradiction between communication efficiency and scalability.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent introduces a hierarchical dimension to the flat membership structure by organizing nodes into groups with leader nodes. This dimensional change allows the system to scale vertically through multiple hierarchy levels while maintaining efficient local communication, thus improving both reliability and productivity.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Reliability

If decentralized membership management is used, then reliability is improved by eliminating single point of failure, but communication overhead increases exponentially

Engineering Contradiction:

Improvefault toleranceVSAvoidcommunication overhead

Core Design Contradiction:

ReliabilityVSLoss of energy

Solution Approach 1:

The patent segments the decentralized system into multiple leader nodes, each responsible for a specific group of member nodes. This segmentation reduces communication overhead by limiting interactions to relevant groups rather than requiring all-to-all communication, thus maintaining reliability while reducing energy loss.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent implements partial decentralization where only leader nodes maintain full membership information while regular member nodes have limited knowledge of their group. This partial action approach reduces communication overhead compared to full decentralization while still eliminating single point of failure.

Inventive Principle:

Principle #16Partial or excessive action

3Reliability

If gossip protocol is used for membership management, then scalability is improved, but information dissemination speed decreases

Engineering Contradiction:

ImprovescalabilityVSAvoidinformation dissemination speed

Core Design Contradiction:

ReliabilityVSSpeed

Solution Approach 1:

The patent pre-establishes leader nodes that maintain up-to-date membership information before needing to respond to queries. This preliminary action allows the system to scale while providing fast information access, as leader nodes can immediately answer queries without waiting for gossip propagation.

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent introduces leader nodes as intermediaries between member nodes and the membership management system. These intermediaries cache membership information and handle queries locally, reducing the need for widespread gossip communication while maintaining scalability and improving information dissemination speed.

Inventive Principle:

Principle #24Intermediary (Mediator)

4Reliability

If nodes are organized in hierarchical structure, then scalability is improved, but communication complexity increases

Engineering Contradiction:

ImprovescalabilityVSAvoidcommunication complexity

Core Design Contradiction:

ReliabilityVSDevice complexity

Solution Approach 1:

The patent segments the hierarchical structure into independent leader nodes that each manage their own groups. This segmentation simplifies communication complexity by allowing parallel operation of multiple leader nodes, each handling local communication independently, thus maintaining scalability without proportionally increasing overall system complexity.

Inventive Principle:

Principle #1Segmentation

Data Source

PatentUS11539584B2Automatic and dynamic adaptation of grouping in a data processing system

Publication Date: 2022.12.27 TELEFONAKTIEBOLAGET LM ERICSSON (PUBL)

AI summary

A system and method to adapt the grouping between a plurality of nodes in a data processing system. In one embodiment, a first leader node (320) in a data processing system (300) is configured to determine (815) a cost map within a first group of nodes (310), calculate (820) costs between the first leader node (320) and first member nodes (330) based on the cost map, determine (830) a candidate member node (330A) from the first member nodes (330), query (840) other leader nodes (350, 380) to accept a transfer of the candidate member node (330A), receive (850) a first transfer response from a second leader node (350) to accept the transfer to a second group of nodes (340), initiate (860) a transfer of and receive (865) an acknowledgement for the transfer of the candidate member node (330A) to the second group of nodes (340).