Distributed Network Aggregate Statistics Computation
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed network systems face challenges in efficiently computing aggregate statistics across a large number of nodes while maintaining scalability, robustness, and minimizing communication overhead, as conventional centralized methods are prone to bottlenecks and failures, and decentralized gossip-based methods have high communication overhead.
Innovation Solution
A method is introduced where nodes in a distributed network determine whether to become leader or non-leader nodes, forming groups, and through a series of 'push', 'pull', 'gossip', and 'sample' steps, they efficiently compute aggregate statistics like MAX, AVERAGE, SUM, and RANK, optimizing node assignments and communication to reduce overhead and enhance robustness.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If a centralized approach is used for aggregate statistic computation, then communication overhead is reduced, but scalability and reliability deteriorate due to bottlenecks and single point of failure
Solution Approach 1:
The patent divides the distributed network into multiple groups, each with a leader node responsible for aggregate computation within that group. This segmentation eliminates the single point of failure problem by distributing coordination responsibilities across multiple leaders, while still maintaining efficient communication within each segmented group.
Solution Approach 2:
Leader nodes act as intermediaries between regular nodes and the rest of the system. Each leader collects data from its group members and performs local aggregate computations, reducing the communication burden on individual nodes while distributing the computational load across multiple intermediaries rather than a single centralized coordinator.
2Reliability
If a decentralized gossip-based approach is used for aggregate statistic computation, then scalability and robustness are improved, but communication overhead increases significantly
Solution Approach 1:
By segmenting the network into groups with designated leaders, the patent reduces the communication overhead compared to fully decentralized gossip-based approaches. Each node only needs to communicate with its group leader rather than potentially all other nodes, while the system maintains robustness through the distributed group structure and multiple leaders.
3Quantity of substance
If more nodes are added to the system, then system capacity and data coverage are improved, but computation time and complexity increase prohibitively
Solution Approach 1:
The patent segments the network into multiple groups, each handling its own aggregate computations independently. This allows the system to scale to a large total number of nodes while keeping computation time within each group manageable and relatively constant, rather than having computation time grow linearly with the total number of nodes as in centralized approaches.
Solution Approach 2:
Each leader node computes aggregates for only its own group rather than for the entire network, performing a partial computation that is sufficient for local decision-making. This partial action approach allows the system to handle large numbers of nodes efficiently by distributing the computational workload across multiple leaders rather than requiring one leader to process all nodes.
Data Source
AI summary
A method of grouping nodes within a distributed network is provided. The example method includes performing a leader node self determination operation by which each node within the distributed network determines whether to become a leader node or a non-leader node, each leader node being the leader of a group including at least one node. Next, requests are sent, from each leader node, requesting at least one non-leader node to join the group associated with the leader node. First received requests are accepted, at each non-leader node, such that accepting non-leader nodes transition from a non-leader node to a dependent node dependent upon the requesting leader node. A next set of requests are sent, from each remaining non-leader node, requesting to join the group associated with at least one leader node. A determination is made, at each requested leader node, as to whether to accept the non-leader node into the group associated with the requested leader node. Based on the determination, at each requested leader node, the non-leader node is either accepted into the group associated with the requested leader node, or is alternatively rejected from the group.