Dynamic Node Subset Assignment for Load Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Increasing the number of operation nodes, such as timelock nodes, leads to increased latency and poor performance in system operations, especially when handling a greater number of transactions, as each node requires consensus, resulting in higher processing time and resource usage.
Innovation Solution
Implementing a system that dynamically assigns subsets of operation nodes to clients based on historical usage data, allowing for load balancing and horizontal scaling, where subsets of operation nodes, including an odd number, communicate to achieve consensus on timestamp and lock operations, ensuring high availability even if some nodes are unavailable.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If the number of operation nodes is increased to handle more transactions, then the system capacity and availability are improved, but the latency and processing time increase due to consensus requirements
Solution Approach 1:
The system segments the set of operation nodes into multiple subsets, where each subset independently performs consensus operations for assigned clients. This segmentation allows parallel processing of consensus operations across different subsets, reducing overall latency while maintaining system availability through distributed redundancy
Solution Approach 2:
The system dynamically assigns clients to different subsets of operation nodes based on load conditions and historical usage patterns. This dynamic assignment optimizes the distribution of consensus operations, allowing the system to handle varying transaction volumes efficiently while maintaining acceptable latency through adaptive load balancing
2Adaptability or versatility
If the number of operation nodes is increased to provide services for more clients, then the system scalability is improved, but the resource usage and processing overhead increase
Solution Approach 1:
By segmenting operation nodes into subsets that can independently serve client groups, the system achieves scalability without requiring all nodes to participate in every consensus operation. This reduces per-operation resource usage while maintaining the capacity to serve growing numbers of clients through parallel subset operations
Solution Approach 2:
Each operation node is designed to be multi-functional, capable of participating in multiple subsets and serving different client groups at different times. This universality allows the system to scale efficiently by flexibly allocating existing nodes to different subsets based on demand, avoiding the need for dedicated nodes for each client group and reducing overall resource requirements
Data Source
AI summary
Systems, methods, and non-transitory computer readable media are provided for load balancing of operations. A set of operation nodes may be run. The set of operation nodes may include operation nodes configured to perform operations. A set of clients that uses one or more of the operations may be identified. Loads of operations associated with the set of clients may be determined. Subsets of operation nodes to be assigned to subsets of clients may be identified based on the loads of operations associated with the set of clients. The subsets of operation nodes may include a given subset of operation nodes to be assigned to a given subset of clients. The subsets of operation nodes may be assigned to the subsets of clients such that the given subset of operation nodes is assigned to perform one or more of the operations for the given subset of clients.