Compressed-Data Load Balancing for Multi-Node I/O Hotspots
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Traditional load balancing mechanisms are ineffective in multi-node systems with uneven distribution of protocol layer compressed data, leading to resource imbalances and hotspots, which affect system performance and user experience.
Innovation Solution
A load balancing solution that considers the characteristics of protocol layer compressed data, such as proportion and compression ratio, to redistribute workloads between nodes, combining with bandwidth-based balancing to alleviate CPU and bandwidth imbalances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Ease of operation
If traditional load balancing mechanisms are used, then general workload distribution is achieved, but resource imbalances and hotspots occur due to uneven distribution of protocol layer compressed data
Solution Approach 1:
The patent applies local quality by differentiating load balancing strategies based on the specific characteristics of compressed data. Instead of uniform load balancing, the system identifies nodes with high proportions of compressed data and applies targeted load balancing actions to those specific nodes, making the load balancing approach adapt to local conditions of each node.
Solution Approach 2:
The patent changes the parameters used for load balancing decisions by incorporating the proportion of compressed data in I/O data as a key parameter. This goes beyond traditional CPU usage metrics and includes compression ratio and other data characteristics to dynamically adjust load balancing actions.
2Loss of energy
If protocol layer compression is applied, then bandwidth consumption is reduced and data transmission time is saved, but CPU overhead increases due to compression and decompression operations
Solution Approach 1:
The system dynamically adjusts load balancing decisions based on compression ratio parameters. By monitoring the proportion of compressed data and compression ratios, the system can identify when CPU overhead from compression operations becomes excessive and redistribute workloads accordingly.
Solution Approach 2:
The patent implements feedback mechanisms by continuously monitoring CPU usage rates, proportions of compressed data, and compression ratios. This feedback information is used to dynamically adjust load balancing actions, ensuring that the system responds to changing conditions and optimizes the trade-off between bandwidth efficiency and CPU overhead.
3Device complexity
If workloads are redistributed based on CPU usage rate alone, then simple load balancing is achieved, but compressed data workloads are not effectively balanced leading to persistent hotspots
Solution Approach 1:
The patent applies local quality by differentiating load balancing strategies based on the specific characteristics of compressed data. Instead of uniform load balancing, the system identifies nodes with high proportions of compressed data and applies targeted load balancing actions to those specific nodes, making the load balancing approach adapt to local conditions of each node.
Solution Approach 2:
The patent changes the parameters used for load balancing decisions by incorporating the proportion of compressed data in I/O data as a key parameter. This goes beyond traditional CPU usage metrics and includes compression ratio and other data characteristics to dynamically adjust load balancing actions.
Data Source
AI summary
The subject technology relates to load balancing. An example method includes determining, in response to a difference between resource consumption of a first node in a group of nodes and resource consumption of another node in the group of nodes exceeding a first threshold, whether a proportion difference between a proportion of compressed data in I/O data of the first node and a proportion of compressed data in I/O data of another node in the group of nodes exceeds a second threshold. When the proportion difference exceeds the second threshold, a workload is selected from a first group of workloads on the first node based on characteristics of compressed data in respective I/O data of the first group of workloads, and the workload is moved to a second node in the group of nodes, system load imbalance caused by protocol layer data compression is alleviated in a multi-node system.