Dynamic Partition Balancing for On-Demand Services
Solution Overview
Problem
In on-demand services environments, workload balancing across partitions is inefficient due to static orgID to partition mapping, leading to uneven resource utilization and manual, labor-intensive monitoring, which results in suboptimal load distribution and high costs.
Innovation Solution
A partition balancing tool that measures database usage at the organization and partition level, using a framework like Hadoop to process logs and merge data with database statistics, assigning 'hotness' values to partitions and dynamically mapping them to nodes based on processor workload and other factors for improved load balancing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Device complexity
If static orgID to partition mapping is used, then system simplicity is maintained, but workload distribution becomes uneven and system efficiency deteriorates
Solution Approach 1:
The patent implements dynamic partition-to-node mapping that automatically adjusts based on real-time processor workload metrics. The system continuously monitors CPU utilization across nodes and reassigns partitions from overloaded nodes to underutilized nodes, transforming the static mapping into a dynamic system that adapts to changing workload conditions and maintains optimal system efficiency.
Solution Approach 2:
The system incorporates feedback mechanisms by continuously monitoring processor workload metrics and using this information to drive partition reassignment decisions. The feedback loop collects performance data, analyzes workload distribution, and triggers automated rebalancing actions to optimize system efficiency while managing complexity through intelligent control.
2Measurement precision
If manual monitoring of partition workload is performed, then resource utilization can be tracked, but labor intensity increases and operational efficiency decreases
Solution Approach 1:
The system implements self-service automation where the partition balancing mechanism autonomously monitors its own performance metrics, analyzes workload distribution, and executes rebalancing operations without human intervention. The system services itself by automatically detecting imbalances and correcting them, eliminating manual monitoring efforts while maintaining precise resource utilization visibility.
Solution Approach 2:
The patent replaces manual mechanical monitoring operations with automated electronic systems that continuously collect and analyze performance data. The system uses software-based workload measurement and automated decision-making algorithms to substitute human-operated monitoring processes, reducing operational effort while improving measurement precision and response time.
3Quantity of substance
If partitions are added statically to satisfy growth, then system capacity increases, but load distribution becomes suboptimal and costs increase
Solution Approach 1:
The system dynamically adjusts partition-to-node mappings based on real-time workload conditions, allowing the system to efficiently accommodate growth without requiring proportional increases in resources. By continuously optimizing load distribution across existing partitions and nodes, the system maintains cost efficiency while supporting capacity expansion through intelligent resource utilization rather than static over-provisioning.
Data Source
AI summary
Techniques for partition balancing. Organization identifiers (orgIDs) are acquired for one or more organizations within a multi-organization on-demand services environment having multiple nodes, each of which have one or more partitions. A mapping of the orgIDs to the partitions within the multi-organization on-demand services environment is determined. Resource consumption for the organizations is determined. Resource consumption for the partitions based on the resource consumption of the organization and the mapping of the organization to the partitions is determined. A score for the partitions based on the resource consumption is generated. The partitions are redistributed among the nodes based on the partition scores.