Object-Level Load Balancing for Asymmetric Database Instances
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Load balancing between asymmetric database instances when sharing intermediate results leads to uneven workload distribution, impacting overall query execution time, and existing methods require significant synchronization efforts and resource contention.
Innovation Solution
Implement object-level load balancing strategies that minimize synchronization overhead by using a global semaphore for round-robin assignment of intermediate results or maintaining metadata for dynamic allocation, ensuring optimal load distribution across database instances.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional load balancing methods are used to share intermediate results between asymmetric database instances, then workload distribution is achieved, but synchronization overhead and resource contention increase significantly
Solution Approach 1:
The system performs preliminary actions by pre-calculating and storing metadata about intermediate result objects (including object identifiers, sizes, and source database instance information) before actual workload distribution occurs. This preliminary metadata preparation enables subsequent load balancing operations to proceed with minimal synchronization overhead, as the distribution logic can operate independently using pre-stored information.
Solution Approach 2:
The patent introduces an intermediary metadata layer that mediates between database instances during intermediate result sharing. Instead of direct complex synchronization between asymmetric instances, the system uses metadata objects stored in a shared filesystem as an intermediary, which contain all necessary information for load balancing decisions. This intermediary approach eliminates the need for continuous inter-instance synchronization protocols.
2Speed
If intermediate results are shared between asymmetric database instances with different node sizes and PU sizes, then query execution speed improves, but uneven workload distribution occurs
Solution Approach 1:
The system applies local quality by tailoring the workload distribution strategy to the specific characteristics of each target database instance. The load balancing logic examines metadata about intermediate result objects and assigns them to target instances based on each instance's local capabilities (node size, PU size, current workload). This ensures that larger instances handle larger objects while smaller instances handle smaller objects, achieving uniform workload distribution across asymmetric hardware configurations.
Solution Approach 2:
The patent changes the parameter of workload assignment from fixed or uniform distribution to dynamic distribution based on instance parameters. The system adjusts workload allocation by considering parameters such as target instance PU size, object size, and current load conditions. This parameter-driven approach enables the system to optimize query execution speed while maintaining balanced workload distribution across asymmetric database instances.
3Productivity
If object-level load balancing is implemented to reduce synchronization overhead, then system throughput improves, but complexity of load balancing logic increases
Solution Approach 1:
The patent segments the load balancing problem into two independent parts: metadata management and object distribution. The metadata layer is segmented into discrete objects with attributes that can be independently evaluated. The distribution logic is segmented into a systematic algorithm that processes metadata and assigns objects to targets based on predefined criteria. This segmentation reduces overall system complexity while enabling efficient object-level load balancing that improves throughput.
Data Source
AI summary
A system and method for balancing database workload in a cloud database system employing multiple database instances, wherein intermediate results are shared between database instances via object store or cloud storage system accessible by the multiple database instances. During sharing of intermediate results between database instances having different node sizes or processing unit sizes, load balancing is achieved by evaluating the target and source database instance sizes to select an appropriate load balancing strategy. Data and metadata contained within the intermediate results shared between source and target database instances are organized to provide optimal load balancing on the target database instance.