Distributed Database Table Grouping for Workload Balancing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In distributed database systems, executing queries across multiple nodes leads to increased network overhead and workload imbalance, as data is often concentrated on a few nodes, causing some nodes to remain idle while others handle a disproportionate number of queries.
Innovation Solution
An allocation engine identifies disjoint table sets based on frequently used queries and determines key performance indicators (KPIs) to split table sets across multiple nodes, balancing workload and minimizing network overhead by distributing tables across nodes to maintain optimal CPU, query, and memory loads.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If data is stored on a few nodes in a distributed database system, then storage efficiency is improved, but workload balance deteriorates causing some nodes to remain idle while others handle disproportionate queries
Solution Approach 1:
The patent segments tables into different groups (hot, warm, cold tables) and distributes them across different nodes based on access frequency. This segmentation allows high-access tables to be placed on nodes with sufficient capacity while low-access tables occupy remaining nodes, thereby balancing workload across the distributed system while maintaining storage efficiency.
2Adaptability or versatility
If queries are executed across multiple nodes, then query coverage is improved, but network overhead increases due to data transfer between nodes
Solution Approach 1:
The patent applies local quality by placing frequently accessed tables (hot tables) on the same node or adjacent nodes to minimize data transfer distance. This localized placement strategy reduces network overhead for high-frequency queries while still allowing the system to handle diverse query types across multiple nodes, thus balancing query coverage with energy efficiency.
3Productivity
If tables are distributed across multiple nodes, then workload balance is improved, but system complexity increases due to group management
Solution Approach 1:
The patent uses parameter changes by dynamically adjusting table group assignments based on access frequency metrics. Tables are automatically reclassified into hot, warm, or cold groups based on their access patterns, and their physical location is adjusted accordingly. This parameter-driven approach simplifies management compared to static manual assignment while maintaining good workload balance.
Data Source
AI summary
An allocation of tables across a distributed database provides for taking into account existing table groups while also obtaining newly calculated table groups that are formed according to a new table group allocation. Such a new table group allocation may be desired to optimize workload balancing and/or data transfers for new workload patterns. Maintaining one or more types of existing table groups allows for the existing table groups to not be overwritten during allocation of the tables. An identification of tables is used, as well as a selected user-defined grouping option to indicate a mode for maintaining or replacing the existing table groups. An iterative process includes assigning, for each table, a group name and a group type.