Dynamic Database Record Redistribution Based on Workload Metrics
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Distributed databases face inefficiencies due to changes in data and workload over time, as existing distribution schemes may become inefficient with new data sets or applications, leading to suboptimal performance.
Innovation Solution
A database system that dynamically re-distributes records among servers based on workload data, identifying performance impacts on columns and re-distributing only if the impact exceeds a threshold, using a workload log to derive performance impact from resource cost and frequency data, and maintaining read-only copies during redistribution.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is distributed among nodes using a static distribution scheme, then initial performance requirements are met, but performance degrades when data or workload changes over time
Solution Approach 1:
The patent implements dynamic data distribution by continuously monitoring workload characteristics and automatically re-distributing data partitions based on current access patterns. The system transitions from static distribution to dynamic adaptation, where the distribution scheme evolves with changing workload requirements without manual intervention.
Solution Approach 2:
The system employs feedback mechanisms by monitoring performance metrics and workload patterns, then using this information to automatically adjust data distribution. The feedback loop enables the system to detect when performance degradation occurs and trigger re-distribution operations to optimize performance based on actual usage patterns.
2Productivity
If data is re-distributed frequently to optimize performance, then system performance improves, but resource overhead and system load increase
Solution Approach 1:
The system changes parameters such as distribution thresholds and monitoring intervals based on system conditions. By adjusting these parameters dynamically, the system can optimize performance while controlling resource consumption, allowing more aggressive re-distribution when resources are abundant and more conservative approaches when resources are constrained.
Solution Approach 2:
The patent implements partial re-distribution by re-distributing only specific data partitions that are causing performance bottlenecks, rather than re-distributing the entire dataset. This selective approach reduces resource overhead while still achieving performance improvements for the most critical data access patterns.
3Productivity
If data distribution is optimized for specific access patterns, then performance for those patterns improves, but performance for other access patterns deteriorates
Solution Approach 1:
The system achieves multi-functionality by maintaining the capability to serve multiple access patterns simultaneously. Through continuous monitoring and adaptive re-distribution, the system can optimize for different access patterns at different times, making the distribution scheme universally applicable to various workload types rather than being specialized for a single pattern.
Data Source
AI summary
A database system comprises a plurality of servers with data storage. The system holds first and second database tables. Records of the first database table are distributed among the servers according to values of a first distribution column. The servers can receive access requests and log workload data representative of the requests. Based on the workload data, the servers can determine a performance impact associated with columns of the tables and, upon determining that a performance impact associated with one of the columns is greater than a performance impact associated with the first distribution column, the servers can re-distribute the records among the plurality of servers according to values of the second column.