Automated Self-Scaling Database System for Write Operations
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current cloud-based computing systems require manual intervention by database administrators to scale stateful transactional database services, which can lead to service disruptions and inefficiencies due to the need for monitoring and provisioning of computation and storage resources.
Innovation Solution
An automated self-scaling database system that includes a primary database, a standby database, an application server, and an automated self-scaling module (SSM) that monitors telemetry data to predict and automatically upscale computation and storage resources, enabling vertical and horizontal scaling without human intervention, and automatically distributes tenants across multiple primary databases for write operations.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If manual scaling by database administrators is used, then service control and resource provisioning can be performed, but service disruptions occur and operational efficiency decreases
Solution Approach 1:
The database system implements self-service through automated self-scaling modules that monitor system metrics, predict scaling needs, and execute scaling operations without human intervention. The module automatically detects when computation or storage resources need to be scaled and performs the scaling operations, eliminating the need for manual administrator intervention and thereby improving both operational efficiency and service availability.
Solution Approach 2:
The system performs preliminary action by predicting future scaling requirements based on monitored telemetry data and system trends. The automated self-scaling module analyzes current resource utilization patterns and proactively initiates scaling operations before resources are fully exhausted, preventing service disruptions and maintaining continuous availability while automating the entire process.
2Reliability
If automated self-scaling is implemented, then service availability and operational efficiency are improved, but system complexity increases
Solution Approach 1:
The automated self-scaling module implements feedback by continuously monitoring system telemetry data, comparing current resource utilization against predefined thresholds and trends, and automatically adjusting scaling operations based on this feedback loop. This closed-loop control system maintains service availability through automated responses to changing conditions while managing complexity through standardized monitoring and control mechanisms.
Solution Approach 2:
The patent introduces an automated self-scaling module as an intermediary layer between the database system and the infrastructure. This module abstracts the complexity of scaling operations, providing a standardized interface that monitors system state and executes scaling decisions without requiring direct complex interactions between database components and infrastructure management, thereby managing system complexity while maintaining high availability.
3Productivity
If multiple primary databases are used for write operations, then write capacity and availability are improved, but data distribution and consistency management becomes more complex
Solution Approach 1:
The system applies segmentation by dividing the tenant workload across multiple primary databases. Each primary database handles a specific subset of tenants, allowing write operations to be distributed and parallelized across multiple independent database instances. This segmentation increases overall write capacity while maintaining data consistency within each segmented database, managing complexity through clear workload partitioning.
Solution Approach 2:
The patent implements local quality by optimizing each primary database instance for its specific tenant subset, allowing each database to be tuned and managed independently for its local workload characteristics. This approach enables specialized optimization for different tenant groups while maintaining overall system consistency, improving write capacity through parallel processing without requiring complex global coordination across all databases.
Data Source
AI summary
A method and system are provided for automatically scaling out write operations in an automated self-scaling database system. The database system initially includes a first primary database (PD) and a first standby database. Telemetry information from the first PD is analyzed to determine whether the first PD has reached its maximum computation and storage capacity. If so, a write scaling process is automatically initiated so storage capacity and computation resources of the system can be upscaled for write operations. To write scale, a new PD can be automatically provisioned, and once it is active, tenants of the system can be distributed among the first PD and the new PD (e.g., equally distributed or distributed based on workload, etc.) such that the first PD handles requests that originate from a first group of the tenants, and the new PD handles requests that originate from a second group of the tenants.