Adaptive Event Data Store Partitioning for Multi-Tenant RPU Scaling
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing distributed event data stores in multi-tenant architectures face challenges in dynamically provisioning resources to handle highly variable and dynamic event storage demands, leading to under-provisioning or over-provisioning issues, which are not effectively addressed by current solutions like Elasticsearch, MongoDB, and Apache Cassandra, and require expert-level manual adjustments.
Innovation Solution
A managed security service provider (MSSP) dynamically adjusts the number of resource provision units (RPUs) in partitions based on real-time demand and time-based calculations to prevent under-provisioning or over-provisioning, using a Virtualization Data Store Layer for transparent resource management.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If static resource provisioning is used for distributed event data stores, then system simplicity is maintained, but the system cannot adapt to highly variable event storage demands, leading to under-provisioning or over-provisioning
Solution Approach 1:
The patent implements dynamic resource provisioning by continuously monitoring event storage demand and automatically adjusting the number of resource provision units (RPUs) allocated to each tenant. The system transitions from static to dynamic provisioning, where RPU allocation changes in real-time based on actual event data volume and growth rate, allowing the system to adapt to highly variable demands without manual intervention
Solution Approach 2:
The system employs feedback mechanisms by monitoring event storage metrics (current volume, growth rate, projection to fill time) and using this information to automatically adjust RPU allocation. The feedback loop continuously evaluates demand patterns and modifies resource provisioning accordingly, enabling the system to respond to changing conditions while maintaining automated control
2Measurement precision
If manual expert-level adjustments are made to provisioning, then resource allocation accuracy improves, but operational complexity and time requirements increase significantly
Solution Approach 1:
The patent implements self-service provisioning where the system automatically monitors its own resource utilization, determines provisioning status (under-provisioned, adequately provisioned, or over-provisioned), and adjusts RPU allocation without requiring expert human intervention. The automated system performs measurements, analyzes trends, and executes provisioning decisions, eliminating manual expert adjustments while maintaining high accuracy
Solution Approach 2:
The system performs preliminary actions by proactively monitoring event data growth patterns and projecting future storage needs before actual resource constraints are reached. By analyzing current event rates and growth trends, the system anticipates future demand and adjusts provisioning in advance, preventing both under-provisioning and over-provisioning conditions
3Reliability
If the number of RPUs is increased to handle peak demand, then storage capacity and processing throughput are sufficient, but resources are wasted during low-demand periods
Solution Approach 1:
The patent implements dynamic RPU allocation that adjusts resource capacity to match actual demand levels. During peak event generation periods, the system automatically increases RPU allocation to ensure sufficient storage and processing throughput, maintaining reliability. During low-demand periods, the system reduces RPU allocation proportionally, optimizing resource utilization efficiency and preventing resource waste while maintaining adequate capacity for actual needs
4Productivity
If the number of RPUs is decreased to optimize cost, then resource efficiency improves, but the system cannot handle sudden increases in event volume
Solution Approach 1:
The system performs preliminary actions by continuously monitoring event data growth rates and projecting future storage requirements. By analyzing current event volumes and growth trends, the system anticipates upcoming demand increases and proactively scales RPU capacity before resources are exhausted, ensuring reliability is maintained while optimizing current resource efficiency
Solution Approach 2:
The patent implements feedback-driven resource scaling where the system continuously monitors event storage metrics and automatically adjusts RPU allocation in response to changing conditions. The feedback mechanism ensures that resource efficiency is optimized during stable low-demand periods while automatically increasing capacity when demand growth patterns indicate future resource constraints, maintaining reliability dynamically
Data Source
AI summary
Systems and methods for adaptively provisioning a distributed event data store of a multi-tenant architecture are provided. According to one embodiment, a managed security service provider (MSSP) maintains a distributed event data store on behalf of each tenant of the MSSP. For each tenant, the MSSP periodically determines a provisioning status for a current active partition of the distributed event data store of the tenant. Further, when the determining indicates an under-provisioning condition exits, the MSSP dynamically increases number of resource provision units (RPUs) to be used for a new partition to be added to the partitions for the tenant by a first adjustment ratio. While, when the determining indicates an over-provisioning condition exists, the MSSP dynamically decreases the number of RPUs to be used for subsequent partitions added to the partitions for the tenant by a second adjustment ratio.