Coupled Compute Storage Autoscaling for Kubernetes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
In containerized applications, misalignment between compute and storage resources can lead to wasted compute resources and application failures due to inadequate storage resources, causing data loss and prolonged downtime, especially since compute and storage regions are separately managed and orchestrated.
Innovation Solution
Implementing a coupled compute and storage autoscaling solution that proactively monitors the health of storage resources and aligns compute and storage resource allocations in real-time, ensuring adequate storage is available before allocating compute resources, and automatically scales or allocates healthy storage resources if issues are detected.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If compute and storage resources are allocated independently through separate management, then resource allocation flexibility is improved, but resource alignment and coordination between compute and storage becomes problematic
Solution Approach 1:
The patent merges compute resource allocation and storage resource allocation into a unified coupled autoscaling system. The autoscaler simultaneously manages both compute and storage resources, ensuring they are allocated and scaled together rather than independently, thereby maintaining flexibility while achieving proper resource alignment.
Solution Approach 2:
The system implements feedback mechanisms where the autoscaler continuously monitors storage resource health and availability, and adjusts compute resource allocation accordingly. This feedback loop ensures that compute and storage resources remain aligned, with the autoscaler responding to storage conditions by modifying compute allocations to maintain proper coordination.
2Productivity
If storage resources are allocated without health verification, then allocation speed is improved, but application failures and data loss risks increase
Solution Approach 1:
The patent performs preliminary health verification of storage resources before allocating them to applications. The autoscaler checks storage health status in advance and only allocates compute resources when storage is confirmed to be healthy, preventing application failures while maintaining efficient allocation processes.
Solution Approach 2:
The system continuously monitors storage resource health and uses this feedback to guide allocation decisions. When storage health deteriorates, the autoscaler detects this through feedback mechanisms and prevents or reverses compute allocations that would depend on unhealthy storage, thereby maintaining reliability without significantly impacting allocation speed.
3Speed
If compute resources are allocated without ensuring storage availability, then compute scaling responsiveness is improved, but application downtime increases
Solution Approach 1:
The patent performs preliminary checks on storage availability and health before proceeding with compute resource allocation. The coupled autoscaler ensures storage is ready and healthy before allocating compute resources, preventing application downtime caused by storage unavailability while maintaining rapid scaling responses.
Solution Approach 2:
The system maintains continuous monitoring and coordination between compute and storage allocation processes. The autoscaler ensures seamless transitions and allocations by keeping the storage and compute systems synchronized, eliminating interruptions and downtime while preserving scaling responsiveness through continuous operation.
4Ease of operation
If separate management of compute and storage regions is maintained, then operational independence is improved, but coordinated scaling and resource alignment becomes difficult
Solution Approach 1:
The patent introduces an intermediary autoscaler component that mediates between compute and storage management systems. This intermediary coordinates scaling operations and resource allocation between the independently managed compute and storage regions, reducing coordination complexity while preserving operational independence through a dedicated coordination layer.
Data Source
AI summary
Techniques are provided for coupled compute and storage resource autoscaling. Applications may be hosted within an application hosting environment (e.g., containerized applications hosted within Kubernetes) that allocations certain amounts of compute resources (e.g., processor and memory resources) to the applications for execution. The applications may store data within persistent of a backend storage platform separate from the application hosting environment. An autoscaler monitors the health of storage resources allocated to an application during deployment and runtime of the application so that issues with the storage resources can be preemptively identified and resolved for non-disruptive operation of the application. The autoscaler scales both the compute resources and the storage resources assigned to the application so that the application can continue operation in a non-disruptive manner.