Role-Based Autoscaling for Stateful Application Nodes
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing autoscaling approaches for stateful applications in large scale virtual data processing environments are inefficient due to their application-specific nature, requiring in-depth knowledge of application logic and failing to account for dependencies among roles, leading to suboptimal resource utilization and potential overprovisioning.
Innovation Solution
A role-based autoscaling approach that uses policies defined by conditions on resource utilization (CPU, memory, network, disk I/O) to scale nodes independently and in tandem with dependent roles, allowing for fine-grained control and decoupling from specific application logic, thereby improving resource utilization and adaptability across various applications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If application-specific autoscaling approaches are used, then autoscaling can be tailored to specific application needs, but it requires in-depth knowledge of application logic and reduces versatility across different applications
Solution Approach 1:
The patent implements a universal autoscaling framework that works across multiple applications by defining roles and policies at a high level, independent of specific application logic. The system can autoscale any stateful application by mapping its components to defined roles (e.g., data processing roles, storage roles), eliminating the need for application-specific customization while maintaining effective autoscaling control.
2Productivity
If traditional autoscaling approaches are used, then scaling can be performed independently for each component, but it fails to account for dependencies among roles leading to suboptimal resource utilization
Solution Approach 1:
The patent merges the scaling control of dependent roles into a single coordinated operation. When a role needs to be scaled, the system automatically identifies and scales all roles that depend on it, ensuring resource efficiency. For example, when data processing roles are scaled out, dependent storage roles are automatically scaled out as well, preventing resource waste while maintaining simple user interaction through a single scaling action.
3Manufacturing precision
If fine-grained control of scaling is implemented, then resource management precision is improved, but the system complexity increases
Solution Approach 1:
The patent segments the autoscaling system into distinct, manageable components: roles, dependencies, and policies. Each role is independently defined with its own scaling parameters and dependencies, allowing precise control of individual components while simplifying overall management. The segmentation enables users to configure scaling policies for specific roles without affecting the entire system, balancing precision with manageability.
Data Source
AI summary
Example implementations relate to a role-based autoscaling approach for scaling of nodes of a stateful application in a large scale virtual data processing (LSVDP) environment. Information is received regarding a role performed by the nodes of a virtual cluster of an LSVDP environment on which a stateful application is or will be deployed. Role-based autoscaling policies are maintained defining conditions under which the roles are to be scaled. A policy for a first role upon which a second role is dependent specifies a condition for scaling out the first role by a first step and a second step by which the second role is to be scaled out in tandem. When load information for the first role meets the condition, nodes in the virtual cluster that perform the first role are increased by the first step and nodes that perform the second role are increased by the second step.