State Decision Tree for Reinforcement Learning State Space Reduction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing automatic scaling systems for multi-tier applications face challenges in managing the large state space in reinforced learning, as even small changes in application performance and resource configuration create numerous states, making the rewards table intractable.
Innovation Solution
The implementation of a state decision tree that updates and categorizes states based on similarity measurements, allowing new states to be placed in existing nodes if they share attributes, reducing the number of states to consider and managing the state space effectively.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If reinforced learning is used to automatically scale multi-tier applications, then the application can adapt to changing workload conditions, but the number of states becomes too large to manage making the rewards table intractable
Solution Approach 1:
The patent merges similar states by computing similarity metrics between state representations and grouping them into equivalence classes. States that are sufficiently similar according to the similarity threshold are combined into a single representative state, reducing the total number of states from exponentially large to a manageable size while preserving the adaptive scaling capability through the use of decision trees to represent merged state groups.
Solution Approach 2:
The patent segments the large state space by using decision trees to hierarchically partition states based on feature importance. The state space is divided into regions defined by decision tree nodes, where each node splits states based on the most discriminative feature. This segmentation reduces complexity by organizing states into structured groups that can be efficiently managed in the rewards table.
2Measurement precision
If detailed state tracking is implemented to ensure accurate performance monitoring, then scaling decisions can be precisely optimized, but the computational overhead and storage requirements become prohibitive
Solution Approach 1:
The patent extracts only the most relevant features from complete state representations using decision tree analysis. By identifying and extracting the key features that most influence scaling decisions, the system maintains measurement precision for critical parameters while discarding redundant information, thereby reducing the quantity of data that needs to be tracked and stored.
Solution Approach 2:
The patent transforms the state representation by changing from tracking all individual state parameters to tracking aggregated similarity-based state groups. The parameter transformation involves computing similarity metrics and using decision tree features to represent states, which reduces data volume while preserving the essential information needed for accurate performance monitoring and scaling decisions.
Data Source
AI summary
An automatic scaling system and method for reducing state space in reinforced learning for automatic scaling of a multi-tier application uses a state decision tree that is updated with new states of the multi-tier application. When a new state of the multi-tier application is received, the new state is placed in an existing node of the state decision tree only if a first attribute of the new state is same as a first attribute of any state contained in the existing node and a second attribute of the new state is sufficiently similar to a second attribute of each existing state contained in the existing node based on a similarity measurement of the second attribute of each state contained in the existing node with the second attribute of the new state.