RL State Pruning With Restricted Boltzmann Machines

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Cloud computing providers face challenges in efficiently allocating resources to meet service level agreements (SLAs) due to dynamic workload demands and varying resource requirements, leading to inefficiencies and resource wastage.

Innovation Solution

Utilizing a reinforcement learning model trained with state pruning techniques, specifically using Restricted Boltzmann Machines, to optimize workload placement and resource allocation, reducing computational costs and training times by combining similar states into superstates.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If reinforcement learning is used to optimize workload placement, then resource allocation efficiency is improved, but computational complexity and training time increase

Engineering Contradiction:

Improveresource allocation efficiencyVSAvoidcomputational complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent segments the state space by identifying and grouping similar states together. The reinforcement learning agent learns to recognize state similarities and transitions between grouped states, effectively dividing the complex state space into manageable segments that reduce computational complexity while maintaining allocation efficiency

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent merges similar states into equivalence classes or groups based on their characteristics and outcomes. By combining states that lead to similar resource allocation decisions, the system reduces the total number of states the agent must evaluate, thereby decreasing computational complexity while preserving the ability to make efficient allocation decisions

Inventive Principle:

Principle #5Merging (Combining)

2Manufacturing precision

If complete state space exploration is performed, then optimal workload placement is achieved, but training time increases

Engineering Contradiction:

Improveworkload placement optimalityVSAvoidtraining time

Core Design Contradiction:

Manufacturing precisionVSLoss of time

Solution Approach 1:

The patent performs preliminary analysis to identify and group similar states before the reinforcement learning training begins. By pre-processing the state space to recognize patterns and similarities, the system prepares a condensed state representation that allows faster training while still achieving optimal workload placement decisions

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The patent implements dynamic state grouping where the granularity and organization of state groups can adapt during training. As the agent learns, the state space representation can be dynamically adjusted to focus on the most relevant distinctions, balancing exploration efficiency with placement optimality

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS12579001B2Reinforcement learning space state pruning using Restricted Boltzmann Machines

Publication Date: 2026.03.17 DELL PROD LP

AI summary

Reinforcement learning with space state pruning is disclosed. States of an environment used in training a reinforcement learning model are pruned using a restricted Boltzmann Machine. Reducing the number of states, by pruning, reduces time to convergence.