Application Resource Management via Reinforcement Learning

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional resource management by computer operating systems lacks knowledge of application workflows, leading to inefficiencies and user dissatisfaction due to sluggish application performance during resource-intensive tasks.

Innovation Solution

Implementing an environment-aware application-based resource management system using reinforcement learning, which identifies current application states, determines actions to optimize resource usage, and updates a reinforcement learning model based on rewards to improve application health and responsiveness.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the operating system allocates memory and schedules processes without knowledge of application workflows, then resource allocation is performed, but application responsiveness deteriorates during resource-intensive tasks

Engineering Contradiction:

Improveresource allocation efficiencyVSAvoidapplication responsiveness

Core Design Contradiction:

ProductivityVSEase of operation

Solution Approach 1:

The patent implements feedback mechanisms where the application monitors its own resource usage and performance metrics, then feeds this information back to the resource management system. This enables dynamic adjustment of resource allocation based on actual application needs and workflow states, resolving the contradiction between efficient resource allocation and application responsiveness.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The application performs self-monitoring of its workflow state and resource consumption patterns. By implementing self-service capabilities within the application layer, the system can make informed decisions about resource allocation without relying solely on operating system-level management, thereby maintaining responsiveness during intensive tasks.

Inventive Principle:

Principle #25Self-service

2Device complexity

If the operating system manages resources at the process level without workflow awareness, then resource allocation is simplified, but resource usage optimization deteriorates

Engineering Contradiction:

Improveresource management complexityVSAvoidresource usage efficiency

Core Design Contradiction:

Device complexityVSProductivity

Solution Approach 1:

The patent segments resource management into multiple layers: operating system-level process management and application-level workflow management. This segmentation allows the OS to maintain simple process scheduling while the application layer handles workflow-aware resource optimization, resolving the contradiction between management simplicity and optimization effectiveness.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent adds a new dimension to resource management by introducing workflow state awareness at the application layer. This dimensional addition enables resource allocation decisions to consider not just process metrics but also workflow context, achieving better optimization without significantly increasing overall system complexity.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

3Adaptability or versatility

If applications use increasing amounts of system resources for complex operations, then application functionality is enhanced, but application responsiveness deteriorates

Engineering Contradiction:

Improveapplication functionalityVSAvoiduser interface responsiveness

Core Design Contradiction:

Adaptability or versatilityVSEase of operation

Solution Approach 1:

The patent implements dynamic resource allocation that adapts to changing workflow states. The system continuously monitors application state and dynamically adjusts resource consumption based on current operational needs, allowing complex functionality to be maintained while preserving responsiveness during user interactions through adaptive resource management.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent employs periodic monitoring and adjustment of resource allocation based on workflow state transitions. By implementing periodic checks of application state and adjusting resource usage accordingly, the system ensures that complex operations receive necessary resources while maintaining responsiveness during user-facing operations.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS11556393B2Environment aware application-based resource management using reinforcement learning

Publication Date: 2023.01.17 ADOBE INC

AI summary

A resource management system of an application takes various actions to improve or maintain the health of the application (e.g., keep the application from becoming sluggish). The resource management system maintains a reinforcement learning model indicating which actions the resource management system is to take for various different states of the application. The resource management system performs multiple iterations of a process of identifying a current state of the application, determining an action to take to manage resources for the application, and taking the determined action. In each iteration, the resource management system determines the result of the action taken in the previous iteration and updates the reinforcement learning model so that the reinforcement learning model learns which actions improve the health of the application and which actions do not improve the health of the application.