ML-Based Resource Availability Prediction for Storage Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Fixed or manual scheduling of system services in computer systems becomes inefficient due to changing resource utilization patterns over time, leading to collisions with busy times and underutilization of resources, especially in 24/7 workloads across different time zones.

Innovation Solution

Applying machine learning to analyze historical resource utilization patterns, predict future utilization, and automate the scheduling of system services, such as file system services, to optimize resource usage by identifying availability windows.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If fixed or manual scheduling of system services is used, then scheduling simplicity is maintained, but resource utilization efficiency deteriorates due to collisions with busy times and underutilization during low-demand periods

Engineering Contradiction:

Improveresource utilization efficiencyVSAvoidscheduling system complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent implements dynamic scheduling by training a machine learning model to predict resource utilization patterns and automatically adjusting service schedules based on predicted availability windows. The scheduler transitions from static fixed-time scheduling to dynamic adaptive scheduling that responds to changing workload conditions, thereby improving resource utilization efficiency without requiring complex manual intervention

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system employs self-service mechanisms where the machine learning model autonomously learns from historical resource utilization data and automatically generates optimized service schedules. The scheduler independently identifies availability windows and assigns services without continuous human input, reducing operational complexity while enhancing productivity through automated intelligent decision-making

Inventive Principle:

Principle #25Self-service

2Adaptability or versatility

If fixed scheduling at off-peak hours is used, then resource collision is reduced, but adaptability to changing workload patterns deteriorates in 24/7 global workloads

Engineering Contradiction:

Improveadaptability to workload changesVSAvoidsystem integrity maintenance

Core Design Contradiction:

Adaptability or versatilityVSReliability

Solution Approach 1:

The patent implements feedback mechanisms where the machine learning model continuously monitors actual resource utilization patterns and compares them with predictions. The system uses this feedback to refine its models and adjust scheduling decisions, enabling adaptability to changing global workload patterns while maintaining system integrity through data-driven reliability assessments

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The system dynamically changes scheduling parameters based on predicted resource availability. Instead of fixed off-peak hours, the scheduler adjusts service execution times based on real-time predictions of resource utilization patterns across different time zones and workload conditions, thereby achieving both adaptability and reliability through parameter optimization

Inventive Principle:

Principle #35Parameter changes

3Productivity

If machine learning-based predictive scheduling is implemented, then resource utilization efficiency is improved, but computational complexity and training requirements increase

Engineering Contradiction:

Improveservice execution efficiencyVSAvoidmachine learning model complexity

Core Design Contradiction:

ProductivityVSDevice complexity

Solution Approach 1:

The patent applies preliminary action by training the machine learning model in advance using historical resource utilization data before deployment. The model pre-learns utilization patterns and is ready to generate predictions when needed, improving service execution efficiency without requiring complex real-time computations during actual scheduling operations

Inventive Principle:

Principle #10Preliminary action

Solution Approach 2:

The system uses partial action by implementing machine learning predictive scheduling for specific critical services or resource types where the benefit justifies the complexity. This selective approach allows improved efficiency in key areas while managing overall computational complexity through targeted rather than universal application of ML models

Inventive Principle:

Principle #16Partial or excessive action

Data Source

PatentUS11514317B2Machine learning based resource availability prediction

Publication Date: 2022.11.29 EMC IP HLDG CO LLC

AI summary

Requests from file system services of a storage system are registered. Each file system service, when executed, utilizes one or more resources of the storage system. Each request includes information describing resource requirements required by a respective file system service. Resource utilization data of the resources are collected over a period of time. The resource utilization data includes an identification of a resource, a timestamp, and a measurement indicating a utilization level of the resource corresponding to the timestamp. A machine learning model is trained to predict utilization patterns of the resources. Execution of the file system services are scheduled based on the predicted utilization patterns. Monitoring is conducted during the execution of the file system services. Based on the monitoring a determination is made as to whether the machine learning model should be retrained.