Adaptive Self-Maintenance Scheduler for Distributed Storage

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Current self-maintenance processes in distributed file systems are resource-intensive and often interfere with foreground processes, as they are typically scheduled based on rigid time, event, or file system state, leading to suboptimal performance due to unforeseen system activity.

Innovation Solution

An adaptive scheduling mechanism that uses a load predictor and maintenance process scheduler to dynamically determine optimal times for self-maintenance processes based on current system activities, historical data, and external events, adjusting schedules in response to resource availability and usage patterns.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Reliability

If self-maintenance processes are scheduled to execute frequently to ensure file system consistency, then reliability is improved, but system performance deteriorates due to resource consumption interfering with foreground processes

Engineering Contradiction:

Improvefile system consistencyVSAvoidsystem performance

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The patent implements dynamic scheduling of self-maintenance processes by continuously monitoring system workload and resource availability. The scheduler adjusts the execution timing and frequency of maintenance tasks based on real-time system state, allowing the system to maintain file system consistency while minimizing interference with foreground processes. This dynamic adaptation resolves the contradiction by making the maintenance schedule flexible rather than fixed.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The system changes scheduling parameters (timing, frequency, resource allocation) based on system conditions. By monitoring metrics such as I/O workload, CPU utilization, and memory availability, the scheduler modifies maintenance process parameters to execute during low-activity periods, thereby maintaining reliability while preserving system performance.

Inventive Principle:

Principle #35Parameter changes

2Reliability

If self-maintenance processes are scheduled based on fixed time intervals to ensure regular maintenance, then reliability is improved, but adaptability deteriorates due to inability to respond to unforeseen system activity

Engineering Contradiction:

Improvemaintenance regularityVSAvoidresponse to system activity

Core Design Contradiction:

ReliabilityVSAdaptability or versatility

Solution Approach 1:

The patent transforms fixed-time scheduling into dynamic scheduling by introducing workload monitoring and adaptive decision-making. The system evaluates current system state before executing maintenance tasks, allowing it to maintain regular maintenance patterns while adapting to unexpected system activity. This resolves the contradiction between schedule regularity and system adaptability.

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The scheduler implements feedback mechanisms by continuously monitoring system workload and adjusting maintenance schedules based on observed conditions. This closed-loop control allows the system to maintain reliable maintenance patterns while responding adaptively to changing system states, resolving the contradiction between fixed scheduling and adaptability.

Inventive Principle:

Principle #23Feedback

3Productivity

If resource-intensive self-maintenance processes are executed during high-activity periods to reduce backlog, then productivity is improved, but harmful factors increase due to interference with foreground processes

Engineering Contradiction:

Improvemaintenance throughputVSAvoidinterference with foreground processes

Core Design Contradiction:

ProductivityVSObject-generated harmful factors

Solution Approach 1:

The patent applies local quality by differentiating between different system states and applying appropriate scheduling strategies. During low-activity periods, the system executes resource-intensive maintenance processes to build backlog reduction momentum. During high-activity periods, it reduces maintenance intensity to minimize interference. This localized adaptation resolves the contradiction between maintenance productivity and foreground process performance.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The system employs periodic execution of maintenance processes during identified low-activity windows, allowing accumulation of maintenance work to be performed in batches rather than continuously. This periodic approach maintains high maintenance throughput during suitable periods while minimizing interference during critical foreground operations.

Inventive Principle:

Principle #19Periodic action

Data Source

PatentUS10303508B2Adaptive self-maintenance scheduler

Publication Date: 2019.05.28 COHESITY INC

AI summary

Embodiments presented herein disclose adaptive techniques for scheduling self-maintenance processes. A load predictor estimates, based on a current state of a distributed storage system, an amount of resources of the system required to perform each of a plurality of self-maintenance processes. A maintenance process scheduler estimates, based on one or more inputs, an amount of resources of the distributed system available to perform one or more of the self-maintenance processes during at least a first time period. The maintenance process scheduler determines a schedule for the one or more of the self-maintenance processes to perform during the first time period, based on the estimated amount of resources required and available.