Rack-Level Scheduler for Long Tail Latency in SSD Systems

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Datacenter-level schedulers fail to efficiently utilize high-performance storage drivers like NVMe-based SSDs, leading to increased latency unpredictability and long tail latency due to inadequate resource management and incorrect workload placement, especially in multi-tenant environments where applications compete for shared resources.

Innovation Solution

A rack-level scheduler with a tail latency anomaly monitor (TLAM) and target discovery unit (TDU) is implemented to monitor latency requirements, collect system utilization information, calculate rewards for workload migration, and determine the optimal target server node within the rack system, minimizing data movement and overhead while maximizing storage performance.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If datacenter-level scheduler is used to manage workload distribution, then global resource visibility is provided, but storage device performance is not optimized due to lack of rack-level scheduling

Engineering Contradiction:

Improveglobal resource visibilityVSAvoidstorage device performance

Core Design Contradiction:

Adaptability or versatilityVSProductivity

Solution Approach 1:

The patent divides the scheduling function into two levels: datacenter-level scheduler for global resource management and rack-level scheduler for local storage optimization. This segmentation allows each scheduler to focus on its specific function, with the rack-level scheduler directly optimizing storage device performance while the datacenter-level scheduler maintains global visibility.

Inventive Principle:

Principle #1Segmentation

Solution Approach 2:

The patent adds a new scheduling dimension at the rack level between the datacenter-level scheduler and individual storage devices. This intermediate scheduling layer operates in a different dimension (rack-level locality) to optimize storage performance without compromising global resource allocation.

Inventive Principle:

Principle #17Another dimension (Dimensionality change)

2Productivity

If workload migration is performed to reduce latency, then storage performance is improved, but data movement overhead increases

Engineering Contradiction:

Improvestorage performanceVSAvoiddata movement overhead

Core Design Contradiction:

ProductivityVSLoss of time

Solution Approach 1:

The rack-level scheduler focuses on local workload migration within the same rack, optimizing storage performance by placing workloads on storage devices with available capacity and similar access patterns. This local approach minimizes data movement distance and overhead while improving storage performance.

Inventive Principle:

Principle #3Local quality

Solution Approach 2:

The patent performs partial workload migration only when necessary to balance storage device utilization and reduce latency. Instead of migrating all workloads, it selectively migrates portions of workloads to storage devices that can handle them with lower latency, reducing unnecessary data movement overhead.

Inventive Principle:

Principle #16Partial or excessive action

3Productivity

If multiple services are co-located on the same server to improve resource utilization, then system efficiency increases, but latency unpredictability increases due to resource competition

Engineering Contradiction:

Improveresource utilizationVSAvoidlatency predictability

Core Design Contradiction:

ProductivityVSReliability

Solution Approach 1:

The rack-level scheduler implements feedback mechanisms to monitor storage device utilization and workload performance in real-time. When latency thresholds are exceeded or utilization becomes unbalanced, the scheduler automatically triggers workload migration to restore predictability, allowing multiple services to co-locate efficiently.

Inventive Principle:

Principle #23Feedback

Solution Approach 2:

The scheduling system dynamically adjusts workload placement based on real-time storage device conditions and workload characteristics. This dynamic approach allows the system to maintain high resource utilization while adapting to changing conditions that affect latency predictability.

Inventive Principle:

Principle #15Dynamics

Data Source

PatentUS11507435B2Rack-level scheduling for reducing the long tail latency using high performance SSDs

Publication Date: 2022.11.22 SAMSUNG ELECTRONICS CO LTD

AI summary

A method for migrating a workload includes: receiving workloads generated from a plurality of applications running in a plurality of server nodes of a rack system; monitoring latency requirements for the workloads and detecting a violation of the latency requirement for a workload; collecting system utilization information of the rack system; calculating rewards for migrating the workload to other server nodes in the rack system; determining a target server node among the plurality of server nodes that maximizes the reward; and performing migration of the workload to the target server node.