VTL Adaptive Commit Latency Optimization

Resolve Bottlenecks,
Find Innovative Solutions
Generate Solutions

Solution Overview

Problem

Virtual tape libraries (VTLs) face long latencies and potential timeouts when writing data to file systems due to constrained resources, leading to inefficient data transfer and potential system crashes.

Innovation Solution

The implementation of an adaptive commit module that dynamically adjusts the size of RPC data sets based on latency, allowing the VTL to optimize writes to the file system by increasing or decreasing the data set size according to current system load, thereby minimizing timeouts and maximizing resource utilization.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Productivity

If the VTL performs writes to the file system as uncommitted writes with periodic commit requests, then the data transfer efficiency is improved, but the latency becomes very long when file system resources are constrained

Engineering Contradiction:
Improvedata transfer efficiencyVSAvoidlatency
Core Design Contradiction:
ProductivityVSLoss of time

Solution Approach 1:

The patent applies dynamics by making the commit request interval adaptive rather than fixed. The VTL monitors latency metrics and dynamically adjusts the interval between commit requests based on current file system conditions. When latency is high, the interval is reduced to prevent timeouts; when latency is low, the interval is increased to maximize throughput. This dynamic adjustment resolves the contradiction between maintaining high productivity and avoiding excessive latency.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent implements feedback mechanisms where the VTL continuously monitors latency metrics from the file system and uses this information to adjust commit request timing. The system measures the time between write completion and data availability, then feeds this information back to control the commit interval. This closed-loop feedback allows the system to automatically optimize the balance between throughput and latency based on real-time file system conditions.

Inventive Principle:
Principle #23Feedback

2Loss of time

If the VTL issues frequent commit requests to reduce latency, then the latency is reduced, but the resource utilization of the file system decreases

Engineering Contradiction:
ImprovelatencyVSAvoidresource utilization
Core Design Contradiction:
Loss of timeVSProductivity

Solution Approach 1:

The system dynamically adjusts the commit request frequency based on monitored latency conditions. Rather than using a fixed frequent commit schedule that would waste resources, the VTL only increases commit frequency when latency metrics indicate the file system is constrained. When resources are available, the system uses longer intervals between commits, thereby optimizing the balance between latency reduction and resource utilization.

Inventive Principle:
Principle #15Dynamics

Solution Approach 2:

The patent changes the timing parameter of commit requests from a static value to a dynamic value that adapts to file system conditions. By monitoring latency as a parameter and using it to adjust the commit interval, the system optimizes resource usage. The parameter change allows the system to issue commits only when necessary, avoiding unnecessary resource consumption while still maintaining acceptable latency performance.

Inventive Principle:
Principle #35Parameter changes

3Device complexity

If the VTL uses a fixed commit interval, then the system operation is simple, but the system cannot adapt to changing file system conditions

Engineering Contradiction:
Improvesystem operation simplicityVSAvoidadaptability to file system conditions
Core Design Contradiction:
Device complexityVSAdaptability or versatility

Solution Approach 1:

The patent introduces feedback mechanisms that monitor file system latency and automatically adjust commit timing without requiring complex manual configuration. The VTL measures latency metrics and uses this information to automatically modulate commit request intervals, providing adaptability to changing conditions while keeping the overall system operation relatively simple through automated control rather than complex user management.

Inventive Principle:
Principle #23Feedback

Data Source

PatentUS9600354B1VTL adaptive commit
Publication Date: 2017.03.21 EMC IP HLDG CO LLC
  • US9600354B1 patent drawing
  • US9600354B1 patent drawing
  • US9600354B1 patent drawing

AI summary

According to one embodiment, a virtual tape library (VTL) receives a plurality of small computer system interface (SCSI) data objects, and translates the SCSI data objects into a first set of one or more remote procedure call (RPC) data objects having a first size, wherein the first size of the first RPC data set is based on a current limit maintained by the VTL. The VTL writes the first RPC data set and a first RPC commit request to a file system. The VTL receives a first RPC status corresponding to the first RPC commit from the file system, and determines a first latency time between writing the first RPC commit and receiving the first RPC status. The VTL updates the current limit based on the first latency time in order to optimize a second latency time between a second RPC commit request and a corresponding second RPC status.