Dynamic Data Chunk Size Adjustment for Network Copy Operations

Resolve Bottlenecks,
Find Innovative SolutionsGenerate Solutions

Solution Overview

Problem

Conventional systems face inefficiencies in data chunk size selection during copy operations, leading to underutilization of resources or server throttling, as fixed chunk sizes fail to optimize resource utilization and transfer speed.

Innovation Solution

A 'copy manager' dynamically adjusts data chunk sizes based on resource statistics and performance thresholds, including 'hard', 'soft', and 'use case' limits, to balance resource utilization and transfer efficiency.

Engineering Contradictions & Design Principles

VSEngineering Contradiction Analysis

1Adaptability or versatility

If a fixed data chunk size is used, then the copy operation completes with stable resource utilization, but the system cannot adapt to changing conditions leading to underutilization or throttling

Engineering Contradiction:

Improveadaptability to changing system conditionsVSAvoidcomplexity of copy management system

Core Design Contradiction:

Adaptability or versatilityVSDevice complexity

Solution Approach 1:

The patent implements dynamic chunk size adjustment where the copy manager continuously monitors system conditions (CPU utilization, memory availability, I/O throughput) and adapts the data chunk size during the copy operation. This allows the system to respond to changing conditions in real-time, improving adaptability without requiring complete system redesign

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The copy manager establishes a feedback loop that monitors system performance metrics and uses this information to adjust chunk sizes. The system measures actual throughput and resource utilization, compares against target thresholds, and modifies chunk sizes accordingly, creating a self-regulating mechanism that balances adaptability with controlled complexity

Inventive Principle:

Principle #23Feedback

2Speed

If a larger data chunk size is used, then transfer speed increases, but server resources may be overwhelmed causing throttling

Engineering Contradiction:

Improvedata transfer speedVSAvoidsystem stability

Core Design Contradiction:

SpeedVSReliability

Solution Approach 1:

The system dynamically adjusts chunk size based on real-time monitoring of server resource utilization. When resources are abundant, larger chunks increase transfer speed; when resources are constrained, smaller chunks prevent throttling. This dynamic adaptation resolves the contradiction between speed and stability

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The patent changes the parameter of data chunk size based on system conditions. By adjusting this critical parameter in response to monitored metrics (CPU usage, memory pressure, I/O throughput), the system optimizes transfer speed while maintaining stability, avoiding both underutilization and throttling

Inventive Principle:

Principle #35Parameter changes

3Reliability

If a smaller data chunk size is used, then resource utilization remains stable, but transfer speed decreases due to underutilization

Engineering Contradiction:

Improvesystem stabilityVSAvoiddata transfer speed

Core Design Contradiction:

ReliabilityVSSpeed

Solution Approach 1:

Rather than using a consistently small chunk size, the system dynamically scales chunk size based on available resources. When system capacity allows, larger chunks are used to maximize transfer speed while maintaining stability through continuous monitoring and adjustment

Inventive Principle:

Principle #15Dynamics

4Reliability

If a fixed compromised chunk size is used, then both underutilization and throttling are avoided, but optimal resource utilization is not achieved

Engineering Contradiction:

Improvesystem stabilityVSAvoidcopy operation efficiency

Core Design Contradiction:

ReliabilityVSProductivity

Solution Approach 1:

The system transitions from a static compromised chunk size to a dynamic adjustment mechanism that seeks optimal performance. By continuously monitoring resources and adapting chunk size, the system achieves better than compromised efficiency while maintaining stability through controlled changes

Inventive Principle:

Principle #15Dynamics

Solution Approach 2:

The copy manager uses feedback from performance monitoring to identify when the current chunk size is suboptimal and makes adjustments to improve efficiency. This feedback-driven approach allows the system to exceed compromised performance while maintaining stability through measured, data-driven adjustments

Inventive Principle:

Principle #23Feedback

Data Source

PatentUS11811870B1Methods and systems for dynamically adjusting data chunk sizes copied over a network

Publication Date: 2023.11.07 DELL PROD LP

AI summary

A method for copying source data from a source server to a destination server, that includes initiating, by a source copy manager, a copy operation of the source data, where during the copy operation, the method further includes obtaining resource statistics, setting a data chunk size based on the resource statistics, copying a data chunk to a read queue, where the data chunk includes a portion of the source data, and where the data chunk has the data chunk size, and sending, from the read queue, the data chunk to a network device, and ending the copy operation.