Network Data Transfer Latency Optimization
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Network latencies in data replication processes across computer networks lead to performance issues due to propagation delay, network congestion, and processing delays, resulting in suboptimal throughput.
Innovation Solution
A system and method that dynamically adjusts transmission parameters such as data chunk sizes and the number of streams processed in response to determined network latencies, optimizing data transfer by selecting the most effective transmission mode to enhance overall throughput and performance.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data is transmitted over a network with fixed transmission parameters, then the system is simple to operate, but network latency reduces data transfer throughput
Solution Approach 1:
The system dynamically adjusts transmission parameters (chunk size, batch size, number of streams) based on real-time network latency measurements. The backup server monitors latency and automatically modifies transmission characteristics to optimize throughput, transforming a static transmission system into an adaptive one that responds to changing network conditions.
Solution Approach 2:
The invention changes physical and operational parameters of data transmission including chunk size, batch size, and number of concurrent streams. By varying these parameters in response to measured latency, the system optimizes data transfer performance without requiring changes to the underlying network infrastructure.
2Productivity
If transmission parameters are dynamically adjusted to optimize throughput, then data transfer performance improves, but system complexity increases
Solution Approach 1:
The backup server performs self-optimization by automatically measuring its own network latency and adjusting its transmission parameters without external intervention. The system monitors its performance metrics and autonomously modifies chunk size, batch size, and stream count to maintain optimal throughput.
Solution Approach 2:
The system implements a feedback loop where transmission performance is measured, compared against targets, and used to drive subsequent parameter adjustments. The backup server continuously monitors latency and throughput, using this feedback to iteratively refine transmission parameters for optimal performance.
Data Source
Figure 1
Figure 2
Figure 3
AI summary
A system and method for efficient transfer of data over a network. A data source and a destination are coupled to one another via a network. The data source is configured to transmit data to the destination and determine a network latency associated with the transmission. In response to determining a first transmission parameter may not be optimized, the source is configured to modify the first transmission parameter and evaluate its effect on data throughput. The source may repeat such modifications as long as throughput is increased. Once the first transmission parameter is deemed optimized, the source may perform a similar procedure for a second transmission parameter. In various embodiments, the first transmission parameter may correspond to a packet size and the second transmission parameter may correspond to a number of streams being processed by the source.