WAN Traffic Optimization via Cached Stream Deduplication
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for optimizing wide area network (WAN) traffic deduplication are inefficient, as they fail to effectively identify and eliminate duplicate data transfers, leading to wastage of network bandwidth.
Innovation Solution
A network protocol is implemented between local and remote proxy devices to communicate deduplication information, using chunking and hashing algorithms to identify and manage data chunks, allowing for the transmission of only new or duplicate data, thereby reducing redundant transfers across the network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of energy
If traditional WAN traffic transmission is used without deduplication, then all data including duplicates is transmitted ensuring complete data delivery, but network bandwidth is wasted due to redundant data transfers
Solution Approach 1:
The patent segments data into fixed-size chunks and creates unique identifiers for each chunk. By dividing the data stream into manageable segments, the system can identify and deduplicate individual chunks without affecting the completeness of data transmission. Each chunk is processed independently, allowing selective retransmission of only non-duplicate chunks.
Solution Approach 2:
The patent creates a copy of the data stream locally at both sender and receiver proxies, storing chunks in a cache. By comparing local copies with incoming data chunks, the system can identify duplicates and eliminate redundant transmissions while ensuring the original data integrity is maintained through the caching mechanism.
2Loss of energy
If deduplication techniques are implemented to eliminate duplicate data transfers, then network bandwidth is conserved, but system complexity increases due to chunking and hashing requirements
Solution Approach 1:
The patent changes the parameter representation of data by transforming raw data chunks into hash-based identifiers. This parameter transformation allows efficient duplicate detection through simple hash comparison rather than full data comparison, reducing computational complexity while maintaining deduplication effectiveness.
Solution Approach 2:
The patent performs preliminary chunking and hashing of data before transmission. By pre-processing data into standardized chunks with unique identifiers at the source, the system simplifies the deduplication process at both ends, avoiding the need for complex real-time analysis during data transmission.
3Loss of time
If all data chunks are transmitted to ensure complete data delivery, then data integrity is maintained, but transmission time increases due to redundant data
Solution Approach 1:
The patent implements a feedback mechanism where the receiving proxy sends acknowledgments for successfully received and cached chunks back to the sending proxy. This feedback allows the sender to identify which chunks were duplicates and can be skipped in subsequent transmissions, reducing transmission time while maintaining data integrity through selective retransmission.
Solution Approach 2:
The patent makes the data transmission process dynamic by adjusting the set of transmitted chunks based on real-time cache state at the receiving end. The system dynamically determines which chunks need transmission by comparing against the receiver's cache, optimizing transmission time while ensuring all necessary data is delivered.
Data Source
AI summary
A system, computer-readable storage device and method relate to a local stream storage device of a local proxy that caches streams of data transmitted to a remote proxy. The method includes caching, in the local stream storage device4 of the local proxy, a stream of data transmitted over a wide area network to the remote proxy, wherein the stream is stored in a continuous manner, to yield a cached stream. The method includes comparing a flow of data received from a device with the cached stream to determine whether a portion of the flow of data has been previously transmitted to the remote proxy and, when the portion of the flow has been previously transmitted to the remote proxy, transmitting a message to the remote proxy to indicate that the portion of the flow of data has been transmitted previously to the remote proxy.