Shared Dictionary De-Duplication for Faster WAN Data Transfer
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Modern computer network systems face delays and increased complexity due to bandwidth limitations, latency, and congestion, especially in wide area networks, leading to high response times and increased costs as data is transmitted across geographically dispersed systems.
Innovation Solution
The implementation of acceleration nodes with dictionaries that include data items and identifiers, allowing for efficient data transmission by providing data identifiers or items between nodes, synchronizing dictionaries across the network to reduce memory requirements, duplicated data, and network traffic, and enabling dynamic adaptation to network conditions.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Adaptability or versatility
If data is transmitted over wide area network connections in distributed systems, then data can be shared between geographically separated nodes, but response times increase due to bandwidth limitations, latency and congestion
Solution Approach 1:
The patent segments data transmission into two parts: transmitting only compact data identifiers ( pointers) across the network rather than entire data items, and storing actual data locally at each node. This segmentation reduces the amount of data traversing the network, thereby reducing latency and improving response times while maintaining data sharing capability.
Solution Approach 2:
The patent implements preliminary action by pre-establishing a shared dictionary of data identifiers at all nodes before data transmission occurs. This dictionary maps data identifiers to actual data items, allowing nodes to quickly resolve references without waiting for data to be transmitted across the network, thus reducing response times.
2Speed
If advanced communication schemes are used to improve data transmission efficiency, then data can be transmitted faster, but the amount of data stored at network nodes increases, increasing cost and complexity
Solution Approach 1:
The patent extracts only the essential identifying information (data identifiers) from the full data items and transmits these compact references across the network. The actual data items are stored locally at each node and retrieved only when needed. This extraction significantly reduces the amount of data that needs to be stored and transmitted, thereby reducing network node complexity and cost while maintaining fast transmission speeds.
3Measurement precision
If multiple dictionaries are maintained at different network nodes to support data identification, then data can be accurately identified and transmitted, but memory requirements and processor load increase
Solution Approach 1:
The patent merges the data identification function into a single shared dictionary that is distributed across all network nodes. Instead of each node maintaining its own separate dictionary, all nodes access the same unified dictionary structure, reducing the total memory requirements while maintaining accurate data identification through consistent identifier mapping.
Data Source
AI summary
This disclosure relates to synchronizing dictionaries of acceleration nodes in a computer network. For example, dictionaries of a plurality of acceleration nodes of a client-server network can be synchronized to each include one or more identical data items and data identifier pairs. Synchronization can include transmitting a particular data item, or a combination of a data item and an associated data identifier, to another acceleration node which includes it in its dictionary. A particular acceleration node can, instead of transmitting a data item, transmit an associated data identifier to another acceleration node. As all (or a subset) of the acceleration nodes can have an identical dictionary when employing the methods described herein, the particular acceleration node can use the same dictionary to communicate with all (or the subset of) other acceleration nodes of the computer network.