Compression History Sharing for Long-Match Network Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression methods in networks face inefficiencies due to insufficiently long matches between input streams and compression histories, leading to poor compression ratios and increased processing overhead, especially when multiple data streams are transmitted simultaneously, and there is a need for improved synchronization and sharing of compression histories across devices.
Innovation Solution
The system uses algorithms to identify long matches in compression histories, shares compression history indexes and caches across devices, and employs application-specific parsing to enhance compression efficiency, synchronizing and leveraging previously transmitted data to reduce bandwidth usage and accelerate communications.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compression histories are stored on disk with long access latencies, then storage capacity is improved, but compression speed and processing efficiency deteriorate
Solution Approach 1:
The compression history is segmented into two storage locations: a disk-based compression history for long-term storage and a memory-based compression history for active compression operations. This segmentation allows the system to maintain large storage capacity while providing fast access for compression matching.
Solution Approach 2:
A memory-based compression history acts as an intermediary between the disk-based compression history and the compression algorithm. It pre-loads and caches compression data from disk, providing fast access to matching algorithms without requiring repeated disk accesses.
2Productivity
If multiple data streams are transmitted simultaneously, then network throughput is improved, but processing overhead and compression history access frequency increase
Solution Approach 1:
The memory-based compression history automatically manages itself by pre-loading data from disk and maintaining active compression histories for multiple data streams. This self-service mechanism reduces the processing overhead on the main compression algorithm, allowing multiple data streams to be handled efficiently.
3Productivity
If compression histories are synchronized between devices, then compression efficiency is improved, but network overhead and synchronization complexity increase
Solution Approach 1:
The patent extracts and transmits only the essential compression history data (fingerprints and location identifiers) between devices, rather than synchronizing entire compression histories. This reduction in transmitted data minimizes network overhead while maintaining compression efficiency.
Data Source
AI summary
Systems and methods of storing previously transmitted data and using it to reduce bandwidth usage and accelerate future communications are described. By using algorithms to identify long compression history matches, a network device may improve compression efficiently and speed. A network device may also use application specific parsing to improve the length and number of compression history matches. Further, by sharing compression histories, compression history indexes and caches across multiple devices, devices can utilize data previously transmitted to other devices to compress network traffic. Any combination of the systems and methods may be used to efficiently find long matches to stored data, synchronize the storage of previously sent data, and share previously sent data among one or more other devices.