Compression History Matching for Shared Network Data Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods in communication networks face inefficiencies due to short matches between input streams and compression histories, synchronization challenges of shared data, and limitations in leveraging previously transmitted data for compression across multiple devices without explicit hierarchies.
Innovation Solution
Implementing algorithms to identify and utilize long matches in compression histories, synchronizing and sharing compression histories and indexes across devices, and removing application layer protocol headers to enhance compression efficiency and data sharing.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If compression histories are stored on disk with long access latencies, then data can be stored for future compression references, but compression performance deteriorates due to multiple separate disk accesses required for short matches
Solution Approach 1:
The compression history is segmented into fixed-size blocks that are pre-loaded into memory. This allows the system to access multiple potential match references simultaneously from memory rather than performing sequential disk accesses, resolving the contradiction between storing comprehensive compression history and maintaining fast compression performance.
Solution Approach 2:
Compression history blocks are pre-loaded into memory before the actual compression operation occurs. This preliminary action ensures that when compression is needed, the data is already available in fast memory rather than requiring slow disk access during the compression process, thus improving productivity while maintaining storage capacity.
2Productivity
If multiple processes access compression histories simultaneously, then multiple data streams can be compressed in parallel, but processing overhead increases due to access conflicts and coordination requirements
Solution Approach 1:
The compression history is divided into multiple blocks that can be independently accessed by different processes. This segmentation allows parallel compression operations to access different blocks simultaneously without conflicts, enabling high productivity while avoiding the complexity of coordinated access to shared resources.
Solution Approach 2:
Each process is assigned specific blocks of the compression history to use, creating localized access patterns. This local quality approach allows multiple processes to operate independently on their assigned blocks without requiring complex coordination mechanisms, thus maintaining parallel productivity while reducing access coordination complexity.
3Reliability
If compression algorithms perform thorough matching to ensure accuracy, then compression reliability improves, but processing time increases due to the complexity and number of iterations required
Solution Approach 1:
The system performs preliminary filtering by checking for exact block matches before applying more complex matching algorithms. This preliminary action quickly identifies obvious matches, ensuring reliability for those cases while avoiding the time-consuming complex algorithms, thus resolving the contradiction between match accuracy and processing time.
Solution Approach 2:
The system performs a sufficient number of matching iterations to achieve reliable compression for typical cases, rather than exhaustively checking every possible match. This partial action approach maintains adequate reliability while avoiding excessive processing time that would result from checking all possible matches in all scenarios.
4Stability of the object's composition
If compression histories are synchronized frequently to maintain consistency across devices, then data sharing accuracy improves, but network overhead increases due to transmission of synchronization information
Solution Approach 1:
Synchronization is performed at the block level rather than requiring full compression history exchange. This segmentation allows devices to synchronize only the specific blocks that have changed, maintaining compression history consistency across devices while significantly reducing network bandwidth consumption compared to full synchronization.
Solution Approach 2:
The system performs partial synchronization by transmitting only the necessary synchronization information for changed blocks rather than synchronizing the entire compression history. This partial action maintains adequate consistency for effective compression while avoiding the excessive network overhead of complete synchronization.
Data Source
AI summary
Systems and methods of storing previously transmitted data and using it to reduce bandwidth usage and accelerate future communications are described. By using algorithms to identify long compression history matches, a network device may improve compression efficiently and speed. A network device may also use application specific parsing to improve the length and number of compression history matches. Further, by sharing compression histories, compression history indexes and caches across multiple devices, devices can utilize data previously transmitted to other devices to compress network traffic. Any combination of the systems and methods may be used to efficiently find long matches to stored data, synchronize the storage of previously sent data, and share previously sent data among one or more other devices.