Hierarchical Compression for Concurrent Link and Higher-Layer Data Processing
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Conventional data de-duplication techniques cannot perform link-layer and higher-layer compression concurrently, as modifying data during higher-layer compression prevents correlation with known data strings at the link-layer, limiting network efficiency and increasing loading on links.
Innovation Solution
Implementing hierarchical compression techniques that allow network nodes to recognize higher-layer compression symbols and adapt link-layer compression by techniques such as data chunking, using cyclic redundancy checks, and decompressing higher-layer symbols before link-layer compression, enabling simultaneous link-layer and higher-layer compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If higher-layer compression is performed on data streams, then compression efficiency is improved, but link-layer compression cannot be applied concurrently because the modified data prevents correlation with known data strings
Solution Approach 1:
The patent segments the data stream into multiple layers: higher-layer compression symbols are identified and extracted, leaving data chunks that can be processed by link-layer compression. This segmentation allows both compression layers to operate independently and concurrently on appropriate portions of the data stream.
Solution Approach 2:
The patent introduces an intermediary processing stage at the network node that receives higher-layer compressed data streams, identifies compression symbols, and prepares data chunks for link-layer compression. This intermediary layer enables the coordination between higher-layer and link-layer compression without direct interference.
2Loss of substance
If data is modified during higher-layer compression, then redundancy reduction is improved, but correlation with known data strings at link-layer is lost
Solution Approach 1:
The patent extracts higher-layer compression symbols from the data stream before applying link-layer compression. By taking out these symbols that have already undergone compression, the remaining data chunks retain their original characteristics and can be correlated with known data strings at the link-layer without interference from higher-layer modifications.
3Device complexity
If single-layer compression is used, then implementation complexity is reduced, but network efficiency and link loading are not optimized
Solution Approach 1:
The patent implements a universal compression system that can handle both higher-layer and link-layer compression concurrently. The network node is designed to perform multiple functions: receiving higher-layer compressed data, identifying compression symbols, preparing data chunks, and applying link-layer compression. This multi-functionality optimizes network efficiency without requiring separate dedicated systems for each compression layer.
Data Source
AI summary
Hierarchical compression includes the contemporaneous implementation of link-layer and higher-layer compression on data flowing over a link. Hierarchical compression can be achieved by configuring network nodes positioned at the link-layer to recognize higher-layer compression symbols embedded in incoming data streams, and to adapt link-layer compression to compensate for those higher-layer compression symbols. One technique for adapting link-layer compression is to perform data chunking in-between higher-layer compression symbols. This may reduce the likelihood that higher-layer compression symbols will interfere with the network nodes ability to identify redundant data chunks at the link-layer. Another technique for adapting link-layer compression is to define the HASH algorithm in such a way that the hash of a data string renders the same hash value as the hash of the higher layer compression symbol corresponding to the data string.