Aggregate Multi-Stream Compression for Out-of-Order Packet Traffic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current lossless data compression techniques are inefficient when dealing with internet traffic classified into multiple data streams at different priority levels, as they require packets to be in order and cannot exploit inter-stream redundancies, leading to sub-optimal performance and increased memory requirements.
Innovation Solution
A multi-stream long-range data compression system that compresses data on an aggregate level, using a circular byte cache and a combination of long-range and short-range compressors to detect and encode duplicate data segments, allowing for out-of-order packet delivery and inter-stream redundancy exploitation.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Reliability
If traditional lossless data compression techniques are applied to multi-stream traffic, then packet order must be maintained, but this prevents exploitation of inter-stream redundancies and reduces compression performance
Solution Approach 1:
The patent segments the compression process by introducing sequence numbers and stream identifiers that allow the decompressor to reassemble packets in the correct order within each stream, while enabling the compressor to freely exploit redundancies across all streams without being constrained by packet delivery order
Solution Approach 2:
The patent introduces an intermediary mechanism (sequence numbers and stream tags) that mediates between the compressor's ability to exploit inter-stream redundancies and the decompressor's need to maintain packet order, allowing the compressor to operate on aggregate traffic while the decompressor reconstructs ordered streams
2Reliability
If compression is applied on a per-stream basis to maintain packet order, then memory pool must be sub-divided, but this reduces the available memory for compression and adversely impacts performance
Solution Approach 1:
The patent merges multiple stream-specific memory pools into a single aggregate memory pool that serves all streams, allowing the compressor to utilize the entire available memory for compression operations while the sequence number mechanism ensures correct packet reassembly at the decompressor
3Productivity
If compression is applied on aggregate traffic level, then inter-stream redundancies can be exploited, but packets may arrive out of order causing decompression failure
Solution Approach 1:
The patent applies preliminary action by embedding sequence numbers and stream identifiers in packets before compression, enabling the decompressor to correctly reassemble packets in order even when they arrive out of sequence, thus preventing decompression failure
Data Source
Figure 1
Figure 2
Figure 3
AI summary
An approach for multi-stream data compression comprises receiving packets of a data stream, wherein the packets comprise respective packets of source data streams compressed on an aggregate basis and in a successive order. A one of the packets is decompressed, and a determination is made whether the packet has been received in a proper order of succession compared to the successive order of compression. When it is determined that the packet has been received in the proper order, the packet is stored at a next location in a decompressor cache. When it is determined that the packet has not been received in the proper order, the packet is stored at a location in the decompressor cache, allowing for subsequent storage of one or more further packets in the proper order of succession, wherein the further packets were processed via the compression process before, but were received after, the one packet.