Multi-Stream Data Compression for Out-of-Order Priority Traffic
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current lossless data compression techniques fail to efficiently compress internet traffic classified into multiple data streams at different priority levels, as they require packets to be in order and cannot exploit inter-stream redundancies, leading to sub-optimal performance and inefficient memory utilization.
Innovation Solution
A method and apparatus for multi-stream data compression that allows for aggregate-level compression of packets across multiple streams, with a decompressor cache larger than the compressor cache, enabling out-of-order packet handling and exploiting inter-stream redundancies through a long-range decompressor and short-range compressor combination.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If traditional lossless data compression techniques are applied to multi-stream traffic, then data integrity is maintained, but compression efficiency deteriorates due to inability to exploit inter-stream redundancies and requirement for in-order packet processing
Solution Approach 1:
The patent segments the packet processing function into two independent components: a compressor that operates on incoming packets in compression order, and a decompressor that handles outgoing packets in decompression order. This segmentation allows the system to process packets out-of-order while maintaining compression efficiency by exploiting inter-stream redundancies across multiple priority streams simultaneously.
Solution Approach 2:
The patent introduces an intermediary mechanism in the form of separate compression and decompression queues that decouple the compression process from the decompression process. This intermediary structure enables packets to be compressed in one order and decompressed in another order, allowing the system to exploit inter-stream redundancies without requiring packets to maintain their original transmission order.
2Reliability
If packets are processed in strict order to maintain compatibility with traditional compression algorithms, then decompression reliability is ensured, but link utilization deteriorates due to inability to handle out-of-order packets from multiple priority streams
Solution Approach 1:
The patent implements dynamic queue management where packets are routed to different queues based on their priority level and stream identification. The system dynamically adjusts the processing order of packets from different streams, allowing high-priority packets to be processed and transmitted before lower-priority packets, while maintaining decompression reliability through the intermediary queue structure that preserves the logical compression order.
3Quantity of substance
If a single shared memory pool is used for compression to improve memory utilization, then memory efficiency improves, but packet reordering issues arise when packets from different priority streams are compressed and transmitted simultaneously
Solution Approach 1:
The patent segments the memory pool into multiple independent compression contexts, one for each priority stream. Each stream maintains its own compression context within the shared memory pool, allowing the system to utilize the full memory capacity while preventing packet reordering issues. This segmentation enables packets from different priority streams to be processed independently without interfering with each other's compression state.
Data Source
AI summary
An approach for multi-stream data compression comprises receiving packets of a data stream, wherein the packets comprise respective packets of source data streams compressed on an aggregate basis and in a successive order. A one of the packets is decompressed, and a determination is made whether the packet has been received in a proper order of succession compared to the successive order of compression. When it is determined that the packet has been received in the proper order, the packet is stored at a next location in a decompressor cache. When it is determined that the packet has not been received in the proper order, the packet is stored at a location in the decompressor cache, allowing for subsequent storage of one or more further packets in the proper order of succession, wherein the further packets were processed via the compression process before, but were received after, the one packet.