Parallel Packet Compression for Wire-Speed Network Links
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
High-speed network links pose a challenge for real-time data compression, as existing methods struggle to compress data efficiently at wire-speeds, leading to network congestion and resource inefficiencies.
Innovation Solution
A compression module that includes an input buffer, a history module, and a compression engine, which performs parallel comparisons of data packets with stored content to generate masks, select optimal masks, and create compression records, indicating whether packets are compressed, along with their size, address, and data, facilitating real-time compression and decompression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If data compression is performed on high-speed network links, then network resource utilization is improved, but real-time compression capability deteriorates due to processing time requirements
Solution Approach 1:
The compression process is divided into multiple parallel comparison operations, where each comparison unit independently compares a portion of the input data with the history buffer, generating multiple candidate masks simultaneously. This segmentation allows the system to process high-speed data streams in parallel, achieving real-time compression at wire-speed network links while maintaining high compression ratios.
Solution Approach 2:
The system performs multiple comparisons beyond what a single sequential comparison would provide, generating multiple candidate masks and selecting the best match. This excessive action (performing more comparisons than minimally required) enables the system to achieve both high compression efficiency and real-time performance by utilizing parallel processing capabilities.
2Productivity
If multiple comparisons are performed in parallel, then compression efficiency is improved, but system complexity increases due to multiple comparison units and mask management
Solution Approach 1:
Multiple comparison units and mask generation logic are merged into a single integrated compression module. The module combines parallel comparison operations, mask generation, mask comparison, and selection logic into one unified structure, reducing overall system complexity while maintaining high compression efficiency through parallel processing.
Solution Approach 2:
The compression module is designed as a universal component that performs multiple functions: data comparison, mask generation, mask comparison, and selection of optimal masks. This multi-functional design eliminates the need for separate dedicated components for each operation, simplifying the overall system architecture while achieving high compression efficiency.
3Measurement precision
If compression records include detailed mask information, then decompression accuracy is improved, but data packet size increases due to additional header fields
Solution Approach 1:
The system extracts only the essential mask information needed for accurate decompression, placing it in a compact header field. By taking out only the necessary compression record data (mask identifiers and relevant parameters) rather than storing complete mask details, the system maintains decompression accuracy while minimizing the increase in packet size.
Solution Approach 2:
The compression header includes partial mask information sufficient for accurate decompression without storing complete mask data. This partial inclusion of mask information strikes a balance between maintaining decompression precision and limiting the overhead added to compressed packet sizes.
Data Source
AI summary
Method and system for compressing a data packet is provided. The method includes receiving a data packet; comparing the data packet with content stored in a history module; wherein plural comparisons are performed in parallel; generating a plurality of masks based on the comparisons; comparing the plurality of masks; selecting one of the plurality of masks, based on the mask comparison; and generating a compression record, wherein the compression record includes; size of a data packet, an address field, a mask field and data; and a data packet header includes a control bit indicating if the data packet is compressed.