Packet Data Compression Using Prefix Arrays for Better Prediction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression methods for packet data across communication networks are inefficient as they rely solely on data within each packet, limiting the degree of compression and network performance.
Innovation Solution
The system generates a sorted prefix array from previously transferred packet data to predict and compress subsequent data, using arithmetic encoding to replace literal data with rank values, allowing for increased compression and parallel processing across multiple CPUs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Speed
If data compression is performed using only data within each packet, then the compression process is simple and fast, but the degree of compression is limited and network performance does not improve significantly
Solution Approach 1:
The system performs preliminary actions by storing previously transferred packet data in a data storage device before compression occurs. The sorted prefix array is pre-built from this historical data, enabling the compression process to access prior context information without waiting for packets to arrive sequentially. This preliminary preparation allows compression to achieve higher degrees by leveraging pre-available historical data patterns.
Solution Approach 2:
The invention transitions from one-dimensional compression (processing only current packet data) to multi-dimensional compression by incorporating historical packet data from previous transmissions. The sorted prefix array structure creates an additional dimension of temporal context, allowing the system to analyze patterns across multiple packets and time periods, thereby achieving superior compression ratios.
2Productivity
If sorted prefix arrays and historical data are used to predict subsequent data, then compression efficiency increases significantly, but the system complexity and data storage requirements increase
Solution Approach 1:
The system segments the compression functionality into distinct modular components: a data storage device for historical packets, a sorted prefix array generation module, a prediction module, and a compression module. This segmentation allows each component to perform its specific function independently, making the overall complex system manageable and enabling parallel processing across multiple CPUs without interdependence.
Solution Approach 2:
The sorted prefix array serves as an intermediary data structure between the historical packet data and the compression process. Instead of directly comparing current packets with all historical data (which would be computationally expensive), the prefix array acts as a pre-processed index that mediates the search and prediction operations, reducing the complexity of the compression algorithm while maintaining high compression efficiency.
3Reliability
If packet data is divided into packets for transmission, then data can be transmitted in manageable units with error control, but the amount of data requiring transmission increases compared to compressed formats
Solution Approach 1:
The invention merges multiple packets' worth of data into a single compressed packet by leveraging patterns across historical packets. Instead of transmitting each packet separately with its own overhead, the system combines redundant information from multiple packets into one compressed unit, reducing the total quantity of data transmitted while maintaining the reliability benefits of packetized transmission through the compression framework.
Data Source
AI summary
A system, method, and computer program for compressing packet data is provided. In exemplary embodiments, one or more prefix arrays may be generated for retrieved data, and used as the basis for predicting subsequent data. The packet data may be compressed based, at least partially, on the predicted subsequent data. Accordingly, the compressed packet data may be transferred over a communication network.