Streaming Data Compression Using History Buffer Sequence Matching
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques face challenges in efficiently compressing streaming data in real-time due to limited bandwidth and the need for immediate data rendering, which complicates pattern matching and sequence representation.
Innovation Solution
A compression module employs a history buffer and lookup table to quickly identify matching sequences in streaming data, replacing them with representations that describe location and length, allowing for efficient compression and decompression without loss of data, and dynamically tunes parameters based on available resources.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If data is compressed by examining the entire data set to provide representations of common sequences, then compression efficiency is improved, but real-time rendering capability deteriorates due to excessive analysis time
Solution Approach 1:
The patent segments the data processing into fixed-size blocks (e.g., 16x16 pixel blocks for video). Each block is processed independently and in parallel, allowing compression to occur on smaller units rather than the entire data set. This enables real-time processing while maintaining compression efficiency through block-level pattern matching and representation.
Solution Approach 2:
The patent performs preliminary actions by pre-defining a set of common sequence representations (templates) that can be applied to data blocks. Instead of analyzing entire data sets to discover patterns, the system prepares representation templates in advance that can be quickly matched against incoming data blocks, significantly reducing analysis time while maintaining compression effectiveness.
2Quantity of substance
If data is compressed to transfer more data over low-bandwidth networks, then network utility is improved, but device complexity increases due to compression and decompression operations
Solution Approach 1:
The patent uses copying by creating simplified representations (copies) of common data sequences instead of transmitting the actual data. These representations are lightweight references that point to previously transmitted or stored data patterns. The copying approach reduces the amount of data that needs to be transmitted over low-bandwidth networks while maintaining the ability to reconstruct the original data at the receiving end.
3Productivity
If streaming data is provided for real-time rendering, then user functionality is improved, but compression capability deteriorates due to limited data availability for comparison
Solution Approach 1:
The patent segments streaming data into fixed-size blocks that can be processed independently. This segmentation allows the compression system to work with manageable data units as they arrive in real-time, enabling compression without requiring access to the entire data stream. Each block can be compressed using the same algorithms and representations, maintaining consistent compression ratios while supporting real-time rendering requirements.
Data Source
AI summary
Compression of streaming data is described. In an implementation, a method includes compressing data for communication in a terminal services environment by finding an index in a lookup table that matches an initial sequence in data. The lookup table includes a plurality of entries. Each of the entries is discoverable utilizing a particular one of a plurality of the indices. Each entry references whether a corresponding index is located in a history buffer, and if so, further references one or more locations of the corresponding index in the history buffer. If the corresponding entry of the matching index references a plurality of locations, for each location, a sequence having the matching index is compared with a sequence in the data that includes the initial sequence. A matching sequence is derived from the comparison based on a length and the location of the sequence at each of the locations. The matching sequence is represented using a representation that includes the length and the location of the matching sequence in the history buffer.