Block-Based Data Encoding for Parallel Compression and Random Access
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Cloud computing environments face challenges with existing compression algorithms like Lempel-Ziv-based zip compression, which have limitations such as fixed window size, inability to randomly access subsets of data, lack of parallel encoding/decoding, and inadequate support for modern data types like numbers and already-compressed data.
Innovation Solution
A massively parallel, block-based encoding and decoding technology that uses a plurality of processing cores to perform operations on encoded block formats with headers and payloads, enabling random access and supporting various data types through unique event parameters and tokens.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If Lempel-Ziv-based zip compression is used, then compression benefits are achieved, but the ability to encode in parallel is lost due to the inherently sequential nature of sliding-window compression
Solution Approach 1:
The patent divides the compressed data stream into independent fixed-size blocks, where each block can be encoded and decoded separately. This segmentation eliminates the sequential dependency of traditional sliding-window compression, enabling parallel processing across multiple blocks simultaneously while maintaining compression efficiency.
2Adaptability or versatility
If fixed window size compression is used, then implementation simplicity is maintained, but adaptability to multiple compression use cases is reduced
Solution Approach 1:
The patent introduces dynamic block size selection where the compression system can adjust the size of data blocks based on the specific compression use case and data characteristics. This dynamic adaptation allows the same compression algorithm to optimize performance for different scenarios (e.g., text, images, already-compressed data) without requiring multiple fixed-configuration systems.
3Adaptability or versatility
If sliding-window compression is used, then historical text compression is effective, but support for modern data types like numbers and already-compressed data is inadequate
Solution Approach 1:
The patent modifies the compression parameters and data representation methods to handle modern data types effectively. By changing how numerical data and already-compressed data are processed and encoded within the block-based framework, the system maintains high compression ratios for contemporary data formats while preserving the core compression algorithm's effectiveness.
4Quantity of substance
If sequential compression processing is used, then memory requirements are reduced, but storage and transfer costs increase due to larger data sizes
Solution Approach 1:
The patent segments data into fixed-size blocks that can be processed independently and concurrently. This segmentation enables parallel compression operations that significantly reduce total processing time while the efficient block-based encoding maintains high compression ratios, thereby reducing storage space requirements and associated costs.
Data Source
AI summary
Massively parallel, block-based encoding and decoding technology that includes an encoded block format uses a plurality of processing cores to perform block-based encoding and decoding operations. The encoded block format includes a header and a payload. The encoded block format's headers represent unique single-Byte and multi-Byte event parameters that occur in the original data block from which each encoded block was generated. The encoded block format's payloads represent a sequence of single-Byte and multi-Byte events using tokens that associate each event with its corresponding parameter(s). Metadata can include an array of encoded block sizes that support random access.