Control-Vector Data Compression for Parallel Pipelined Decompression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression methods for seismic data processing face challenges in achieving high-speed parallelized compression and decompression, particularly due to the complexity of run-length encoding and variable rate schemes, which result in memory bandwidth limitations and reduced computational performance.
Innovation Solution
A method involving data compression by partitioning data into superblocks, quantizing to increase zero-value items, generating a presence vector, removing zero-value blocks, and compressing non-zero blocks using a fixed-rate scheme, along with a separate control vector for decompression, enables efficient compression and decompression on hardware accelerators like FPGAs.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If variable rate compression schemes are used, then compression efficiency is improved, but device complexity and memory bandwidth requirements increase
Solution Approach 1:
The patent divides the data stream into fixed-size blocks and processes them independently. Each block is compressed using the same fixed-rate scheme, eliminating the complexity of variable rate decision-making while maintaining efficient compression through block-level parallelization.
Solution Approach 2:
The patent changes the compression approach from variable rate (adapting compression strength based on data characteristics) to fixed rate (consistent compression strength). This parameter change simplifies the compression scheme while achieving efficient compression through fixed-block processing and hardware optimization.
2Productivity
If parallelized decompression is implemented, then processing speed is improved, but memory bandwidth limitations worsen
Solution Approach 1:
The patent performs preliminary organization of compressed data into fixed-block structures with associated control information during the compression phase. This preliminary action enables the decompression hardware to process multiple blocks in parallel without requiring complex real-time memory arbitration, thus improving speed while managing bandwidth requirements.
Solution Approach 2:
The patent introduces control vectors as intermediary data structures that manage the parallel decompression process. These control vectors act as mediators between the compressed data blocks and the decompression logic, enabling efficient parallel processing by pre-establishing the processing order and data dependencies without requiring high memory bandwidth during decompression.
3Device complexity
If fixed-block compression is used, then device complexity is reduced, but compression efficiency deteriorates
Solution Approach 1:
The patent segments data into fixed blocks and applies compression at the block level. This segmentation enables simple, repeatable compression logic that reduces device complexity while the block structure itself becomes the basis for achieving efficient compression through parallel processing and hardware optimization.
Solution Approach 2:
The patent uses identical compression logic copied across multiple fixed blocks. This copying approach simplifies the device design by repeating the same simple compression unit rather than implementing complex variable-rate decision logic, while achieving good compression ratios through the volume of parallel processing.
4Productivity
If large buffers are used to sustain high throughput, then productivity is improved, but device complexity and memory requirements increase
Solution Approach 1:
The patent implements periodic processing of fixed-size data blocks through the compression and decompression pipeline. This periodic action with fixed block boundaries enables high throughput through parallel processing while minimizing buffer requirements, as data flows in regular, predictable chunks rather than requiring large buffers to handle variable-length streams.
Solution Approach 2:
The patent performs preliminary organization of data into fixed blocks with associated control information before processing. This preliminary structuring enables high throughput by allowing parallel processing units to operate independently on pre-prepared blocks, eliminating the need for large buffers to sustain throughput during processing.
Data Source
AI summary
A method of data compression includes obtaining a data set comprising a sequence of data blocks comprising a predetermined number of data items, partitioning said data set into one or more groups each comprising a predetermined number of data blocks, and performing data compression on one or more groups of data blocks. Data compression is performed by associating a control data item with each of said blocks, generating a control vector comprising the control data items assigned to each of said blocks within a group, removing data blocks comprising entirely data items having said specified value, compressing data blocks comprising at least one data item having a value different from said specified value using a fixed-rate compression scheme, providing a compressed data stream comprising said compressed data blocks, and providing an associated control vector stream to enable control of said compressed data stream.