Run-Length Base-Delta Encoding for High-Speed CPU-Accelerator Compression
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression and decompression technologies face challenges in achieving high-speed compression while maintaining reasonable compression ratios, especially when transferring data between a central processing unit (CPU) and an accelerator device, as they often require fixed-length data segments and specialized hardware, limiting configuration options and increasing costs.
Innovation Solution
The implementation of a run-length base-delta (RLBD) encoding scheme, which is software-based and operates with variable-length data segments, allowing for efficient compression and decompression speeds comparable to memory copy operations, using a compression-transmission-decompression pipeline (CTDP) system that breaks data into smaller chunks and employs a data segment directory for parallel decoding.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If hardware-based data compression and decompression is used, then compression speed is improved, but device cost and complexity increase due to specialized hardware requirements
Solution Approach 1:
The patent replaces hardware-based compression mechanisms with a software-based run-length base-delta encoding scheme that can be implemented using general-purpose processors. This substitution eliminates the need for specialized compression hardware while achieving high compression speeds through efficient algorithms and data structures.
Solution Approach 2:
The patent uses base-delta encoding where instead of storing actual values, it stores differences (deltas) from a base value. This copying approach with differential storage achieves high compression ratios and speeds using standard software operations on general-purpose hardware.
2Device complexity
If fixed-length data segments are used, then hardware implementation becomes simpler, but configuration flexibility and adaptability are reduced
Solution Approach 1:
The patent implements variable-length data segment processing where the encoding adapts dynamically to the actual data characteristics. The run-length encoding component adjusts segment lengths based on data patterns, and the base-delta encoding adapts to value ranges, providing both flexibility and efficiency without requiring fixed-length constraints.
Solution Approach 2:
The patent changes the parameter of data segment length from fixed to variable, allowing the compression algorithm to adapt to different data patterns and characteristics. This enables the system to handle diverse data types and compression scenarios with a single flexible implementation.
3Quantity of substance
If compression ratio is increased, then data storage efficiency improves, but compression and decompression time increase
Solution Approach 1:
The patent segments the data processing into two independent components: run-length encoding for handling repetitive patterns and base-delta encoding for handling value sequences. This segmentation allows each component to optimize for its specific data pattern, achieving high compression ratios without excessive processing time through specialized handling of different data characteristics.
Solution Approach 2:
The patent applies compression selectively based on data characteristics - using run-length encoding when repetition is detected and base-delta encoding when value sequences are present. This partial application of different encoding strategies achieves high overall compression efficiency while minimizing processing time by avoiding unnecessary compression operations on already-compressed or incompressible data.
Data Source
AI summary
A method, computer system, and a computer program product for high-speed data compression is provided. The present invention may include receiving an input stream. The present invention may include selecting a header based on the received input stream, wherein the header includes a base, a scheme and a delta count. The present invention may include determining whether there are any remaining values in an uncompressed input stream. The present invention may include reading a first next value from the input stream. The present invention may include determining whether the read first next value is representable with a current base scheme. The present invention may include calculating the delta count based on determining that the read first next value is representable with the current base scheme. The present invention may include writing the calculated delta count to the selected header. The present invention may include incrementing the written delta count.