Hardware Data Compression With Interleaved Tokens for FPGA Decoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques, such as LZ77 and LZ78, are not optimized for hardware-friendly implementations on Field Programmable Gate Arrays (FPGA) or Application Specific Integrated Circuits (ASIC), as they prioritize compression ratio over speed or vice versa, lacking efficiency in these specific hardware environments.
Innovation Solution
A lossless compression method that interleaves literal length fields with literal fields, match length fields, and repeat length fields to efficiently compress data by copying literal segments and replacing matched segments with references, using a hash table to track data snippets and determine match positions, allowing for hardware-friendly encoding and decoding processes.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If traditional LZ77 or LZ78 compression schemes are used, then compression ratio may be improved, but processing speed and hardware compatibility deteriorate
Solution Approach 1:
The compressed data stream is segmented into distinct token types (literal tokens, match tokens, repeat tokens) with fixed-size fields. Each token has a predetermined structure with literal length, match length, and position fields, eliminating the need for complex variable-length decoding and enabling parallel hardware processing while maintaining high compression ratios through efficient token selection.
2Productivity
If compression algorithms are optimized for general-purpose CPUs, then compression performance improves, but hardware-friendly implementation on FPGA or ASIC deteriorates
Solution Approach 1:
The patent replaces complex software-based compression logic with hardware-optimized structures including fixed-size token fields, direct-mapped hash tables, and parallel processing units. The deterministic token structure allows for systematic hardware implementation using standard FPGA/ASIC design methodologies, substituting general-purpose CPU execution with dedicated hardware logic that achieves higher throughput and efficiency.
3Loss of substance
If variable-length fields are used in compressed data, then compression efficiency improves, but decoding complexity and hardware implementation difficulty increase
Solution Approach 1:
The patent changes the parameter structure from variable-length to fixed-size fields within each token. Literal length, match length, and position are encoded in predetermined bit widths (e.g., 8 bits each), creating a uniform token structure that simplifies hardware decoding to simple field extraction and concatenation operations, while maintaining compression efficiency through optimal field size selection based on data characteristics.
Data Source
AI summary
Systems, apparatus and methods are provided for compressing data. An exemplary method may comprise interleaving one or more literal length fields with one or more literal fields to an output. The literal fields may contain a first data segment literally copied to the output, and each of the one or more literal length fields may contain a value representing a length of a succeeding literal field. The method may further comprise determining a second data segment being matched to a previously literally copied sequence of data and a match position and writing to the output one or more match length fields and a match position field containing the match position. The literal length fields may contain a total length of the first data segment and the match length fields may contain a total length of the second data segment.