Data Compression Stream Layout for FPGA and ASIC Throughput
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data compression techniques, such as the LZ family, are not optimized for hardware-friendly implementations on Field Programmable Gate Arrays (FPGAs) or Application-Specific Integrated Circuits (ASICs), as they prioritize compression ratio over speed or vice versa, lacking efficiency in these specialized environments.
Innovation Solution
A lossless compression method that interleaves literal length fields with literal fields, match length fields, and repeat length fields to efficiently compress data by copying literal segments and replacing matched segments with references, using a hash table to track data snippets and determine match positions, thereby optimizing for hardware-friendly compression.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If LZ family compression techniques are used to prioritize compression ratio, then data compression efficiency is improved, but processing speed deteriorates
Solution Approach 1:
The patent segments the compressed data stream into distinct token types (literal tokens, match tokens, repeat tokens) with fixed-length encoding. This segmentation allows parallel processing of different token types in hardware, improving processing speed while maintaining compression ratio through efficient representation of different data patterns
2Loss of substance
If LZ family compression techniques are used to prioritize compression ratio, then data compression efficiency is improved, but device complexity increases
Solution Approach 1:
The patent changes the parameter representation by using fixed-length token encoding instead of variable-length dictionaries. This parameter change simplifies the hardware structure by eliminating complex dictionary management while maintaining compression efficiency through optimized token sequences representing literals, matches, and repeats
3Adaptability or versatility
If general-purpose CPU is used for data compression, then adaptability is improved, but processing speed deteriorates
Solution Approach 1:
The patent replaces the mechanical system of general-purpose CPU instruction execution with a dedicated hardware logic circuit that directly implements the compression algorithm. This substitution achieves FPGA/ASIC-level processing speeds while maintaining adaptability through configurable hardware design that can be deployed across different platforms
Data Source
AI summary
Systems, apparatus and methods are provided for compressing data. An exemplary method may comprise interleaving one or more literal length fields with one or more literal fields to an output. The literal fields may contain a first data segment literally copied to the output, and each of the one or more literal length fields may contain a value representing a length of a succeeding literal field. The method may further comprise determining a second data segment being matched to a previously literally copied sequence of data and a match position and writing to the output one or more match length fields and a match position field containing the match position. The literal length fields may contain a total length of the first data segment and the match length fields may contain a total length of the second data segment.