Sorted Data Compression Using Delta and Binary Event Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing methods for compressing sorted data representations, particularly in the Burrows-Wheeler compression process, face challenges with high computational complexity and inefficiency, especially when dealing with low-redundancy data, and lack effective algorithms for direct probabilistic modeling.
Innovation Solution
A lossless compression method that processes sorted data representations symbol-by-symbol, utilizing binary events and delta processing, with optional arithmetic coding and run-length encoding, to achieve a balance between efficiency and speed, while reducing computational complexity.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Productivity
If dynamic symbol ranking methods are used for compression, then compression efficiency is improved, but computational complexity increases
Solution Approach 1:
The patent segments the compression process into distinct phases: transform stage (Burrows-Wheeler or Sort Transform) and encoding stage. By separating these stages and using specific ranking methods (MTF, IF, DC, TS, WFC, QLFC) in the encoding stage, the patent achieves efficient compression while managing computational complexity through modular design.
Solution Approach 2:
The patent changes the parameter representation from direct symbol encoding to rank encoding. Instead of encoding symbols directly, the system encodes ranks of symbols after transformation, which improves compression efficiency. Various ranking methods provide different parameter transformation strategies to balance efficiency and complexity.
2Productivity
If complicated adaptive probabilistic modeling is used, then compression efficiency is improved, but speed decreases
Solution Approach 1:
The patent employs dynamic symbol ranking where ranks are updated based on recently processed symbols. This dynamic approach adapts to the data characteristics during compression, improving efficiency without requiring overly complex probabilistic models. The dynamic nature allows the system to respond to patterns in the data while maintaining reasonable processing speed.
3Device complexity
If run-length encoding is used to supplement rank encoding, then computational complexity is reduced, but compression efficiency may be limited for certain data types
Solution Approach 1:
The patent merges rank encoding with run-length encoding in a hybrid approach. After transforming data and assigning ranks, the system applies run-length encoding to handle sequences of repeating symbols. This combination leverages the strengths of both methods: rank encoding captures symbolic patterns while run-length encoding efficiently compresses repetitions, achieving a balance between complexity and efficiency.
Data Source
AI summary
A method of compressing sorted data representation symbols sequentially. Steps include determining whether a symbol currently being encoded is identical to an immediately preceding symbol; encoding the result as a binary event; if the symbols are identical, encoding the symbol ends; if the symbols are different and there is a constant difference between several previous pairs of symbols, determining whether the difference between the current symbol a and the symbol immediately preceding it b is the same as the difference between the symbol b and the symbol immediately preceding it c, and encoding the determination result as another binary event; in the case of positive result of the latest determination, encoding the symbol ends; otherwise, encoding the symbol using a bitwise operation in which bits of the symbol's binary representation are processed sequentially from the most significant bit to the least significant bit. A corresponding decompression method is provided.