Bit Reordering Compression for Non-Geometric Integer Columns
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Existing data storage techniques, such as Golomb-Rice encoding, are suboptimal for datasets with non-geometrically distributed integers, leading to inefficient compression, as they are complex and CPU-intensive, and only effective on a narrow range of datasets.
Innovation Solution
A data store system that generates bit value representations, creates an index for each bit position, reorders bits according to a predetermined pattern, and encodes them using Golomb-Rice encoding, optimizing the compression process for non-geometrically distributed datasets.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If Golomb-Rice encoding is used to compress datasets with non-geometrically distributed integers, then compression is achieved, but the compression ratio is suboptimal
Solution Approach 1:
The patent segments the integer values into multiple bit positions and creates separate encoding streams for each bit position. This segmentation allows the encoding process to handle non-geometrically distributed data more effectively by treating each bit position independently, thereby improving compression ratios compared to traditional Golomb-Rice encoding applied to entire values.
Solution Approach 2:
The patent changes the encoding parameter from encoding entire integer values to encoding individual bit positions. By transforming the data representation from value-based to bit-position-based encoding, the system achieves better compression efficiency for non-geometrically distributed datasets while maintaining compatibility with Golomb-Rice encoding principles.
2Loss of substance
If Integer Wavelet Transform is used to transform datasets for Golomb-Rice coding, then compression is improved, but the transform is complex and CPU intensive
Solution Approach 1:
The patent replaces complex, CPU-intensive transforms like Integer Wavelet Transform with a simpler, more efficient bit reordering approach. This simpler method achieves comparable or better compression ratios without the high computational overhead, making it suitable for general-purpose use rather than being limited to specific dataset types.
Solution Approach 2:
The patent substitutes the mechanical complexity of wavelet transforms with a more elegant bit manipulation approach. By replacing the complex mathematical transformations with straightforward bit reordering operations, the system achieves similar compression results with significantly reduced computational requirements.
3Ease of operation
If traditional encoding techniques are used, then encoding is straightforward, but the number of bits required for encoding is large
Solution Approach 1:
The patent introduces a new dimension to the encoding process by reordering bits across multiple values rather than encoding each value independently in the traditional sequence. This dimensional reorganization of bit positions enables more efficient compression while maintaining encoding simplicity, achieving better bit utilization without complicating the encoding algorithm.
Data Source
AI summary
A data store system may include a storage device configured to store a plurality of data store tables. The data store system a further include a processor in communication with the storage device. The processor may receive a request to encode a column of a data store table from the plurality of data store tables. The processor may further generate a bit value representation of each value in the column of the data store table. The processor may further generate an index. The index may include an index value representative of each bit position of the bit value representations. The processor may further reorder bits of each bit value representation according to a predetermined pattern. The processor may further encode each reordered bit value representation according to an encoding technique. The processor may further store each encoded reordered bit value representations and the index. A method and computer-readable medium are also disclosed.