Tabular Numeric Compression Using Base-14 Delta Encoding
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data compression systems, such as DEFLATE, ZIP, and RAR, are inadequate for efficiently compressing tabular numeric data, resulting in insufficient compression ratios and increased costs for storage and processing.
Innovation Solution
A system and method for lossless compression of tabular numeric data that converts tables of integers into base-14 strings, rewrites rows based on differences between values, and concatenates these strings into a byte array for efficient storage, utilizing data compression servers, storage databases, and decompression servers connected via a communication network.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Quantity of substance
If existing compression systems (DEFLATE, ZIP, RAR) are used on tabular numeric data, then general compression functionality is provided, but compression ratios are insufficient
Solution Approach 1:
The patent converts numeric data from standard base-10 representation to base-14 representation, changing the numerical parameter system to achieve more efficient encoding. This parameter change allows 4 numeric values to be encoded in 3 bytes, achieving superior compression ratios for tabular numeric data compared to general-purpose compression algorithms
Solution Approach 2:
The patent segments the compression process into distinct stages: converting numeric values to base-14 strings, encoding signs separately, packing multiple values into fixed-width fields, and organizing data into structured records. This segmentation allows optimization of each stage specifically for numeric tabular data characteristics
2Ease of operation
If data is stored in tabular format, then data organization and accessibility are maintained, but compression efficiency is reduced
Solution Approach 1:
The patent implements a nested structure where base-14 encoded values are packed within fixed-width fields, which are then organized into records with metadata headers, which themselves are organized into larger data structures. This nested organization maintains tabular data accessibility while achieving compression through efficient packing at each level
Solution Approach 2:
The patent transforms the traditional row-column tabular structure into a compressed representation that adds dimensional organization through record headers, field delimiters, and base-14 encoding layers. This dimensional transformation enables both compression and efficient data retrieval
3Quantity of substance
If compression is applied to reduce storage costs, then storage efficiency improves, but processing time and computational resources increase
Solution Approach 1:
The patent performs preliminary conversion of numeric data to base-14 representation and structures it into compressed format during the compression phase. This preliminary action organizes the data in a way that minimizes processing overhead during decompression, as the structured format with fixed-width fields and metadata enables direct parsing without complex analysis
Data Source
AI summary
Systems and methods for lossless compression of tabular numeric data are provided. The system can include one or more data compression servers executing data compression system code to compress the tabular numeric data, a storage database to store the compressed tabular numeric data, and one or more data decompression servers to decompress the tabular numeric data for use. The one or more data compression servers, the storage database, and the one or more data decompression servers can communicate via a communication network. The system can receive the uncompressed tabular numeric data from one or more data generation systems, processes the uncompressed tabular numeric data with the data compression system code, and generate a compressed table of numeric information, which can be stored in the database, or later decompressed by the one or more data decompression servers.