Delta-Encoded Entropy Compression for Multi-Pass Data Compaction
Find Innovative SolutionsGenerate Solutions
Solution Overview
Problem
Current data storage technologies face limitations in capacity and efficiency, particularly with the exponential growth of data demand exceeding storage capacity, and existing entropy encoding methods are inefficient for certain data types and characteristics, leading to bottlenecks in data transmission and security concerns.
Innovation Solution
The application of delta encoding to entropy encoding methods reduces the number of sourceblocks in a codebook to commonly encountered data and uses delta encoding of bit differences, improving data compaction by representing data as reference codes, thereby enhancing storage capacity and security.
Engineering Contradictions & Design Principles
Engineering Contradiction Analysis
1Loss of substance
If traditional entropy encoding methods are used, then data can be compacted, but the compression efficiency is insufficient for certain data types and characteristics
Solution Approach 1:
The patent applies delta encoding to transform the data representation parameters. Instead of encoding absolute values, the system encodes differences (deltas) between consecutive values, which fundamentally changes the statistical properties of the data and makes it more suitable for entropy encoding, thereby improving compression efficiency for data with sequential patterns
Solution Approach 2:
The patent segments the data into sourceblocks and processes them through multiple encoding passes. The first pass identifies commonly occurring sourceblocks for direct encoding, while the second pass handles remaining data with delta encoding. This segmentation approach allows the system to optimize compression for different data patterns separately
2Quantity of substance
If physical storage capacity is increased to meet data storage demand, then storage capacity is improved, but the cost and feasibility are limited by manufacturing capacity
Solution Approach 1:
The patent extracts and removes redundant information from the data through compression. By eliminating unnecessary data representation and encoding only the essential information (using reference codes for common sourceblocks and delta values for variations), the system effectively increases storage capacity without requiring additional physical storage media
3Adaptability or versatility
If data is transmitted over networks, then data communication is enabled, but bandwidth requirements increase and transmission bottlenecks occur
Solution Approach 1:
The patent extracts and eliminates redundant data before transmission. By compressing data using delta encoding and reference codes, the system reduces the quantity of data that needs to be transmitted over the network, thereby decreasing bandwidth requirements and avoiding transmission bottlenecks while maintaining full data fidelity
4Reliability
If traditional encryption algorithms are used for data security, then data protection is provided, but computational overhead and complexity increase
Solution Approach 1:
The patent introduces a library of commonly occurring sourceblocks as an intermediary between the data and the encoding process. This library acts as a reference that simplifies the encoding of frequent patterns while maintaining security, as the library itself can be protected and is essential for accurate decoding, thereby providing security with reduced computational overhead
Data Source
AI summary
The inventor has conceived, and reduced to practice, a system and method for data compaction using that applies delta encoding methods to entropy encoding methods to improve data compaction of entropy encoding methods under certain conditions and when compacting data having certain characteristics. Delta encoding may be applied to entropy encoding methods to further compact data sets by reducing the number of sourceblocks included in a codebook to those most commonly encountered in data to be encoded and, where mismatches occur during encoding, using delta encoding of bit differences with existing sourceblocks in the codebook rather than adding new sourceblocks to the codebook.